An image processing method and apparatus

Abstract

The application provides an image processing method and device, and a computer readable storage medium. The method comprises the following steps: performing feature point extraction on a to-be-processed image according to an initial threshold value to obtain a plurality of first feature points; determining an optimized threshold value according to the number of the first feature points, the number of target feature points required for processing the to-be-processed image, and the initial threshold value; performing feature point detection on each of the first feature points according to the optimized threshold value to screen a plurality of second feature points therefrom; and processing the to-be-processed image according to the second feature points of the to-be-processed image. Through the execution of these steps, the image processing method can optimize the feature extraction threshold value and the number of actually extracted feature points according to the number of target feature points required for image processing, thereby improving the accuracy and real-time performance of image processing.

Description

Technical Field

[0001] This invention relates to image processing technology, and more particularly to an image processing method, an image processing apparatus, and a computer-readable storage medium. Background Technology

[0002] ORB (Oriented Fast and Rotated Brief) feature extraction algorithm is a widely used image processing technique that can quickly extract key feature points in an image and identify target objects in the image based on these key feature points, thereby meeting various image processing needs such as image recognition, image matching, object localization / tracking, and scene reconstruction.

[0003] Existing ORB feature extraction algorithms rely on a pre-set threshold θ to extract a corresponding number N key feature points. However, in practical applications of ORB feature extraction on different images, the given threshold θ often differs significantly from the actual number of extracted feature points N due to various factors such as image sharpness, resolution, and content richness. This results in a significant discrepancy between the actual number of extracted feature points N and the target number of feature points N0 to be extracted. On the one hand, an excessively large number of feature points N increases the data processing load of the feature extraction algorithm and subsequent image processing steps, thereby increasing the time consumed in the image processing steps and affecting the real-time performance of the image processing results. On the other hand, an excessively small number of feature points N is prone to mismatches between feature points, which not only fails to meet the accuracy requirements of subsequent image processing steps but also necessitates the re-extraction of key feature points, similarly affecting the real-time performance of the image processing results.

[0004] In order to overcome the above-mentioned defects in the existing technology, there is an urgent need in the field for an image processing technology that can improve the accuracy and real-time performance of image processing by optimizing the feature extraction threshold θ and the actual number of feature points N extracted. Summary of the Invention

[0005] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed descriptions that follow.

[0006] In order to overcome the above-mentioned defects in the prior art, the present invention provides an image processing method, an image processing apparatus, and a computer-readable storage medium.

[0007] Specifically, the image processing method provided by the first aspect of the present invention includes the following steps: performing feature point extraction on the image to be processed according to an initial threshold to obtain multiple first feature points; determining an optimized threshold based on the number of first feature points, the number of target feature points required to process the image to be processed, and the initial threshold; performing feature point detection on each of the first feature points according to the optimized threshold to select multiple second feature points; and processing the image to be processed based on the second feature points of the image to be processed. By performing these steps, the image processing method can optimize the feature extraction threshold θ and the actual number of extracted feature points N according to the number of target feature points N0 required for image processing, thereby improving the accuracy and real-time performance of image processing.

[0008] Furthermore, the image processing apparatus provided in the second aspect of the present invention includes a memory processor. The processor is connected to the memory and configured to implement the image processing method provided in the first aspect of the present invention. By implementing this image processing method, the image processing apparatus can optimize the feature extraction threshold θ and the actual number of extracted feature points N based on the number of target feature points N0 required for image processing, thereby improving the accuracy and real-time performance of image processing.

[0009] Furthermore, the computer-readable storage medium provided in the third aspect of the present invention stores computer instructions thereon. When the computer instructions are executed by a processor, the image processing method provided in the first aspect of the present invention is implemented. By implementing this image processing method, the computer-readable storage medium can optimize the feature extraction threshold θ and the actual number of extracted feature points N according to the number of target feature points N0 required for image processing, thereby improving the accuracy and real-time performance of image processing. Attached Figure Description

[0010] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0011] Figure 1 A schematic diagram of the structure of an image processing apparatus provided according to some embodiments of the present invention is shown.

[0012] Figure 2 A schematic flowchart of an image processing method provided according to some embodiments of the present invention is shown.

[0013] Figure 3 A flowchart illustrating an image matching method provided according to some embodiments of the present invention is shown. Detailed Implementation

[0014] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention is presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a thorough understanding of the invention, many specific details will be included in the following description. The invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.

[0015] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0016] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0017] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0018] As mentioned above, existing ORB feature extraction algorithms rely on a pre-set threshold θ to extract a corresponding number N of key feature points. However, in practical applications of ORB feature extraction on different images, due to various factors such as image clarity, resolution, and content richness, there is often a significant difference between the given threshold θ and the actual number of extracted feature points N. This results in a significant difference between the actual number of extracted feature points N and the target number of feature points N0 to be extracted. On the one hand, an excessively large number of feature points N increases the data processing load of the feature extraction algorithm and subsequent image processing procedures, thereby increasing the time consumed by the image processing procedures and affecting the real-time performance of the image processing results. On the other hand, an excessively small number of feature points N is prone to mismatches between feature points, which not only fails to meet the accuracy requirements of subsequent image processing procedures but also triggers the need to re-extract key feature points, similarly affecting the real-time performance of the image processing results.

[0019] To overcome the aforementioned deficiencies in the prior art, the present invention provides an image processing method, an image processing apparatus, and a computer-readable storage medium, which can optimize the feature extraction threshold θ and the actual number of extracted feature points N based on the number of target feature points N0 required for image processing, thereby improving the accuracy and real-time performance of image processing.

[0020] In some non-limiting embodiments, the image processing method provided in the first aspect of the present invention can be implemented by the image processing apparatus provided in the second aspect of the present invention. This image processing apparatus can be configured in various electronic devices, such as vehicles, user terminals, cameras, radars, and servers, that involve various image processing needs, including image recognition, face recognition, object localization, object tracking, and 3D modeling, in the form of software programs and / or hardware devices.

[0021] Please refer to Figure 1 , Figure 1 A schematic diagram of the structure of an image processing apparatus provided according to some embodiments of the present invention is shown.

[0022] like Figure 1 As shown, in some embodiments, the image processing apparatus 10 is configured with a memory 11 and a processor 12. The memory 11 includes, but is not limited to, the computer-readable storage medium described in the third aspect of the present invention, on which computer instructions are stored. The processor 12 is connected to the memory 11 and is configured to execute the computer instructions stored in the memory 11 to implement the image processing method described in the first aspect of the present invention.

[0023] The working principle of the image processing apparatus 10 will be described below with reference to some embodiments of image processing methods. Those skilled in the art will understand that these embodiments of image processing methods are merely non-limiting implementations provided by the present invention, intended to clearly demonstrate the main concepts of the invention and provide specific solutions convenient for public implementation, rather than limiting all functions or operating methods of the image processing apparatus 10. Similarly, the image processing apparatus 10 is also only one non-limiting implementation provided by the present invention and does not constitute a limitation on the entities implementing the steps in these image processing methods.

[0024] Please refer to Figure 2 , Figure 2 A schematic flowchart of an image processing method provided according to some embodiments of the present invention is shown.

[0025] like Figure 2 As shown, in some embodiments of the present invention, during the image processing of the image F1 to be processed, the image processing device 10 may first use the minimum threshold θ1 of the FAST (Features From Accelerated Segment Test) of the ORB feature extraction algorithm as the initial threshold for feature extraction, and perform feature point extraction on the image F1 to be processed to obtain N1 first feature points.

[0026] Here, the FAST minimum threshold θ1 is the minimum universal threshold provided by the ORB feature extraction algorithm. It is less than the ideal threshold θ0 corresponding to the number of target feature points N0, and can extract the first feature point that is greater than the number of target feature points N0 (i.e., N1>N0). The specific process of determining the FAST minimum threshold θ1 and extracting the first feature point from the image F1 to be processed based on the FAST minimum threshold θ1 is existing technology in this field and will not be elaborated here.

[0027] After determining the number of first feature points N1 that can be extracted from the image F1 by the FAST minimum threshold θ1, the image processing device 10 can calculate and determine the optimized threshold θ for feature extraction based on the number of first feature points N1, the number of target feature points N0 required to process the image F1, and the initial threshold (i.e., the FAST minimum threshold θ1).

[0028] θ = θ1 * N1 / N0

[0029] By performing the above linear optimization on the feature extraction threshold θ, the optimized threshold θ can be made closer to the ideal threshold θ0 corresponding to the number of target feature points N0.

[0030] Furthermore, in some embodiments, after performing the above-described linear optimization and obtaining the optimization threshold θ, the image processing device 10 may also preferably determine whether the optimization threshold θ meets the preset feature extraction criteria based on the FAST default threshold θ2 of the ORB feature extraction algorithm and the preset scaling factor F, that is:

[0031]

[0032] In the formula, the default threshold θ2 of FAST is the default universal threshold provided by the ORB feature extraction algorithm, and the scaling factor F indicates the feature degree of the feature points to be extracted.

[0033] In some embodiments, technicians can freely set a scaling factor F to preliminarily check and correct unreasonable optimization thresholds θ based on the application scenario, accuracy requirements, and precision requirements of image processing. Specifically, if the optimization threshold θ is less than or equal to the standard threshold indicated by the feature extraction standard (i.e., θ²*F), the image processing device 10 can determine that the optimization threshold θ cannot meet the accuracy requirements of subsequent image processing, and thus replace the optimization threshold θ with the FAST default threshold θ² for subsequent feature point detection. Conversely, if the optimization threshold θ is greater than the standard threshold indicated by the feature extraction standard (i.e., θ²*F), the image processing device 10 can determine that the optimization threshold θ better meets the accuracy requirements of subsequent image processing than the FAST default threshold θ², and thus continue to use the optimization threshold θ for subsequent feature point detection.

[0034] By using the aforementioned FAST default threshold θ2 and scaling factor F to perform preliminary screening and correction of the optimized threshold θ, this invention can ensure that the optimized threshold θ for the feature point detection process is sufficient to meet the precision and accuracy requirements of image processing, thereby guaranteeing the accuracy of the image processing results and avoiding the need for re-detection of feature points due to errors in the processing results.

[0035] like Figure 2 As shown, after obtaining an optimized threshold θ that meets the precision and accuracy requirements of image processing, the image processing device 10 can perform feature point detection on each of the first feature points obtained by wide extraction according to the optimized threshold θ, so as to select N2 second feature points from them.

[0036] Specifically, the image processing device 10 can first divide the image F1 into grids of unequal area based on the position of each first feature point in the image F1 to be processed and a preset number range (e.g., 5 to 20), so that the number of first feature points n1 in each grid is within the preset number range (i.e., 5 to 20). Then, the image processing device 10 can determine whether the number of grids is greater than a preset number threshold (e.g., 500). If the number of grids is less than or equal to the preset number threshold, the image processing device 10 can sequentially detect the first feature points in each grid according to an optimization threshold θ, selecting n1 from each grid. 2i N2 second feature points are selected from the entire image F1 to be processed. Conversely, if the number of grids is greater than a preset threshold, the image processing device 10 can perform parallel feature point detection on the first feature points in each grid according to the optimized threshold θ, and select n second feature points from each grid respectively. 2i N2 second feature points are selected from the entire image F1 to be processed to obtain N2 second feature points.

[0037] Compared to directly extracting features from the image under processing using an optimized threshold θ, the scheme of detecting feature points based on the optimized threshold θ has a smaller data processing volume, thus improving the efficiency of the image processing method and the real-time performance of the image processing results. Furthermore, by employing an optimized scheme of grid division and parallel feature point detection, this invention combines the advantage of the small data processing volume of the feature point detection scheme with the full utilization of the data processing capabilities of the image processing device 10 to improve the efficiency of the image processing method and the real-time performance of the image processing results.

[0038] Furthermore, considering the nonlinear relationship between the given threshold and the actual number of extracted feature points, it can be determined that the optimized threshold θ obtained by linear optimization is still less than the ideal threshold θ0 (i.e., θ < θ0), and the number of second feature points N2 obtained by feature point detection through the optimized threshold θ is still greater than the target number of feature points N0. Therefore, in some preferred embodiments, the image processing device 10 can further homogenize and filter the N2 second feature points according to the position of each second feature point in the image to be processed F1 and the target number of feature points N0 to obtain third feature points that meet the target number of feature points N0.

[0039] For example, the image processing device 10 can first create quadtree nodes in the image F1 to be processed, and assign each second feature point to a corresponding quadtree node according to the coordinate position of each second feature point in the image F1. Then, the image processing device 10 can determine the number of feature points in each quadtree node one by one. If the number of feature points in a node is equal to 1, the image processing device 10 will retain this node and designate its feature points as third feature points. Conversely, if the number of feature points in a node is greater than 1, the image processing device 10 will further subdivide this node into four child nodes and further assign the feature points in this node to these four child nodes. And so on, the image processing device 10 can assign each second feature point to each level of child nodes in the multi-level quadtree node according to the coordinate position of each second feature point in the image F1 to be processed, until the number of child nodes reaches the target number of feature points N0. Subsequently, for child nodes where the number of feature points is still greater than 1, the image processing device 10 can filter out feature points with smaller response values ​​on each child node, and retain only the feature point with the largest response value on each child node as the third feature point.

[0040] Then, the image processing device 10 can perform various subsequent image processing procedures such as image matching and image recognition on the image to be processed F1 based on each third feature point in the image to be processed F1.

[0041] By performing the above-described homogenization filtering operation, the present invention can further filter out redundant feature points to improve the real-time performance of image processing results while meeting the requirements for image processing precision and accuracy. Furthermore, with a fixed number of feature points N0, the above-described homogenization filtering operation can effectively filter out clustered feature points while retaining dispersed feature points. This allows the image processing device 10 to identify objects in corresponding regions based on the third feature points dispersed at various locations in the image to be processed F1, thereby improving the accuracy and success rate of image processing results such as image recognition and image matching.

[0042] The following will describe the subsequent image processing flow of the present invention with reference to some embodiments of the image matching process. Those skilled in the art will understand that these embodiments of the image matching process are merely non-limiting implementations provided by the present invention, intended to clearly demonstrate the main concept of the invention and provide specific solutions convenient for public implementation, rather than being intended to limit the scope of protection of the present invention.

[0043] Please refer to further information. Figure 3 , Figure 3 A flowchart illustrating an image matching method provided according to some embodiments of the present invention is shown.

[0044] exist Figure 3In a specific application of the image matching process shown, the image processing device 10 can first divide the image F1 to be processed into layers, and obtain multiple first image layers F with progressively decreasing resolution from bottom to top by adjusting the filtering function (e.g., Gaussian filtering function) layer by layer. 1i This constructs an image pyramid of the image to be processed, F1. Then, the image processing device 10 determines the number N0 of target feature points to be extracted based on the specific requirements of the matching precision and / or matching accuracy of the image matching, and performs the steps of feature point extraction, threshold optimization, feature point detection, and homogenization screening as described above, thereby extracting the target feature points from each first image layer F of the image pyramid F1. 1i Extract N0 third feature points respectively.

[0045] Furthermore, the image processing apparatus 10 can also use the same method to divide the image F2 to be matched into layers, and obtain multiple second image layers F with progressively decreasing resolution from bottom to top by adjusting the filtering function (e.g., Gaussian filtering function) layer by layer. 2i This constructs an image pyramid of the image to be matched, F2. Then, given that the two matching images typically have similar sharpness, resolution, and content richness, the image processing device 10 can directly perform feature point extraction, threshold optimization, feature point detection, and homogenization filtering steps based on the number N0 of target feature points in the image F1, thereby extracting the image from each of the second image layers F of the image pyramid F2. 2i N0 third feature points are extracted from each of the second image layers F. Then, the image processing device 10 can process the data from each second image layer F... 2i The extracted N0 third feature points are respectively used as the corresponding second image layer F. 2i The feature points to be matched.

[0046] In determining the first image layers F of the image pyramid F1 1i The N0 third feature points are used to determine the second image layers F of the image F2 to be matched. 2i At least one feature point FPa to be matched ij Then, the image processing device 10 can calculate each first image layer F separately. 1i The third feature points are mapped to the corresponding second image layer F. 2i Each feature point to be matched, FPa ij The Euclidean distance is used to determine the FPa of each feature point to be matched. ij The nearest neighbor feature point FPb ij and the next nearest neighbor feature point FPc ij Then, the image processing device 10 can determine the nearest neighbor feature points FPb based on the data. ij To the corresponding feature point FPa to be matched ij The Euclidean distance and the next nearest neighbor feature points FPcij To the corresponding feature point FPa to be matched ij The Euclidean distance is used to perform feature matching between the image to be processed F1 and the image to be matched F2.

[0047] Specifically, in determining whether the image to be processed F1 matches the image to be matched F2, the image processing device 10 can first determine whether the image to be processed F1 matches the image to be matched F2 based on the nearest neighbor feature points FPb. ij To the corresponding feature point FPa to be matched ij The Euclidean distance and the second nearest neighbor feature point FPc ij To the corresponding feature point FPa to be matched ij Calculate the Euclidean distance between the nearest neighbor feature points FPb. ij With the nearest neighbor feature point FPc ij To the corresponding feature point FPa to be matched ij The Euclidean distance ratio, i.e.:

[0048]

[0049] If the Euclidean distance ratio R is calculated... ij If the ratio is less than a preset ratio threshold R0, the image processing device 10 can determine that it is located in the first image layer F of the image to be processed F1. 1i The nearest neighbor feature point FPb ij It is the second image layer F located in the image to be matched F2 2i The feature point FPa to be matched ij The optimal matching point. Conversely, if the Euclidean distance ratio R is calculated... ij If the ratio is greater than or equal to a preset ratio threshold R0, the image processing device 10 can determine the first image layer F of the image F1 to be processed. 1i There is no second image layer F2 to be matched. 2i The feature point FPa to be matched ij The best matching point.

[0050] Similarly, the image processing device 10 determines, as described above, the first image layer F of the image F1 to be processed. 1i Does a second image layer F2 exist above the image to be matched? 2i Each feature point to be matched, FPa ij The best matching point, to statistically analyze the second image layer F 2i Each feature point to be matched, FPa ij First image layer F 1i The number of optimal matching points. Then, the image processing device 10 can preferably employ the Random Sample Consensus (RANSAC) algorithm to calculate the number of optimal matching points for each feature point FPa to be matched.ij The best matching points are used for mismatch filtering to remove those that are incorrectly matched. Then, if the number of best matching points reaches a preset threshold (e.g., 91.15% of the number of feature points to be matched), the image processing device 10 can determine that the first image layer F... 1i With the second image layer F 2i Matching. Conversely, if the number of optimal matching points does not reach a preset threshold, the image processing device 10 can determine that the first image layer F... 1i With the second image layer F 2i Mismatch.

[0051] Furthermore, the image processing device 10 can also sequentially determine each of the first image layers F of the image F1 to be processed. 1i The corresponding second image layers F in the image to be matched F2 2i The image processing device 10 determines whether the two images match and counts the number of matching image layers in the image to be processed F1 and the image to be matched F2. If the number of matching image layers reaches a preset threshold, the image processing device 10 determines that the image to be processed F1 and the image to be matched F2 match. Conversely, if the number of matching image layers does not reach the preset threshold, the image processing device 10 determines that the image to be processed F1 and the image to be matched F2 do not match.

[0052] Those skilled in the art will understand that the above-described scheme of extracting a third feature point from the image F2 to be matched and using the extracted third feature point as the feature point to be matched is merely a preferred embodiment provided by the present invention. It is intended to clearly demonstrate the main concept of the present invention and to provide an image matching scheme with high precision, high accuracy and high success rate, rather than to limit the scope of protection of the present invention.

[0053] Alternatively, in other embodiments, the image processing device 10 may also use other means to determine the feature points to be matched in the image F2 to be matched, and determine whether the two images match based on the matching of the feature points to be matched with the third feature points of the image F1 to be processed.

[0054] Those skilled in the art will also understand that the above-described scheme for determining whether images match based on Euclidean distance ratio is merely a preferred embodiment of the present invention, intended to clearly demonstrate the main concept of the present invention and provide a specific solution that is easy for the public to implement, rather than intended to limit the scope of protection of the present invention.

[0055] like Figure 3As shown, in some embodiments of the present invention, after extracting N0 third feature points, the image processing apparatus 10 may further preferably calculate a descriptor for each third feature point in the image to be processed F1 to determine a second environmental feature for each third feature point. This second environmental feature describes the characteristics of other pixels surrounding each third feature point in the image to be processed F1.

[0056] Furthermore, the image processing apparatus 10 can also calculate a descriptor for at least one feature point in the image F2 to be matched, in order to determine a first environmental feature for each feature point to be matched. This first environmental feature describes the characteristics of other pixels surrounding each feature point in the image F2 to be matched.

[0057] Subsequently, the image processing device 10 can determine the similarity between each first environmental feature and each second environmental feature by calculating the cosine value of the first environmental feature of each feature point to be matched in the image to be matched F2 and the second environmental feature of each third feature point in the image to be processed F1, and then determine the best matching point for each feature point to be matched from each third feature point based on the similarity. After that, the image processing device 10 can, as described above, determine the optimal matching point for each feature point to be matched from each first image layer F1. 1i The number of optimal matching points in each first image layer F is determined. 1i With the corresponding second image layer F 2i The matching process is determined based on the number of matching image layers in the image to be matched (F2) and the image to be processed (F1). The specific process for determining whether each image layer and each image matches is similar to the above embodiment and will not be repeated here.

[0058] Those skilled in the art will understand that Figure 3 The image matching process shown is just one core step in various image processing applications and does not limit specific image processing applications. For example, a technician can use the image to be recognized as the image to be processed F1 in the above process, and multiple standard images in the image database as the images to be matched F2 in the above process. Then, the image matching process between the image to be recognized F1 and each standard image F2 is performed separately to achieve the effect of image recognition.

[0059] For example, technicians can use the face image to be identified as the image to be processed F1 in the above process, and use multiple standard face images from the face database as the images to be matched F2 in the above process. Then, they can perform image matching processes between the face image to be identified F1 and each standard face image F2 to achieve the effect of face recognition.

[0060] For example, technicians can use the next adjacent frame in the video as the image to be processed F1 in the above process, and the previous frame as the image to be matched F2 in the above process, and then perform the image matching process between the next frame F1 and the previous frame F2 to achieve the effects of object positioning / tracking and video anti-tampering.

[0061] Based on the above description, by employing the image processing method provided by this invention to optimize the feature extraction threshold θ and the actual number of extracted feature points N, this invention can extract key feature points that are closer to, or even equal to, the target feature point number N0, according to the accuracy, precision, and / or success rate requirements of subsequent image processing. This improves the accuracy of feature matching and reduces the false matching rate of feature points. Especially in image feature extraction applications under long-term, large-scene conditions, this invention can effectively improve the success rate of various image processing applications such as image recognition, face recognition, object localization, object tracking, and 3D modeling, thereby reducing the number of times feature points are repeatedly detected, thus solving the problems of long processing time and poor real-time performance of existing ORB feature extraction algorithms.

[0062] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0063] Those skilled in the art will understand that information, signals, and data can be represented using any of a variety of different techniques and arts. For example, the data, instructions, commands, information, signals, bits, symbols, and chips described throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0064] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0065] Although the image processing apparatus 10 described in the above embodiments can be implemented through a combination of software and hardware, it is understood that the image processing apparatus 10 can also be implemented in software or hardware alone. For hardware implementation, the image processing apparatus 10 can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for performing the above functions, or selected combinations of the above devices. For software implementation, the image processing apparatus 10 can be implemented in the form of independent software modules such as procedures and functions running on a general-purpose chip, each module performing one or more functions and operations described herein.

[0066] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0067] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that, Includes the following steps: Based on the initial threshold, feature points are extracted from the image to be processed to obtain multiple first feature points. The initial threshold is the FAST minimum threshold θ1 of the ORB feature extraction algorithm. Based on the number of the first feature points N1, the number of target feature points N0 required to process the image to be processed, and the initial threshold, an optimized threshold θ is determined, where θ = θ1 * N1 / N0; Feature point detection is performed on each of the first feature points according to the optimized threshold θ, so as to select multiple second feature points; as well as The image to be processed is processed based on the second feature point of the image to be processed.

2. The image processing method as described in claim 1, wherein, After determining the optimization threshold θ, the image processing method further includes the following steps: Determine whether the optimized threshold θ meets the preset feature extraction criteria; and In response to the determination result that the optimized threshold θ is less than or equal to the standard threshold indicated by the feature extraction criterion, the optimized threshold θ is replaced by the standard threshold; It also includes determining whether the optimized threshold θ satisfies the preset feature extraction criteria based on the default threshold θ2 of the ORB feature extraction algorithm's FAST and the preset scaling factor F, i.e.: Among them, the default threshold θ2 of FAST is the default universal threshold provided by the ORB feature extraction algorithm, and the scaling factor F indicates the feature degree of the feature points to be extracted. The standard threshold is θ2*F.

3. The image processing method as described in claim 1, wherein, The step of performing feature point detection on each of the first feature points according to the optimized threshold θ to select multiple second feature points includes: The image is divided into grids according to the positions of each first feature point, such that the number N1 of first feature points in each grid is within a preset range; and Parallel feature point detection is performed on the first feature points in each of the grids according to the optimized threshold θ, so as to filter out multiple second feature points.

4. The image processing method as described in claim 1, wherein, The step of processing the image based on the second feature point of the image to be processed includes: Based on the positions of the second feature points in the image to be processed and the number of target feature points N0, the plurality of second feature points are uniformly filtered to obtain the third feature points of the number of target feature points N0; and The image to be processed is processed based on the third feature point of the image to be processed.

5. The image processing method as described in claim 4, wherein, The step of processing the image based on the third feature point of the image to be processed includes: Determine at least one feature point to be matched from the image to be matched; Calculate the Euclidean distance between each of the third feature points in the image to be processed and the feature points to be matched, to determine their nearest neighbor and second nearest neighbor feature points; and Feature matching between the image to be matched and the image to be processed is performed based on the Euclidean distance from the nearest neighbor feature point to the feature point to be matched, and the Euclidean distance from the second nearest neighbor feature point to the feature point to be matched.

6. The image processing method as described in claim 5, wherein, The step of determining at least one feature point to be matched from the image to be matched includes: Extract the third feature point from the image to be matched, where the number of target feature points N0 is N0; and At least one of the third feature points of the image to be matched is determined.

7. The image processing method as described in claim 6, wherein, The step of performing feature matching between the image to be matched and the image to be processed based on the Euclidean distance from the nearest neighbor feature point to the feature point to be matched, and the Euclidean distance from the second nearest neighbor feature point to the feature point to be matched, includes: Based on the Euclidean distance from the nearest neighbor feature point to the feature point to be matched, and the Euclidean distance from the second nearest neighbor feature point to the feature point to be matched, calculate the ratio of the Euclidean distances from the nearest neighbor feature point to the feature point to be matched to that of the second nearest neighbor feature point; and In response to the Euclidean distance ratio being less than a preset ratio threshold, the nearest neighbor feature point in the image to be processed is determined to be the best matching point for the feature point to be matched in the image to be matched.

8. The image processing method as described in claim 7, wherein, The step of performing feature matching between the image to be matched and the image to be processed based on the Euclidean distance from the nearest neighbor feature point to the feature point to be matched, and the Euclidean distance from the second nearest neighbor feature point to the feature point to be matched, further includes: Based on the number of best matching points of each feature point in the image to be matched, it is determined whether the image to be processed matches the image to be matched.

9. The image processing method as described in claim 8, wherein, Before performing the step of determining whether the image to be processed matches the image to be matched based on the number of best matching points of each of the feature points to be matched in the image to be processed, the image processing method further includes the following steps: A random sampling consensus algorithm is used to filter out the best matching points of each feature point to be matched by mismatches.

10. The image processing method as described in claim 8, wherein, The step of determining whether the image to be processed matches the image to be matched based on the number of best matching points of each feature point of the image to be matched in the image to be processed includes: The number of feature points to be matched in multiple second image layers of the image to be matched is obtained, wherein the resolution of each second image layer is different; Determine the number of optimal matching points for each of the feature points to be matched in the corresponding second image layer of the multiple first image layers of the image to be processed, wherein the resolution of each first image layer is the same as the resolution of the corresponding second image layer; Based on the number of best matching points in each of the first image layers, determine whether each first image layer matches the corresponding second image layer; and The matching status of the image to be processed and the image to be matched is determined based on the number of matching image layers in the image to be matched and the image to be processed.

11. The image processing method as described in claim 4 or 7, wherein, The step of processing the image based on the third feature point of the image to be processed includes: Determine at least one feature point to be matched from the image to be matched; Calculate the descriptor of each of the feature points to be matched in the image to be matched, so as to determine the first environmental feature of each of the feature points to be matched; Calculate the descriptor of each of the third feature points in the image to be processed to determine the second environmental features of each of the third feature points; Based on the similarity between the first environmental features of each of the three feature points to be matched and the second environmental features of each of the three feature points, the optimal matching point for each of the three feature points to be matched is determined from each of the three feature points; and Based on the number of best matching points of each feature point in the image to be matched, it is determined whether the image to be processed matches the image to be matched.

12. An image processing apparatus, characterized in that, include: Memory; as well as A processor, connected to the memory, and configured to implement the image processing method as described in any one of claims 1 to 11.

13. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, the image processing method as described in any one of claims 1 to 11 is implemented.

Images