Feature point detection method, device, readable storage medium and computer program product

Abstract

The present application discloses a feature point detection method, device, readable storage medium and computer program product, which relate to the field of feature point detection technology, and solve the problem that it is difficult to stably and repeatedly detect the same feature point due to changes in viewing angles and environmental conditions. The method includes: determining the illumination evaluation value, contrast evaluation value, dynamic evaluation value and scene complexity evaluation value of the current image frame based on image information; using the Shi-Tomasi corner point detection algorithm to determine multiple initial feature points and feature information of each initial feature point, and then determining the feature point stability, feature point repeatability and feature point density as well as the response value, contrast and edge strength of each initial feature point; when it is detected that any of the above multiple index values ​​does not exceed the index threshold, using the feature point detection neural network model to determine multiple target feature points and the target position coordinates of each target feature point.

Description

Technical Field

[0001] The present application relates to the technical field of feature point detection, and in particular to a feature point detection method, device, readable storage medium and computer program product. Background Art

[0002] With the rise of short videos, video shooting has become increasingly important. At present, users generally use handheld devices to shoot videos, which can easily cause video shaking and lead to poor video quality. Video stabilization technology can reduce or eliminate the instability of video images caused by the movement or shaking of the photographic equipment. In video stabilization, feature point detection technology plays a key role because it determines whether the stabilization algorithm can accurately identify the relative motion between video frames. However, in the process of shooting videos, due to changes in viewing angles and environmental conditions, it is often difficult to stably and repeatedly detect the same feature points in continuous video frames, and the feature point detection accuracy is low. Summary of the invention

[0003] The embodiments of the present application provide a feature point detection method, device, readable storage medium and computer program product, the main purpose of which is to solve the current problem that due to changes in viewing angle and environmental conditions, it is often difficult to stably and repeatedly detect the same feature points in consecutive video frames, and the feature point detection accuracy is low.

[0004] According to the first aspect of the present application, a feature point detection method is provided, comprising:

[0005] Determining a current image frame, and acquiring image information of the current image frame;

[0006] Determine an illumination evaluation value, a contrast evaluation value, a dynamics evaluation value, and a scene complexity evaluation value of the current image frame based on the image information;

[0007] Performing feature point detection on the current image frame using the Shi-Tomasi corner point detection algorithm to determine a plurality of initial feature points and feature information of each initial feature point, and determining feature point stability, feature point repeatability and feature point density as well as a response value, contrast and edge strength of each initial feature point based on the feature information of each initial feature point and the image information;

[0008] When it is detected that the illumination evaluation value does not exceed the illumination evaluation threshold, or the contrast evaluation value does not exceed the contrast evaluation threshold, or the dynamics evaluation value does not exceed the dynamics evaluation threshold, or the scene complexity evaluation value does not exceed the scene complexity evaluation threshold, or the response value does not exceed the response value threshold, or the contrast does not exceed the contrast threshold, or the edge intensity does not exceed the edge intensity threshold, or the feature point stability does not exceed the stability threshold, or the feature point repeatability does not exceed the repeatability threshold, or the feature point density does not exceed the density threshold, perform feature point detection on the current image frame using a feature point detection neural network model to determine multiple target feature points and the target position coordinates of each target feature point, wherein the feature point detection neural network model is composed of a preprocessing module, an enhancement module, a detection module and a description module;

[0009] When it is detected that the illumination evaluation value exceeds the illumination evaluation threshold, the contrast evaluation value exceeds the contrast evaluation threshold, the dynamics evaluation value exceeds the dynamics evaluation threshold, the scene complexity evaluation value exceeds the scene complexity evaluation threshold, the response value exceeds the response value threshold, the contrast exceeds the contrast threshold, the edge intensity exceeds the edge intensity threshold, the feature point stability exceeds the stability threshold, the feature point repeatability exceeds the repeatability threshold, and the feature point density exceeds the density threshold, the multiple initial feature points are used as multiple target feature points, and the target position coordinates of each target feature point are extracted from the feature information of each target feature point.

[0010] According to the second aspect of the present application, a feature point detection device is provided, comprising:

[0011] An acquisition module, used to determine a current image frame and acquire image information of the current image frame;

[0012] A determination module, configured to determine an illumination evaluation value, a contrast evaluation value, a dynamics evaluation value, and a scene complexity evaluation value of the current image frame based on the image information;

[0013] A first detection module is used to perform feature point detection on the current image frame using a Shi-Tomasi corner point detection algorithm, determine a plurality of initial feature points and feature information of each initial feature point, and determine feature point stability, feature point repeatability and feature point density, as well as a response value, contrast, and edge strength of each initial feature point based on the feature information of each initial feature point and the image information;

[0014] A second detection module is used to perform feature point detection on the current image frame using a feature point detection neural network model to determine multiple target feature points and target position coordinates of each target feature point when it is detected that the illumination evaluation value does not exceed the illumination evaluation threshold, or the contrast evaluation value does not exceed the contrast evaluation threshold, or the dynamics evaluation value does not exceed the dynamics evaluation threshold, or the scene complexity evaluation value does not exceed the scene complexity evaluation threshold, or the response value does not exceed the response value threshold, or the contrast does not exceed the contrast threshold, or the edge intensity does not exceed the edge intensity threshold, or the feature point stability does not exceed the stability threshold, or the feature point repeatability does not exceed the repeatability threshold, or the feature point density does not exceed the density threshold, wherein the feature point detection neural network model is composed of a preprocessing module, an enhancement module, a detection module, and a description module;

[0015] The third detection module is used to take the multiple initial feature points as multiple target feature points and extract the target position coordinates of each target feature point from the feature information of each target feature point when it is detected that the illumination evaluation value exceeds the illumination evaluation threshold, the contrast evaluation value exceeds the contrast evaluation threshold, the dynamics evaluation value exceeds the dynamics evaluation threshold, the scene complexity evaluation value exceeds the scene complexity evaluation threshold, the response value exceeds the response value threshold, the contrast exceeds the contrast threshold, the edge intensity exceeds the edge intensity threshold, the feature point stability exceeds the stability threshold, the feature point repeatability exceeds the repeatability threshold, and the feature point density exceeds the density threshold.

[0016] According to a third aspect of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0017] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0018] By means of the above technical scheme, the present application provides a feature point detection method, device, readable storage medium and computer program product. The present application can determine the illumination evaluation value, contrast evaluation value, dynamic evaluation value and scene complexity evaluation value of the current image frame through the image information of the current image frame. At the same time, the Shi-Tomasi corner point detection algorithm is used to perform feature point detection on the current image frame to determine multiple initial feature points and feature information of each initial feature point, and determine the feature point stability, feature point repeatability and feature point density based on the feature information and image information of each initial feature point, as well as the response of each initial feature point. Value, contrast, edge strength, compare the index values ​​of these indicators with the corresponding index thresholds. If any of the index values ​​of these indicators does not exceed the corresponding index threshold, it means that the quality of the currently acquired image frame is poor, and a method with higher detection accuracy is needed for feature point detection, that is, the feature point detection neural network model is used to detect the feature points of the current image frame, determine multiple target feature points and the target position coordinates of each target feature point, and when the index values ​​of all the indicators in these index values ​​exceed the corresponding index thresholds, it means that the quality of the currently acquired image frame is good and no further feature point detection is required. The above steps can improve the ability to handle feature point detection tasks on images of poor quality, thereby improving the accuracy of feature point detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flow chart of a feature point detection method provided by an embodiment of the present application is shown;

[0020] Figure 2 A flowchart of another feature point detection method provided by an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram showing the structure of a feature point detection neural network model of another feature point detection method provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram showing the structure of a self-attention layer of another feature point detection method provided in an embodiment of the present application is shown;

[0023] Figure 5A A structural schematic diagram of a feature point detection device provided in an embodiment of the present application is shown;

[0024] Figure 5B A schematic diagram showing the structure of another feature point detection device provided in an embodiment of the present application is shown; DETAILED DESCRIPTION

[0025] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0026] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0027] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0028] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0029] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0030] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0031] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0032] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0033] The present application embodiment provides a feature point detection method, such as Figure 1 As shown, including:

[0034] 101. Determine a current image frame, and obtain image information of the current image frame.

[0035] 102. Determine an illumination evaluation value, a contrast evaluation value, a dynamics evaluation value, and a scene complexity evaluation value of a current image frame based on the image information.

[0036] 103. Perform feature point detection on the current image frame using the Shi-Tomasi corner point detection algorithm to determine multiple initial feature points and feature information of each initial feature point, and determine feature point stability, feature point repeatability and feature point density as well as response value, contrast and edge strength of each initial feature point based on the feature information and image information of each initial feature point.

[0037] 104. When it is detected that the illumination evaluation value does not exceed the illumination evaluation threshold, or the contrast evaluation value does not exceed the contrast evaluation threshold, or the dynamics evaluation value does not exceed the dynamics evaluation threshold, or the scene complexity evaluation value does not exceed the scene complexity evaluation threshold, or the response value does not exceed the response value threshold, or the contrast does not exceed the contrast threshold, or the edge intensity does not exceed the edge intensity threshold, or the feature point stability does not exceed the stability threshold, or the feature point repeatability does not exceed the repeatability threshold, or the feature point density does not exceed the density threshold, a feature point detection neural network model is used to perform feature point detection on the current image frame to determine multiple target feature points and the target position coordinates of each target feature point, wherein the feature point detection neural network model is composed of a preprocessing module, an enhancement module, a detection module and a description module.

[0038] 105. When it is detected that the illumination evaluation value exceeds the illumination evaluation threshold, the contrast evaluation value exceeds the contrast evaluation threshold, the dynamics evaluation value exceeds the dynamics evaluation threshold, the scene complexity evaluation value exceeds the scene complexity evaluation threshold, the response value exceeds the response value threshold, the contrast exceeds the contrast threshold, the edge intensity exceeds the edge intensity threshold, the feature point stability exceeds the stability threshold, the feature point repeatability exceeds the repeatability threshold, and the feature point density exceeds the density threshold, multiple initial feature points are used as multiple target feature points, and the target position coordinates of each target feature point are extracted from the feature information of each target feature point.

[0039] The method provided in the present application can determine the illumination evaluation value, contrast evaluation value, dynamic evaluation value and scene complexity evaluation value of the current image frame through the image information of the current image frame. At the same time, the Shi-Tomasi corner point detection algorithm is used to perform feature point detection on the current image frame to determine multiple initial feature points and the feature information of each initial feature point, and the feature point stability, feature point repeatability and feature point density as well as the response value, contrast and edge strength of each initial feature point are determined based on the feature information and image information of each initial feature point. The index values ​​of these indicators are compared with the corresponding index thresholds. If the index value of any indicator among the index values ​​of these indicators does not exceed the corresponding index threshold, it means that the quality of the currently acquired image frame is poor, and a method with higher detection accuracy is needed to perform feature point detection, that is, the feature point detection neural network model is used to perform feature point detection on the current image frame to determine multiple target feature points and the target position coordinates of each target feature point. When the index values ​​of all indicators among the index values ​​of these indicators exceed the corresponding index thresholds, it means that the quality of the currently acquired image frame is good and no further feature point detection is required. The above steps can improve the ability to handle feature point detection tasks on images of poor quality, thereby improving the accuracy of feature point detection.

[0040] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation of this embodiment, the embodiment of the present application provides another feature point detection method, such as Figure 2 As shown, including:

[0041] 201. Determine a current image frame, and obtain image information of the current image frame.

[0042] In the embodiment of the present application, the execution subject is a detection system that can execute the feature point detection method. The detection system first obtains the current video, where the current video is composed of multiple image frames with time information, and then extracts the color image frame of the latest acquisition time from the current video as the current initial image frame, and grayscales the current initial image frame to obtain the grayscale current image frame. By grayscale processing the color image frame, the complexity of subsequent calculations can be reduced.

[0043] It should be noted that the embodiment of the present application is described with one image frame. Since there will be many new image frames as the video is being shot, an image frame can be obtained at a preset time interval as the current image frame for feature point detection. It should be further noted that the current image frame here is a grayscale image frame.

[0044] 202. Determine an illumination evaluation value, a contrast evaluation value, a dynamics evaluation value, and a scene complexity evaluation value of a current image frame based on the image information.

[0045] In an embodiment of the present application, the environmental conditions of the current image frame are analyzed by determining the illumination evaluation value, contrast evaluation value, dynamics evaluation value and scene complexity evaluation value of the current image frame to determine whether it is suitable for feature point detection.

[0046] Furthermore, the specific steps of determining the illumination evaluation value, contrast evaluation value, dynamics evaluation value and scene complexity evaluation value of the current image frame based on the image information are as follows:

[0047] 2021. Extract the image size and the grayscale value of each pixel from the image information, and determine the illumination evaluation value of the current image frame using the first formula based on the image size and the grayscale value of each pixel. The first formula is:

[0048] ,

[0049] in, is the pixel point in the current image frame ( i , j ), the gray value of the current image frame M and high N Component image size, is the lighting evaluation value.

[0050] It should be noted that the overall brightness of the image can be evaluated through the illumination evaluation value to determine whether the illumination conditions are suitable for feature point detection; The value range is usually between 0-255, where 0 represents complete darkness and 255 represents maximum brightness. When the value is between 50 and 200, the feature point detection effect is the best.

[0051] 2022. Determine a contrast evaluation value of the current image frame using a second formula based on the image size and the grayscale value of each pixel. The second formula is:

[0052] ;

[0053] in, C is the contrast evaluation value.

[0054] It should be noted that the contrast evaluation value can be used to measure the global contrast of the image to evaluate the clarity of the image and the recognizability of feature points; C Essentially the standard deviation of the image, used as a measure of contrast, C The larger the value, the higher the contrast of the image, and the easier it is to detect and identify feature points. Generally, when C>30, the image can be considered to have good contrast.

[0055] 2023. Obtain a previous image frame adjacent to the current image frame, determine previous image information of the previous image frame, extract a target grayscale value of each target pixel from the previous image information, and determine a dynamic evaluation value of the current image frame using a third formula based on the image size, the grayscale value of each pixel, and the target grayscale value of each target pixel. The time interval between a previous sampling time corresponding to the previous image frame and a current sampling time corresponding to the current image frame is a preset time interval. The third formula is:

[0056] ;

[0057] in, D is the dynamic evaluation value, is the pixel point in the current image frame ( i , j ), is the pixel point in the previous image frame ( i , j )’s grayscale value.

[0058] It should be noted that the dynamic evaluation value can be used to analyze the motion pattern in the image sequence and determine the dynamic changes of the image; D The value reflects the degree of motion in the image. D A value of 0 indicates that the image has changed dramatically and the feature point detection strategy may need to be adjusted.

[0059] 2024. Extract the gradient magnitude of each pixel from the image information, and determine the scene complexity evaluation value of the current image frame using the fourth formula based on the image size and the gradient magnitude of each pixel. The fourth formula is:

[0060] .

[0061] in, is the scene complexity evaluation value, is the pixel point in the current image frame ( i , j )’s gradient magnitude.

[0062] It should be noted that the texture complexity and edge density of an image can be evaluated through the scene complexity evaluation value to adapt to different types of scenes; A higher value indicates a more complex scene, which may require more feature points to accurately describe the scene.

[0063] 203. Use the Shi-Tomasi corner point detection algorithm to perform feature point detection on the current image frame to determine multiple initial feature points and feature information of each initial feature point.

[0064] In an embodiment of the present application, before using the Shi-Tomasi corner detection algorithm to perform feature point detection, it is necessary to first determine the Shi-Tomasi corner detection algorithm, where the Shi-Tomasi corner detection algorithm is an optimal algorithm that has been trained. Since feature point detection is a crucial step in video stabilization, the present application preferably uses the Shi-Tomasi corner detection algorithm for feature point detection, which is a prior art that is easily accessible to those skilled in the art and is not described in detail here. By using the Shi-Tomasi corner detection algorithm to perform feature point detection on the current image frame, multiple initial feature points and feature information of each initial feature point can be determined.

[0065] In another optional implementation of the embodiment of the present application, the Harris corner detection algorithm can also be used to perform feature point detection. The Harris corner detection algorithm is based on the matrix of the local autocorrelation function of the image. The core idea of ​​this method is to determine whether this point is a corner point by calculating the sum of the intensity changes caused by the movement u and v of the pixel points in a small window around each pixel. The Harris corner detection algorithm is a prior art and can be easily obtained by those skilled in the art, so it will not be described in detail here.

[0066] 204. Determine feature point stability, feature point repeatability, and feature point density, as well as a response value, contrast, and edge strength of each initial feature point based on feature information and image information of each initial feature point.

[0067] In the embodiment of the present application, the specific process of determining the stability, repeatability and density of feature points and the response value, contrast and edge strength of each initial feature point based on the feature information and image information of each initial feature point is as follows:

[0068] 2041. Determine feature information of the initial feature point, extract a first eigenvalue and a second eigenvalue of the gradient covariance matrix of the initial feature point from the feature information, and determine a response value of the initial feature point using a Shi-Tomasi corner point response function based on the first eigenvalue and the second eigenvalue. The Shi-Tomasi corner point response function is:

[0069] ;

[0070] in, R is the response value of the initial feature point, is the first eigenvalue of the gradient covariance matrix of the initial feature point, is the second eigenvalue of the gradient covariance matrix of the initial feature point. R If it is greater than a preset threshold, the point is considered to be a corner point.

[0071] It should be noted that after determining the current image frame, the image information of the current image frame can be known. This image information includes each pixel point in the current image frame ( i , j ) horizontal gradient and vertical gradient , horizontal gradient and vertical gradient This can be achieved by using the Sobel operator, Prewitt operator or other gradient operators. For each pixel ( i , j ), construct the gradient covariance matrix M ,Right now,

[0072] ,

[0073] in,( u , v ) is the pixel ( i , j ), is a window function, usually a Gaussian weight, used to weight the pixels in the neighborhood.

[0074] The general form of the Gaussian window function is:

[0075] ,

[0076] in, is the standard deviation of the Gaussian function, which controls the spatial distribution range of the weight.

[0077] In practical applications, the Gaussian window is usually defined in a limited neighborhood, such as , or larger window size. This means that we usually only calculate weights within a certain range from the center pixel and restrict the window function to this neighborhood.

[0078] For example, for a The window with the center pixel coordinates is ( i , j ),So u and v Will be respectively arrive and arrive Gaussian weight Each pixel will be adjusted by a Gaussian function based on its distance from the center pixel.

[0079] For example, for The matrix M , its eigenvalue and It can be obtained by solving the following characteristic equation:

[0080] ,

[0081] The specific expansion is:

[0082] ,

[0083] Solving this quadratic equation, we get:

[0084] ,

[0085] in,

[0086] ,

[0087] ,

[0088] .

[0089] It should be noted that the reliability or significance of the corner points can be expressed by the response value; R The larger the value, the more significant the feature point. Usually, a threshold is set. ,only The points are considered as valid feature points.

[0090] 2042. Extract the pixel value of each local pixel point in the preset area around the initial feature point from the feature information, determine the pixel average value based on the pixel value of each local pixel point in the preset area around the initial feature point, and determine the contrast of the initial feature point using the fifth formula based on the pixel average value and the pixel value of each local pixel point. The fifth formula is:

[0091] ;

[0092] in, is the contrast of the initial feature points, The first k The pixel value of a pixel, In the preset area around the initial feature point n The average value of pixels.

[0093] It should be noted that contrast can be used to characterize the difference in pixel intensity between feature points and surrounding areas; It can measure the pixel difference between the feature point and the surrounding area, reflecting the clarity of the feature point. A higher contrast means that the difference between the feature point and the surrounding area is more significant. The larger the value, the higher the contrast between the feature point and the surrounding area, and the easier it is to track.

[0094] 2043. Extract the gradient amplitude of the initial feature point from the feature information, and determine the edge strength of the initial feature point based on the gradient amplitude using the sixth formula, where the sixth formula is:

[0095] ;

[0096] in, E is the edge strength of the initial feature point, is the initial feature point ( i , j )’s gradient magnitude.

[0097] In the embodiment of the present application, the image gradient strength at the location of the feature point is characterized by edge strength; E The gradient strength of the feature point can be measured to reflect the edge characteristics of the feature point. A higher edge strength means that the feature point is at a significant edge position in the image. E A larger value indicates that the feature point is located at a significant edge of the image, which usually has better stability.

[0098] 2044. Obtain the response value, contrast, and edge strength of each initial feature point.

[0099] In the embodiment of the present application, the response value, contrast, and edge strength of each initial feature point can be obtained through the above steps 2041 to 2043.

[0100] 2045. Extract the current position coordinates of each initial feature point from the feature information, obtain the previous image frame adjacent to the current image frame, and determine the adjacent position coordinates of each initial feature point based on the previous image frame, count the number of intersections of common feature points between the current image frame and the previous image frame, and determine the stability of the feature point based on the number of intersections, the current position coordinates of each initial feature point, and the adjacent position coordinates using the seventh formula, where the seventh formula is:

[0101] ,

[0102] in, S is the stability of feature points, The current image frame The current position coordinates of the initial feature points, is the first image frame in the previous image frame. The adjacent position coordinates of the initial feature points, N is the number of intersections.

[0103] It should be noted that the stability of feature points in consecutive frames can be measured by feature point stability; S The smaller the value, the more stable the feature point. Usually, a threshold is set. ,only The feature points are considered stable.

[0104] 2046. Determine a plurality of adjacent feature points and adjacent feature information of each adjacent feature point based on the previous image frame, and search for at least one initial feature point matching at least one adjacent feature point among a plurality of initial feature points based on the feature information and the adjacent feature information, and count the intermediate number of the at least one initial feature point. , count the number of previous targets of multiple adjacent feature points , calculate the intermediate quantity The previous target quantity The ratio of the characteristic point repeatability is obtained. .

[0105] It should be noted that the feature point repeatability can measure the proportion of feature points that appear repeatedly in two consecutive frames; The higher the value, the better the repeatability of the feature points under different video frame viewing angles, and the more suitable it is for video stabilization.

[0106] Furthermore, the repeatability of feature points is obtained The specific formula is:

[0107] .

[0108] 2047. Extracting image area from image information ,in, . Count the number of targets of multiple initial feature points in the current image frame , calculate the number of targets With the image area The ratio of the feature point density is obtained. D .

[0109] It should be noted that the density of feature points can be used to measure the distribution density of feature points in the image; D The value needs to be kept in an appropriate range. Too high a density may lead to increased computational burden, while too low a density may not adequately describe the scene.

[0110] 205. Obtain a video sample of the current shooting scene, and determine an illumination evaluation threshold, a contrast evaluation threshold, a dynamics evaluation threshold, a scene complexity evaluation threshold, a stability threshold, a repeatability threshold, a density threshold, a response value threshold, a contrast threshold, and an edge intensity threshold based on the video sample. The video sample does not include the current image frame.

[0111] In an embodiment of the present application, in the current shooting scene, video samples can be collected in advance for determining the threshold values ​​of various indicators. That is to say, the indicator threshold values ​​used for each comparison are dynamically determined according to the current shooting scene. Taking rainy days and sunny days at Square A as an example, when shooting at Square A, the light changes dramatically due to different weather. Therefore, the present application can automatically adjust the threshold values ​​of various indicators according to the characteristics of the video samples to further improve the accuracy of feature point detection.

[0112] Specifically, firstly, a plurality of initial real image samples can be determined based on the video sample, and the sample sequence number, the first target illumination evaluation value, the first target contrast evaluation value, the first target dynamics evaluation value, the first target scene complexity evaluation value, the first target feature point stability, the first target feature point repeatability, and the first target feature point density of each initial real image sample can be determined; the initial real image sample can be labeled using the sample sequence number, the first target illumination evaluation value, the first target contrast evaluation value, the first target dynamics evaluation value, the first target scene complexity evaluation value, the first target feature point stability, the first target feature point repeatability, and the first target feature point density. Obtain intermediate annotated real image samples, and determine the first target response value, the first target contrast, and the first target edge strength of each feature point in each initial real image sample, use the first target response value, the first target contrast, and the first target edge strength to annotate the corresponding feature points in each intermediate annotated real image sample, obtain a plurality of annotated real image samples, obtain a first real label for indicating image quality, and use the first real label to annotate each of the annotated real image samples to obtain a plurality of real image samples, wherein the first real label is 1 or 0, wherein 1 is used to indicate good image quality, and 0 is used to indicate poor image quality.

[0113] It should be noted that the sample number of each initial real image sample, the first target illumination evaluation value, the first target contrast evaluation value, the first target dynamics evaluation value, the first target scene complexity evaluation value, the first target feature point stability, the first target feature point repeatability, and the first target feature point density determination method refer to the specific methods of steps 201 to 204, which will not be repeated here.

[0114] Subsequently, the following operations are performed on each initial real image sample: noise processing and blur processing are performed on the initial real image sample to obtain an intermediate processed image sample, and the contrast of the intermediate processed image sample is adjusted to a target contrast to obtain an initial virtual image sample.

[0115] Next, determine the sample number, second target illumination evaluation value, second target contrast evaluation value, second target dynamics evaluation value, second target scene complexity evaluation value, second target feature point stability, second target feature point repeatability, and second target feature point density of the initial virtual image sample, use the sample number, second target illumination evaluation value, second target contrast evaluation value, second target dynamics evaluation value, second target scene complexity evaluation value, second target feature point stability, second target feature point repeatability, and second target feature point density to annotate the initial virtual image sample to obtain an intermediate annotated virtual image sample, and determine the second target response value, second target contrast, and second target edge strength of each feature point in each initial virtual image sample, use the second target response value, second target contrast, and second target edge strength to annotate the corresponding feature point in each intermediate annotated virtual image sample to obtain a plurality of annotated virtual image samples, obtain a second real label for indicating image quality, use the second real label to annotate each annotated virtual image sample to obtain a plurality of virtual image samples, and the second real label is 1 or 0, wherein 1 is used to indicate good image quality and 0 is used to indicate poor image quality;

[0116] It should be noted that the method for determining the sample number of the initial virtual image sample, the second target illumination evaluation value, the second target contrast evaluation value, the second target dynamics evaluation value, the second target scene complexity evaluation value, the second target feature point stability, the second target feature point repeatability, and the second target feature point density is also the specific method of referring to steps 201 to 204, which will not be repeated here.

[0117] Finally, based on multiple real image samples and multiple virtual image samples, the binarization algorithm is used to obtain the illumination evaluation threshold, contrast evaluation threshold, dynamics evaluation threshold, scene complexity evaluation threshold, stability threshold, repeatability threshold, density threshold, response value threshold, contrast threshold, and edge intensity threshold.

[0118] It should be noted that in order to quickly and adaptively select the optimal thresholds of each indicator so as to distinguish the labels of this set of data as accurately as possible, ROC curve and Youden index method, F1 score maximization method, minimum misclassification rate method, etc. can be used. This application gives priority to the ROC curve and Youden index method. ROC curve (Receiver Operating Characteristic curve) and Youden index are powerful tools for evaluating the performance of binary classification models, especially suitable for the threshold selection problem of known labels.

[0119] Specifically, the detailed implementation steps are:

[0120] 2051. Data preparation:

[0121] Determine a set of initial real image samples ,in, is the sample serial number, is the first true label, is the index value of the initial real image sample indicated by the sample sequence number of an index pair.

[0122] 2052. Sorting:

[0123] By indicator value Sort all samples from smallest to largest.

[0124] 2053. Calculate TPR and FPR:

[0125] For each unique indicator value As a potential threshold T :

[0126] a. Calculate True Positive Rate (TPR):

[0127] ,

[0128] b. Calculate the False Positive Rate (FPR):

[0129] ,

[0130] Among them, TP (True Positive): and Number of samples; FP (FalsePositive): and Number of samples; TN (True Negative): and Number of samples; FN (False Negative): and The number of samples.

[0131] 2054. Draw the ROC curve:

[0132] Draw a curve with FPR as the horizontal axis and TPR as the vertical axis.

[0133] 2055. Calculate the Youden index: The Youden index ranges from -1 to 1. A value of 1 indicates a perfect test (no false positives and false negatives). A value of 0 or lower indicates that the test is useless (no better than random guessing).

[0134] For each candidate threshold T, calculate the Youden index:

[0135] ,

[0136] 2056. Select the optimal threshold:

[0137] The optimal threshold is the threshold that maximizes the Youden index:

[0138] ,

[0139] It should be noted that by selecting the threshold that maximizes the Youden index, a cut-off point that achieves the best balance between the true positive rate and the false positive rate can be found. This provides a statistically reasonable decision boundary for binary classification problems, especially for situations where sensitivity and specificity need to be balanced.

[0140] 206. When it is detected that the illumination evaluation value does not exceed the illumination evaluation threshold, or the contrast evaluation value does not exceed the contrast evaluation threshold, or the dynamics evaluation value does not exceed the dynamics evaluation threshold, or the scene complexity evaluation value does not exceed the scene complexity evaluation threshold, or the response value does not exceed the response value threshold, or the contrast does not exceed the contrast threshold, or the edge intensity does not exceed the edge intensity threshold, or the feature point stability does not exceed the stability threshold, or the feature point repeatability does not exceed the repeatability threshold, or the feature point density does not exceed the density threshold, a feature point detection neural network model is used to perform feature point detection on the current image frame to determine multiple target feature points and the target position coordinates of each target feature point.

[0141] In the embodiment of the present application, if the index value of any indicator exceeds its corresponding index threshold, it means that the traditional Shi-Tomasi corner point detection algorithm cannot detect the feature points with high quality and high precision. At this time, the feature point detection neural network model can be used to perform feature point detection on the current image frame to determine multiple target feature points and the target position coordinates of each target feature point. The feature point detection neural network model is suitable for poor image conditions such as low light, low contrast, and blur. It is a robust feature enhancement network. The feature point detection neural network model consists of a preprocessing module, an enhancement module, a detection module, and a description module. The specific structure of the feature point detection neural network model is as follows: Figure 3 As shown by Figure 3It can be seen that the preprocessing module includes a preprocessing subnetwork, which includes an adaptive enhancement layer, a deblurring layer and a denoising layer connected in sequence. The enhancement module includes an enhanced shared encoder, which includes a first convolutional layer, a first LReLU layer, a first conditional attention layer, namely, conditional attention module 1, a first residual block layer, a first maximum pooling layer, a second convolutional layer, a second LReLU layer, a second conditional attention layer, namely, conditional attention module 2, a second residual block layer and a second maximum pooling layer connected in sequence. The detection module includes a detector head, which includes a third convolutional layer, a third LReLU layer, a hole convolution layer and a third convolutional layer connected in sequence. The description module includes a descriptor head, which includes a fourth convolutional layer, a fourth LReLU layer, a self-attention layer and a fourth convolutional layer connected in sequence.

[0142] Furthermore, after determining the feature point detection neural network, the current image frame can first be input into the adaptive enhancement layer, the deblurring layer and the denoising layer in the preprocessing subnetwork in the preprocessing module in sequence to obtain a preprocessed image frame with target image quality.

[0143] It should be noted that the preprocessing module can improve the quality of the input image and prepare for subsequent processing. The above-mentioned adaptive enhancement layer can use 3x3 convolution layer (16 filters) + Instance Normalization layer + PReLU layer to learn adaptive image enhancement; the deblurring layer can use 5x5 separable convolution (32 filters) + BatchNorm layer + ReLU layer to apply a learnable deblurring filter; the denoising layer can use 3x3 convolution (32 filters) + ResNet layer + ReLU layer to achieve adaptive denoising, and use a trainable convolution layer to implement a denoising algorithm to remove image noise while retaining detail information.

[0144] Next, the preprocessed image frame is sequentially input into the first convolutional layer, the first LReLU layer, the first conditional attention layer, the first residual block layer, the first maximum pooling layer, the second convolutional layer, the second LReLU layer, the second conditional attention layer, the second residual block layer and the second maximum pooling layer in the enhanced shared encoder in the enhancement module to obtain a target feature map with multi-scale information.

[0145] It should be noted that the enhancement module can extract multi-scale features from the preprocessed image. The first convolution layer can include a 7x7 convolution layer (64 filters) + a BatchNorm layer; the first LReLU layer is called Leaky ReLU, where α=0.1; the first conditional attention layer includes a feature conversion unit, an attention generation unit and a feature re-weighting unit. The feature conversion unit may include a 1x1 convolution layer (64 filters), the attention map generation unit may include a 3x3 convolution layer (1 filter) + Sigmoid, and the feature re-weighting unit may use element-wise multiplication; the first residual block layer includes two 3x3 convolution layers (64 filters) + BatchNorm layer + Leaky ReLU layer; the first maximum pooling layer may include a 2x2 convolution layer, and the step size is set to 2; the second convolution layer may include a 3x3 convolution layer (128 filters) + BatchNorm layer; the second conditional attention layer includes a feature conversion unit, an attention map generation unit and a feature re-weighting unit. The feature conversion unit may include a 1x1 convolution layer (128 filters), the attention map generation unit may include a 3x3 convolution (1 filter) + Sigmoid, and the feature re-weighting unit may use element-wise multiplication; the second residual block layer includes two 3x3 convolution layers (128 filters) + BatchNorm layer + Leaky ReLU layer; the second maximum pooling layer can use a 2x2 convolution layer with a step size of 2.

[0146] Subsequently, the target feature map with multiple scale information is sequentially input into the third convolutional layer, the third LReLU layer, the hole convolutional layer, and the third convolutional layer in the detector head to obtain the key point heat map.

[0147] It should be noted that the preprocessing module can generate a key point heat map to predict the location of key points in the image. The third convolution layer includes a 3x3 convolution layer (256 filters) + BatchNorm layer + Leaky ReLU layer; the dilated convolution layer includes a 3x3 dilated convolution layer (expansion rate = 2, 256 filters) + BatchNorm layer + Leaky ReLU layer; the third convolution layer includes a 1x1 convolution layer (65 filters, corresponding to 64 bins + 1 no-interest point).

[0148] Afterwards, the target feature map with multiple scale information is sequentially input into the fourth convolutional layer, the fourth LReLU layer, the self-attention layer, and the fourth convolutional layer in the descriptor head to obtain the position descriptor of each pixel in the key point heat map.

[0149] It should be noted that the description module can generate a key point heat map to predict the location of key points in the image. The fourth convolution layer includes a 3x3 convolution layer (256 filters) + BatchNorm layer + Leaky ReLU layer; Figure 4As shown in the figure, the self-attention layer includes an input conversion unit, a multi-head attention calculation unit, a head merging unit, an output projection unit, and a residual connection unit connected in sequence; the fourth convolutional layer includes a 1x1 convolutional layer (256 filters) + an L2 normalization layer, which makes it rotation and scale invariant. Furthermore, the input conversion unit in the self-attention layer can convert the size of The 256-channel feature map is converted into query (Q), key (K), and value (V) using a 1x1 convolutional layer. It should be noted that each of the converted Q, K, and V has 256 channels and the size is kept unchanged; the multi-head attention calculation unit includes 8 attention heads, each with 32 dimensions (256 / 8 = 32), and the following functions can be implemented for each head: a. Divide Q, K, and V into 32 channels respectively; b. Calculate the dot product of Q and K and divide it by sqrt(32) for scaling; c. Apply Softmax to get the attention weight; d. Multiply the attention weight with V; the head merging unit can connect the output of the 8 heads back to 256 channels; the output projection unit can use a 1x1 convolutional layer to map the merged features back to 256 channels; the residual connection unit can add the output to the original input to achieve residual connection.

[0150] Finally, multiple target feature points and the target position coordinates of each target feature point are determined based on the key point heat map and the position descriptor of each pixel point in the key point heat map.

[0151] It should be noted that the key point heat map is used to detect the position coordinates of the feature points. The key point heat map is of fixed size and is related to the current image frame size. It is 1 / 8 of the current image frame size, and each pixel position in the key point heat map represents the probability that the position is a key point. The descriptor is used to track and match the features of the feature points of the previous and next frames. The descriptor is of fixed dimension, usually 256 dimensions, and it generates a descriptor vector for each pixel position. Exemplarily, the original input image size of the current image frame is H×W, the key point heat map size is usually H / 8 × W / 8, and the descriptor output size is 256× H / 8× W / 8.

[0152] Specifically, the feature point detection neural network model can significantly improve the performance of the model under poor image conditions. For example, the preprocessing module helps to improve the input image quality, the conditional attention module enables the network to dynamically adjust the feature extraction strategy according to the image quality, the residual block layer and the LReLU layer help to alleviate the gradient vanishing problem, and the self-attention mechanism improves the expressiveness of the descriptor. After determining the key point heat map and the position descriptor of each pixel in the key point heat map, the key point heat map needs to be processed. First, the key point heat map is processed by non-maximum suppression, that is, a 3×3 or 5×5 sliding window is used on the heat map to retain the points whose window center value is greater than all surrounding pixels; then threshold screening is performed, and a threshold is set (for example, 0.015), and points whose heat map values ​​are greater than the threshold are retained; the coordinates are continued to be converted to the original image frame. For each retained point (hx, hy) on the heat map, where, , y=hy * 8+4, where +4 is to locate the center of the 8×8 area; then extract the descriptor, for each feature point (x, y), extract the 256-dimensional vector of the corresponding position in the descriptor output. It should be noted that bilinear interpolation can also be used to obtain more accurate descriptors; then integrate the information, collect the following information for each feature point: x, y coordinates in the current image frame (possibly sub-pixel accuracy), the corresponding 256-dimensional descriptor vector, and the confidence score extracted from the heat map, and integrate the above information; finally, get a list of feature points, each feature point contains: x, y coordinates in the current image frame, the corresponding descriptor, and an optional confidence score.

[0153] It should be noted that before using the feature point detection neural network, the feature point detection neural network needs to be trained. The specific steps are: obtaining the initial feature point detection neural network, and extracting multiple image frames from the high-definition video at a predetermined frame rate, then adding noise processing, blurring processing, adjusting contrast processing, random cropping processing, color transformation processing and rotating image processing to multiple image frames to obtain multiple processed image frames, determine the key points and corresponding descriptors in each image frame, and use the processed image frames and the corresponding key points and descriptors as training data. The initial feature point detection neural network is trained using the training data to obtain the optimal feature point detection neural network. If the model performance does not meet the expected standards, increase the number, adjust the training parameters or enhance the strategy according to the evaluation results, and then repeat the training process. Through this detailed training method, the neural network can show excellent performance and high adaptability when processing actual images, and is suitable for complex and dynamically changing visual environments.

[0154] 207. When it is detected that the illumination evaluation value exceeds the illumination evaluation threshold, the contrast evaluation value exceeds the contrast evaluation threshold, the dynamics evaluation value exceeds the dynamics evaluation threshold, the scene complexity evaluation value exceeds the scene complexity evaluation threshold, the response value exceeds the response value threshold, the contrast exceeds the contrast threshold, the edge intensity exceeds the edge intensity threshold, the feature point stability exceeds the stability threshold, the feature point repeatability exceeds the repeatability threshold, and the feature point density exceeds the density threshold, multiple initial feature points are used as multiple target feature points, and the target position coordinates of each target feature point are extracted from the feature information of each target feature point.

[0155] Further, as Figure 1 The specific implementation of the method is as follows: Figure 5A As shown, the embodiment of the present application provides a feature point detection device, such as Figure 5A As shown, it includes: an acquisition module 501, a determination module 502, a first detection module 503, a second detection module 504 and a third detection module 505.

[0156] The acquisition module 501 is used to determine the current image frame and acquire image information of the current image frame;

[0157] The determination module 502 is used to determine the illumination evaluation value, contrast evaluation value, dynamics evaluation value and scene complexity evaluation value of the current image frame based on the image information;

[0158] The first detection module 503 is used to perform feature point detection on the current image frame using the Shi-Tomasi corner point detection algorithm, determine multiple initial feature points and feature information of each initial feature point, and determine feature point stability, feature point repeatability and feature point density as well as a response value, contrast and edge strength of each initial feature point based on the feature information of each initial feature point and the image information;

[0159] The second detection module 504 is used to perform feature point detection on the current image frame using a feature point detection neural network model to determine multiple target feature points and target position coordinates of each target feature point when it is detected that the illumination evaluation value does not exceed the illumination evaluation threshold, or the contrast evaluation value does not exceed the contrast evaluation threshold, or the dynamics evaluation value does not exceed the dynamics evaluation threshold, or the scene complexity evaluation value does not exceed the scene complexity evaluation threshold, or the response value does not exceed the response value threshold, or the contrast does not exceed the contrast threshold, or the edge intensity does not exceed the edge intensity threshold, or the feature point stability does not exceed the stability threshold, or the feature point repeatability does not exceed the repeatability threshold, or the feature point density does not exceed the density threshold, wherein the feature point detection neural network model is composed of a preprocessing module, an enhancement module, a detection module, and a description module;

[0160] The third detection module 505 is used to use the multiple initial feature points as multiple target feature points and extract the target position coordinates of each target feature point from the feature information of each target feature point when it is detected that the illumination evaluation value exceeds the illumination evaluation threshold, the contrast evaluation value exceeds the contrast evaluation threshold, the dynamics evaluation value exceeds the dynamics evaluation threshold, the scene complexity evaluation value exceeds the scene complexity evaluation threshold, the response value exceeds the response value threshold, the contrast exceeds the contrast threshold, the edge intensity exceeds the edge intensity threshold, the feature point stability exceeds the stability threshold, the feature point repeatability exceeds the repeatability threshold, and the feature point density exceeds the density threshold.

[0161] In a specific application scenario, the acquisition module 501 is also used to acquire the current video, which is composed of multiple image frames with time information; extract the color image frame with the latest acquisition time from the current video as the current initial image frame, and grayscale the current initial image frame to obtain the grayscale current image frame.

[0162] In a specific application scenario, the determination module 502 is further used to extract the image size and the grayscale value of each pixel from the image information, and determine the illumination evaluation value of the current image frame based on the image size and the grayscale value of each pixel using a first formula, wherein the first formula is:

[0163] ,

[0164] in, is the pixel point in the current image frame ( i , j ), the gray value of the current image frame M and high N The image size is composed of is the light evaluation value;

[0165] The contrast evaluation value of the current image frame is determined using a second formula based on the image size and the grayscale value of each pixel, where the second formula is:

[0166] ;

[0167] in, C is the contrast evaluation value;

[0168] Acquire a previous image frame adjacent to the current image frame, determine previous image information of the previous image frame, extract a target grayscale value of each target pixel from the previous image information, and determine a dynamic evaluation value of the current image frame using a third formula based on the image size, the grayscale value of each pixel, and the target grayscale value of each target pixel, wherein the time interval between the previous sampling time corresponding to the previous image frame and the current sampling time corresponding to the current image frame is a preset time interval, and the third formula is:

[0169] ;

[0170] in, D is the dynamic evaluation value, is the pixel point in the current image frame ( i , j ), is the pixel point in the previous image frame ( i , j )’s gray value;

[0171] The gradient magnitude of each pixel is extracted from the image information, and the scene complexity evaluation value of the current image frame is determined by using a fourth formula based on the image size and the gradient magnitude of each pixel. The fourth formula is:

[0172] .

[0173] in, is the scene complexity evaluation value, is the pixel point in the current image frame ( i , j )’s gradient magnitude.

[0174] In a specific application scenario, the first detection module 503 is also used to determine the feature information of the initial feature point, extract the first eigenvalue and the second eigenvalue of the gradient covariance matrix of the initial feature point from the feature information, and determine the response value of the initial feature point based on the first eigenvalue and the second eigenvalue using the Shi-Tomasi corner point response function, wherein the Shi-Tomasi corner point response function is:

[0175] ;

[0176] in, R is the response value of the initial feature point, is the first eigenvalue of the gradient covariance matrix of the initial feature point, is the second eigenvalue of the gradient covariance matrix of the initial feature point;

[0177] The pixel value of each local pixel point in the preset area around the initial feature point is extracted from the feature information, and a pixel average value is determined based on the pixel value of each local pixel point in the preset area around the initial feature point. The contrast of the initial feature point is determined by using a fifth formula based on the pixel average value and the pixel value of each local pixel point. The fifth formula is:

[0178] ;

[0179] in, is the contrast of the initial feature point, The first k The pixel value of a pixel, The preset area around the initial feature point n The average value of pixels;

[0180] The gradient magnitude of the initial feature point is extracted from the feature information, and the edge strength of the initial feature point is determined based on the gradient magnitude using a sixth formula, where the sixth formula is:

[0181] ;

[0182] in, E is the edge strength of the initial feature point, is the initial feature point ( i , j )’s gradient amplitude;

[0183] Obtaining a response value, a contrast ratio, and an edge strength of each of the initial feature points;

[0184] The current position coordinates of each of the initial feature points are extracted from the feature information, a previous image frame adjacent to the current image frame is obtained, and the adjacent position coordinates of each of the initial feature points are determined based on the previous image frame, and the number of intersections of common feature points between the current image frame and the previous image frame is counted. Based on the number of intersections, the current position coordinates of each of the initial feature points, and the adjacent position coordinates, the stability of the feature point is determined by a seventh formula, and the seventh formula is:

[0185] ,

[0186] in, S is the stability of the feature points, is the first The current position coordinates of the initial feature points, is the first The adjacent position coordinates of the initial feature points,N is the number of intersections;

[0187] Determine a plurality of adjacent feature points and adjacent feature information of each of the adjacent feature points based on the previous image frame, and query at least one initial feature point matching at least one adjacent feature point among the plurality of initial feature points based on the feature information and the adjacent feature information, and count the intermediate number of the at least one initial feature point , count the previous target number of the multiple adjacent feature points , calculate the intermediate quantity The previous target quantity The ratio of the feature point repeatability is obtained. ;

[0188] Extracting the image area from the image information , count the number of targets of the multiple initial feature points in the current image frame , calculate the target quantity With the image area The ratio of the feature point density is obtained. D .

[0189] In specific application scenarios, such as Figure 5B As shown, the device also includes: a threshold determination module 506.

[0190] The threshold determination module 506 is used to obtain a video sample in the current shooting scene, and determine the illumination evaluation threshold, the contrast evaluation threshold, the dynamics evaluation threshold, the scene complexity evaluation threshold, the stability threshold, the repeatability threshold, the density threshold, the response value threshold, the contrast threshold, and the edge intensity threshold based on the video sample, wherein the video sample does not include the current image frame;

[0191] The threshold determination module 506 is further used to determine a plurality of initial real image samples based on the video sample, determine a sample sequence number, a first target illumination evaluation value, a first target contrast evaluation value, a first target dynamics evaluation value, a first target scene complexity evaluation value, a first target feature point stability, a first target feature point repeatability, and a first target feature point density of each of the initial real image samples, use the sample sequence number, the first target illumination evaluation value, the first target contrast evaluation value, the first target dynamics evaluation value, the first target scene complexity evaluation value, the first target feature point stability, the first target feature point repeatability, and the first target feature point density to annotate the initial real image samples to obtain intermediate annotated real image samples, and determine the sample sequence number, the first target illumination evaluation value, the first target contrast evaluation value, the first target dynamics evaluation value, the first target scene complexity evaluation value, the first target feature point stability, the first target feature point repeatability, and the first target feature point density of each of the initial real image samples. A first target response value, a first target contrast, and a first target edge strength are used to mark the corresponding feature points in each of the intermediate marked real image samples to obtain a plurality of marked real image samples, and a first real label for indicating image quality is obtained. The first real label is used to mark each of the marked real image samples to obtain a plurality of real image samples, and the first real label is 1 or 0, wherein 1 is used to indicate good image quality and 0 is used to indicate poor image quality; the following operations are performed on each of the initial real image samples: noise processing and blur processing are performed on the initial real image samples to obtain intermediate processed image samples, and the contrast of the intermediate processed image samples is adjusted to the target contrast to obtain initial virtual image samples;Determine the sample sequence number, the second target illumination evaluation value, the second target contrast evaluation value, the second target dynamics evaluation value, the second target scene complexity evaluation value, the second target feature point stability, the second target feature point repeatability, and the second target feature point density of the initial virtual image sample; use the sample sequence number, the second target illumination evaluation value, the second target contrast evaluation value, the second target dynamics evaluation value, the second target scene complexity evaluation value, the second target feature point stability, the second target feature point repeatability, and the second target feature point density to annotate the initial virtual image sample to obtain an intermediate annotated virtual image sample; and determine the second target response value, the second target contrast, and the second target edge strength of each feature point in each of the initial virtual image samples; use the second target response number, the second target contrast evaluation value, the second target dynamics evaluation value, the second target scene complexity evaluation value, the second target feature point stability, the second target feature point repeatability, and the second target feature point density to annotate the initial virtual image sample to obtain an intermediate annotated virtual image sample; and determine the second target response value, the second target contrast, and the second target edge strength of each feature point in each of the initial virtual image samples; and use the second target response number, the second target contrast evaluation value, the second target dynamics evaluation value, the second target scene complexity evaluation value, the second target feature point stability, the second target feature point repeatability, and the second target feature point density to annotate the initial virtual image sample to obtain an intermediate annotated virtual image sample; and use the second target response number, the second target contrast, and the second target edge strength of each feature point in the initial virtual image sample ... The corresponding feature points in each of the intermediate annotated virtual image samples are annotated with the response value, the second target contrast, and the second target edge strength to obtain multiple annotated virtual image samples, and a second real label for indicating image quality is obtained. The second real label is used to annotate each of the annotated virtual image samples to obtain multiple virtual image samples, and the second real label is 1 or 0, wherein 1 is used to indicate good image quality and 0 is used to indicate poor image quality; based on the multiple real image samples and the multiple virtual image samples, the illumination evaluation threshold, the contrast evaluation threshold, the dynamic evaluation threshold, the scene complexity evaluation threshold, the stability threshold, the repeatability threshold, the density threshold, the response value threshold, the contrast threshold, and the edge strength threshold are obtained by using a binarization algorithm. ;

[0192] In a specific application scenario, the second detection module 504 is also used to obtain the feature point detection neural network, which is composed of the preprocessing module, the enhancement module, the detection module and the description module, wherein the preprocessing module includes a preprocessing subnetwork, which includes an adaptive enhancement layer, a deblurring layer and a denoising layer connected in sequence, the enhancement module includes an enhanced shared encoder, which includes a first convolutional layer, a first LReLU layer, a first conditional attention layer, a first residual block layer, a first maximum pooling layer, a second convolutional layer, a second LReLU layer, a second conditional attention layer, a second residual block layer and a second maximum pooling layer connected in sequence, the detection module includes a detector head, which includes a third convolutional layer, a third LReLU layer, a hole convolution layer and a third convolutional layer connected in sequence, the description module includes a descriptor head, which includes a fourth convolutional layer, a fourth LReLU layer, a self-attention layer and a fourth convolutional layer connected in sequence; the current image frame is sequentially input into the adaptive enhancement layer, the deblurring layer and The denoising layer obtains a preprocessed image frame with target image quality; the preprocessed image frame is sequentially input into the first convolution layer, the first LReLU layer, the first conditional attention layer, the first residual block layer, the first maximum pooling layer, the second convolution layer, the second LReLU layer, the second conditional attention layer, the second residual block layer and the second maximum pooling layer in the enhanced shared encoder to obtain a target feature map with multi-scale information; the target feature map with multiple scale information is sequentially input into the third convolution layer, the third LReLU layer, the hole convolution layer, and the third convolution layer in the detector head to obtain a key point heat map; the target feature map with multiple scale information is sequentially input into the fourth convolution layer, the fourth LReLU layer, the self-attention layer, and the fourth convolution layer in the descriptor head to obtain a position descriptor of each pixel in the key point heat map; the multiple target feature points and the target position coordinates of each target feature point are determined based on the key point heat map and the position descriptor of each pixel in the key point heat map.

[0193] In a specific application scenario, the second detection module 504 is also used in the self-attention layer, which includes an input conversion unit, a multi-head attention calculation unit, a head merging unit, an output projection unit and a residual connection unit connected in sequence.

[0194] The device provided in the embodiment of the present application can determine the illumination evaluation value, contrast evaluation value, dynamic evaluation value and scene complexity evaluation value of the current image frame according to the image information of the current image frame through the determination module. At the same time, the Shi-Tomasi corner point detection algorithm is used to perform feature point detection on the current image frame through the first detection module to determine multiple initial feature points and feature information of each initial feature point, and the feature point stability, feature point repeatability and feature point density as well as the response value, contrast and edge strength of each initial feature point are determined based on the feature information and image information of each initial feature point. The index values ​​of these indicators are compared with the corresponding index thresholds. If the index value of any indicator among the index values ​​of these indicators does not exceed the corresponding index threshold through the second detection module, it means that the quality of the currently acquired image frame is poor, and a method with higher detection accuracy is needed to perform feature point detection, that is, the feature point detection neural network model is used to perform feature point detection on the current image frame to determine multiple target feature points and the target position coordinates of each target feature point. If the index values ​​of all indicators among the index values ​​of these indicators exceed the corresponding index threshold through the third detection module, it means that the quality of the currently acquired image frame is good and no further feature point detection is required. The above steps can improve the ability to handle feature point detection tasks on images of poor quality, thereby improving the accuracy of feature point detection.

[0195] It should be noted that for other corresponding descriptions of the functional units involved in the feature point detection method provided in the embodiment of the present invention, reference can be made to Figure 1 and Figure 5A , Figure 5B The corresponding description in will not be repeated here.

[0196] In an exemplary embodiment, the present application further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present application are implemented.

[0197] If the module / unit integrated in the computer unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0198] The present application also provides a computer program product, comprising a computer program, which implements the steps of the method described above when executed by a processor.

[0199] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0200] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A feature point detection method, characterized in that: include: Determining a current image frame, and acquiring image information of the current image frame; Determine an illumination evaluation value, a contrast evaluation value, a dynamics evaluation value, and a scene complexity evaluation value of the current image frame based on the image information; Performing feature point detection on the current image frame using the Shi-Tomasi corner point detection algorithm to determine a plurality of initial feature points and feature information of each initial feature point, and determining feature point stability, feature point repeatability and feature point density as well as a response value, contrast and edge strength of each initial feature point based on the feature information of each initial feature point and the image information; When it is detected that the illumination evaluation value does not exceed the illumination evaluation threshold, or the contrast evaluation value does not exceed the contrast evaluation threshold, or the dynamics evaluation value does not exceed the dynamics evaluation threshold, or the scene complexity evaluation value does not exceed the scene complexity evaluation threshold, or the response value does not exceed the response value threshold, or the contrast does not exceed the contrast threshold, or the edge intensity does not exceed the edge intensity threshold, or the feature point stability does not exceed the stability threshold, or the feature point repeatability does not exceed the repeatability threshold, or the feature point density does not exceed the density threshold, perform feature point detection on the current image frame using a feature point detection neural network model to determine multiple target feature points and the target position coordinates of each target feature point, wherein the feature point detection neural network model is composed of a preprocessing module, an enhancement module, a detection module and a description module; When it is detected that the illumination evaluation value exceeds the illumination evaluation threshold, the contrast evaluation value exceeds the contrast evaluation threshold, the dynamics evaluation value exceeds the dynamics evaluation threshold, the scene complexity evaluation value exceeds the scene complexity evaluation threshold, the response value exceeds the response value threshold, the contrast exceeds the contrast threshold, the edge intensity exceeds the edge intensity threshold, the feature point stability exceeds the stability threshold, the feature point repeatability exceeds the repeatability threshold, and the feature point density exceeds the density threshold, the multiple initial feature points are used as multiple target feature points, and the target position coordinates of each target feature point are extracted from the feature information of each target feature point; Wherein, determining the illumination evaluation value, contrast evaluation value, dynamics evaluation value and scene complexity evaluation value of the current image frame based on the image information includes: The image size and the grayscale value of each pixel are extracted from the image information, and the illumination evaluation value of the current image frame is determined by using a first formula based on the image size and the grayscale value of each pixel. The first formula is: ， in, is the pixel point in the current image frame ( i , j ), the gray value of the current image frame M and high N The image size is composed of is the light evaluation value; The contrast evaluation value of the current image frame is determined using a second formula based on the image size and the grayscale value of each pixel, where the second formula is: ； in, C is the contrast evaluation value; Acquire a previous image frame adjacent to the current image frame, determine previous image information of the previous image frame, extract a target grayscale value of each target pixel from the previous image information, and determine a dynamic evaluation value of the current image frame using a third formula based on the image size, the grayscale value of each pixel, and the target grayscale value of each target pixel, wherein the time interval between the previous sampling time corresponding to the previous image frame and the current sampling time corresponding to the current image frame is a preset time interval, and the third formula is: ； in, D is the dynamic evaluation value, is the pixel point in the current image frame ( i , j ), is the pixel point in the previous image frame ( i , j )’s grayscale value; The gradient magnitude of each pixel is extracted from the image information, and the scene complexity evaluation value of the current image frame is determined by using a fourth formula based on the image size and the gradient magnitude of each pixel. The fourth formula is: ； in, is the scene complexity evaluation value, is the pixel point in the current image frame ( i , j )’s gradient amplitude; Wherein, determining the feature point stability, feature point repeatability and feature point density as well as the response value, contrast and edge strength of each initial feature point based on the feature information of each initial feature point and the image information includes: Determine feature information of the initial feature point, extract a first eigenvalue and a second eigenvalue of a gradient covariance matrix of the initial feature point from the feature information, and determine a response value of the initial feature point using a Shi-Tomasi corner point response function based on the first eigenvalue and the second eigenvalue, wherein the Shi-Tomasi corner point response function is: ； in, R is the response value of the initial feature point, is the first eigenvalue of the gradient covariance matrix of the initial feature point, is the second eigenvalue of the gradient covariance matrix of the initial feature point; The pixel value of each local pixel point in the preset area around the initial feature point is extracted from the feature information, and a pixel average value is determined based on the pixel value of each local pixel point in the preset area around the initial feature point. The contrast of the initial feature point is determined by using a fifth formula based on the pixel average value and the pixel value of each local pixel point. The fifth formula is: ； in, is the contrast of the initial feature point, The first k The pixel value of a pixel, The preset area around the initial feature point n The average value of pixels; The gradient magnitude of the initial feature point is extracted from the feature information, and the edge strength of the initial feature point is determined based on the gradient magnitude using a sixth formula, where the sixth formula is: ； in, E is the edge strength of the initial feature point, is the initial feature point ( i , j )’s gradient amplitude; Obtaining a response value, a contrast ratio, and an edge strength of each of the initial feature points; The current position coordinates of each of the initial feature points are extracted from the feature information, a previous image frame adjacent to the current image frame is obtained, and the adjacent position coordinates of each of the initial feature points are determined based on the previous image frame, and the number of intersections of common feature points between the current image frame and the previous image frame is counted. Based on the number of intersections, the current position coordinates of each of the initial feature points, and the adjacent position coordinates, the stability of the feature point is determined by a seventh formula, and the seventh formula is: ， in, S is the stability of the feature points, is the first The current position coordinates of the initial feature points, is the first The adjacent position coordinates of the initial feature points, N is the number of intersections; Determine a plurality of adjacent feature points and adjacent feature information of each of the adjacent feature points based on the previous image frame, and query at least one initial feature point matching at least one adjacent feature point among the plurality of initial feature points based on the feature information and the adjacent feature information, and count the intermediate number of the at least one initial feature point , count the previous target number of the multiple adjacent feature points , calculate the intermediate quantity The previous target quantity The ratio of the feature point repeatability is obtained. ; Extracting the image area from the image information , count the number of targets of the multiple initial feature points in the current image frame , calculate the target quantity With the image area The ratio of the feature point density is obtained. D .

2. The feature point detection method according to claim 1, characterized in that: The determining of the current image frame comprises: Acquire a current video, where the current video consists of a plurality of image frames with time information; A color image frame at the latest acquisition time is extracted from the current video as a current initial image frame, and the current initial image frame is grayed to obtain the grayed current image frame.

3. The feature point detection method according to claim 1, characterized in that: After determining the feature point stability, feature point repeatability and feature point density as well as the response value, contrast and edge strength of each initial feature point based on the feature information of each initial feature point and the image information, the method further includes: Acquire a video sample in a current shooting scene, and determine the illumination evaluation threshold, the contrast evaluation threshold, the dynamics evaluation threshold, the scene complexity evaluation threshold, the stability threshold, the repeatability threshold, the density threshold, the response value threshold, the contrast threshold, and the edge intensity threshold based on the video sample, wherein the video sample does not include the current image frame; Specifically, the determining, based on the video sample, the illumination evaluation threshold, the contrast evaluation threshold, the dynamics evaluation threshold, the scene complexity evaluation threshold, the stability threshold, the repeatability threshold, the density threshold, the response value threshold, the contrast threshold, and the edge intensity threshold includes: Determine a plurality of initial real image samples based on the video sample, determine a sample sequence number, a first target illumination evaluation value, a first target contrast evaluation value, a first target dynamics evaluation value, a first target scene complexity evaluation value, a first target feature point stability, a first target feature point repeatability, and a first target feature point density of each of the initial real image samples, and use the sample sequence number, the first target illumination evaluation value, the first target contrast evaluation value, the first target dynamics evaluation value, the first target scene complexity evaluation value, the first target feature point stability, the first target feature point repeatability, and the first target feature point density to perform a Annotation, obtaining an intermediate annotated real image sample, and determining a first target response value, a first target contrast, and a first target edge strength of each feature point in each of the initial real image samples, annotating the corresponding feature points in each of the intermediate annotated real image samples with the first target response value, the first target contrast, and the first target edge strength, obtaining a plurality of annotated real image samples, obtaining a first real label for indicating image quality, annotating each of the annotated real image samples with the first real label, obtaining a plurality of real image samples, wherein the first real label is 1 or 0, wherein 1 is used to indicate good image quality, and 0 is used to indicate poor image quality; Performing the following operations on each of the initial real image samples: performing noise adding processing and blurring processing on the initial real image sample to obtain an intermediate processed image sample, and adjusting the contrast of the intermediate processed image sample to a target contrast to obtain an initial virtual image sample; Determine the sample sequence number, the second target illumination evaluation value, the second target contrast evaluation value, the second target dynamics evaluation value, the second target scene complexity evaluation value, the second target feature point stability, the second target feature point repeatability, and the second target feature point density of the initial virtual image sample, use the sample sequence number, the second target illumination evaluation value, the second target contrast evaluation value, the second target dynamics evaluation value, the second target scene complexity evaluation value, the second target feature point stability, the second target feature point repeatability, and the second target feature point density to label the initial virtual image sample to obtain an intermediate labeled virtual image sample, and determine the second target response value, the second target contrast, and the second target edge strength of each feature point in each of the initial virtual image samples, use the second target response value, the second target contrast, and the second target edge strength to label the corresponding feature point in each of the intermediate labeled virtual image samples to obtain a plurality of labeled virtual image samples, obtain a second real label for indicating image quality, use the second real label to label each of the labeled virtual image samples to obtain a plurality of virtual image samples, and the second real label is 1 or 0, wherein 1 is used to indicate good image quality and 0 is used to indicate poor image quality; Based on the multiple real image samples and the multiple virtual image samples, a binarization algorithm is used to obtain the illumination evaluation threshold, the contrast evaluation threshold, the dynamics evaluation threshold, the scene complexity evaluation threshold, the stability threshold, the repeatability threshold, the density threshold, the response value threshold, the contrast threshold, and the edge intensity threshold.

4. The feature point detection method according to claim 1, characterized in that: The method of performing feature point detection on the current image frame using a feature point detection neural network model to determine a plurality of target feature points and target position coordinates of each target feature point includes: Acquire the feature point detection neural network, which is composed of the preprocessing module, the enhancement module, the detection module and the description module, wherein the preprocessing module includes a preprocessing subnetwork, and the preprocessing subnetwork includes an adaptive enhancement layer, a deblurring layer and a denoising layer connected in sequence, the enhancement module includes an enhanced shared encoder, and the enhanced shared encoder includes a first convolutional layer, a first LReLU layer, a first conditional attention layer, a first residual block layer, a first maximum pooling layer, a second convolutional layer, a second LReLU layer, a second conditional attention layer, a second residual block layer and a second maximum pooling layer connected in sequence, the detection module includes a detector head, and the detector head includes a third convolutional layer, a third LReLU layer, a hole convolution layer and a third convolutional layer connected in sequence, and the description module includes a descriptor head, and the descriptor head includes a fourth convolutional layer, a fourth LReLU layer, a self-attention layer and a fourth convolutional layer connected in sequence; Inputting the current image frame into the adaptive enhancement layer, the deblurring layer and the denoising layer in the preprocessing subnetwork in sequence to obtain a preprocessed image frame with target image quality; Inputting the preprocessed image frame into the first convolution layer, the first LReLU layer, the first conditional attention layer, the first residual block layer, the first maximum pooling layer, the second convolution layer, the second LReLU layer, the second conditional attention layer, the second residual block layer and the second maximum pooling layer in the enhanced shared encoder in sequence to obtain a target feature map with multi-scale information; Inputting the target feature map with multiple scale information into the third convolution layer, the third LReLU layer, the hole convolution layer, and the third convolution layer in the detector head in sequence to obtain a key point heat map; Input the target feature map with multiple scale information into the fourth convolutional layer, the fourth LReLU layer, the self-attention layer, and the fourth convolutional layer in the descriptor head in sequence to obtain a position descriptor of each pixel in the key point heat map; The plurality of target feature points and the target position coordinates of each target feature point are determined based on the key point heat map and the position descriptor of each pixel point in the key point heat map.

5. The feature point detection method according to claim 4, characterized in that: The self-attention layer includes an input conversion unit, a multi-head attention calculation unit, a head merging unit, an output projection unit and a residual connection unit which are connected in sequence.

6. A feature point detection device, characterized in that: include: An acquisition module, used to determine a current image frame and acquire image information of the current image frame; A determination module, configured to determine an illumination evaluation value, a contrast evaluation value, a dynamics evaluation value, and a scene complexity evaluation value of the current image frame based on the image information; A first detection module is used to perform feature point detection on the current image frame using a Shi-Tomasi corner point detection algorithm, determine a plurality of initial feature points and feature information of each initial feature point, and determine feature point stability, feature point repeatability and feature point density, as well as a response value, contrast, and edge strength of each initial feature point based on the feature information of each initial feature point and the image information; A second detection module is used to perform feature point detection on the current image frame using a feature point detection neural network model to determine multiple target feature points and target position coordinates of each target feature point when it is detected that the illumination evaluation value does not exceed the illumination evaluation threshold, or the contrast evaluation value does not exceed the contrast evaluation threshold, or the dynamics evaluation value does not exceed the dynamics evaluation threshold, or the scene complexity evaluation value does not exceed the scene complexity evaluation threshold, or the response value does not exceed the response value threshold, or the contrast does not exceed the contrast threshold, or the edge intensity does not exceed the edge intensity threshold, or the feature point stability does not exceed the stability threshold, or the feature point repeatability does not exceed the repeatability threshold, or the feature point density does not exceed the density threshold, wherein the feature point detection neural network model is composed of a preprocessing module, an enhancement module, a detection module, and a description module; A third detection module is used for, when it is detected that the illumination evaluation value exceeds the illumination evaluation threshold, the contrast evaluation value exceeds the contrast evaluation threshold, the dynamics evaluation value exceeds the dynamics evaluation threshold, the scene complexity evaluation value exceeds the scene complexity evaluation threshold, the response value exceeds the response value threshold, the contrast exceeds the contrast threshold, the edge intensity exceeds the edge intensity threshold, the feature point stability exceeds the stability threshold, the feature point repeatability exceeds the repeatability threshold, and the feature point density exceeds the density threshold, taking the multiple initial feature points as multiple target feature points, and extracting the target position coordinates of each of the target feature points from the feature information of each of the target feature points; Wherein, determining the illumination evaluation value, contrast evaluation value, dynamics evaluation value and scene complexity evaluation value of the current image frame based on the image information includes: The image size and the grayscale value of each pixel are extracted from the image information, and the illumination evaluation value of the current image frame is determined by using a first formula based on the image size and the grayscale value of each pixel. The first formula is: ， in, is the pixel point in the current image frame ( i , j ), the gray value of the current image frame M and high N The image size is composed of is the light evaluation value; The contrast evaluation value of the current image frame is determined using a second formula based on the image size and the grayscale value of each pixel, where the second formula is: ； in, C is the contrast evaluation value; Acquire a previous image frame adjacent to the current image frame, determine previous image information of the previous image frame, extract a target grayscale value of each target pixel from the previous image information, and determine a dynamic evaluation value of the current image frame using a third formula based on the image size, the grayscale value of each pixel, and the target grayscale value of each target pixel, wherein the time interval between the previous sampling time corresponding to the previous image frame and the current sampling time corresponding to the current image frame is a preset time interval, and the third formula is: ； in, D is the dynamic evaluation value, is the pixel point in the current image frame ( i , j ), is the pixel point in the previous image frame ( i , j )’s grayscale value; The gradient magnitude of each pixel is extracted from the image information, and the scene complexity evaluation value of the current image frame is determined by using a fourth formula based on the image size and the gradient magnitude of each pixel. The fourth formula is: ； in, is the scene complexity evaluation value, is the pixel point in the current image frame ( i , j )’s gradient amplitude; Wherein, determining the feature point stability, feature point repeatability and feature point density as well as the response value, contrast and edge strength of each initial feature point based on the feature information of each initial feature point and the image information includes: Determine feature information of the initial feature point, extract a first eigenvalue and a second eigenvalue of a gradient covariance matrix of the initial feature point from the feature information, and determine a response value of the initial feature point using a Shi-Tomasi corner point response function based on the first eigenvalue and the second eigenvalue, wherein the Shi-Tomasi corner point response function is: ； in, R is the response value of the initial feature point, is the first eigenvalue of the gradient covariance matrix of the initial feature point, is the second eigenvalue of the gradient covariance matrix of the initial feature point; The pixel value of each local pixel point in the preset area around the initial feature point is extracted from the feature information, and a pixel average value is determined based on the pixel value of each local pixel point in the preset area around the initial feature point. The contrast of the initial feature point is determined by using a fifth formula based on the pixel average value and the pixel value of each local pixel point. The fifth formula is: ； in, is the contrast of the initial feature point, The first k The pixel value of a pixel, The preset area around the initial feature point n The average value of pixels; The gradient magnitude of the initial feature point is extracted from the feature information, and the edge strength of the initial feature point is determined based on the gradient magnitude using a sixth formula, where the sixth formula is: ； in, E is the edge strength of the initial feature point, is the initial feature point ( i , j )’s gradient amplitude; Obtaining a response value, a contrast ratio, and an edge strength of each of the initial feature points; The current position coordinates of each of the initial feature points are extracted from the feature information, a previous image frame adjacent to the current image frame is obtained, and the adjacent position coordinates of each of the initial feature points are determined based on the previous image frame, and the number of intersections of common feature points between the current image frame and the previous image frame is counted. Based on the number of intersections, the current position coordinates of each of the initial feature points, and the adjacent position coordinates, the stability of the feature point is determined by a seventh formula, and the seventh formula is: ， in, S is the stability of the feature points, is the first The current position coordinates of the initial feature points, is the first The adjacent position coordinates of the initial feature points, N is the number of intersections; Determine a plurality of adjacent feature points and adjacent feature information of each of the adjacent feature points based on the previous image frame, and query at least one initial feature point matching at least one adjacent feature point among the plurality of initial feature points based on the feature information and the adjacent feature information, and count the intermediate number of the at least one initial feature point , count the previous target number of the multiple adjacent feature points , calculate the intermediate quantity The previous target quantity The ratio of the feature point repeatability is obtained. ; Extracting the image area from the image information , count the number of targets of the multiple initial feature points in the current image frame , calculate the target quantity With the image area The ratio of the feature point density is obtained. D .

7. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images