YOLO-based robust angular point rapid detection method

Abstract

The invention discloses a rapid detection method for robust angular points based on YOLO, and the method is used for image intersection point detection. The method comprises the steps that an edge intersection point serves as a structured feature object, two types of structured feature areas are set, only two edge lines exist in each structured feature area and intersect at one point, and the structured feature areas are divided into two types according to the opening direction; constructing a training image set of the two types of structured feature regions to train and identify a YOLO model of the two types of structured feature regions; detecting an image to be detected by using the trained YOLO model, and calculating edge lines and intersection points in the detected target area; and finally, positioning angular points in a neighborhood of the intersection point of the edge lines by using a Harris algorithm. According to the method, the structured feature regions in the image are accurately positioned and classified, robust angular points in the image can be quickly positioned, and the problem that excessive redundant feature points are generated when the feature points of the image are recognized at present is avoided.

Description

Technical Field

[0001] The present invention relates to the field of image corner detection, and particularly to a robust corner fast detection method based on YOLO. Background Art

[0002] Identifying image feature points is one of the important tasks in the field of computer vision and can be used in various applications such as target tracking, image matching, and 3D reconstruction. Corners are very important feature points in images, usually referring to the intersection points of edge lines in the image or feature points with significant local autocorrelation, usually corresponding to positions such as the corners of objects and the intersections of roads. Identifying corners is a finer-grained feature detection of images. Traditional feature point detection algorithms include Harris corner detection, FAST (Features from Accelerated Segment Test), SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), etc. These methods perform well in some simple scenarios and problems, but there are still many deficiencies. For example, many traditional algorithms are sensitive to scale and rotation changes, which means that the same feature points may not be correctly detected at different scales and angles. With the development of deep learning technology, neural network-based methods have gradually become an alternative to traditional methods, including SuperPoint, LF-Net, D2-Net, TILDE, etc. However, both deep learning-based methods and traditional methods have a problem that in the case of complex textures or low image resolutions, there may be too many redundant feature points.

[0003] YOLO (You Only Look Once) is a deep learning model for real-time object detection that transforms the object detection problem into a regression problem and simultaneously predicts multiple bounding boxes and class probabilities through a single forward pass. Summary of the Invention

[0004] Aiming at the problem of generating too many redundant feature points when identifying image feature points, the present invention proposes a fast detection method for robust corners based on YOLO. Taking the edge intersection points as structured feature objects, YOLO is used to identify the structured feature regions in the image, and robust corners are quickly located within the regions, so as to better complete visual tasks with a limited number of feature points.

[0005] The fast detection method for robust corners based on YOLO provided by the present invention includes the following steps:

[0006] Step 1: Taking the edge intersection point as the structured feature object, set two types of structured feature regions. Within the structured feature region, there are exactly two edge lines that intersect at a point. The structured feature region can be divided into two types according to the opening direction of the included angle formed by the two edge lines: ① The first type of structured feature region: the opening direction of the included angle is to the left or to the right, the included angle θ satisfies 30° ≤ θ ≤ 150°, and the length of the edge line is at least greater than 30 pixels. ② The second type of structured feature region: the opening direction of the included angle is upward or downward, the included angle θ satisfies 30° ≤ θ ≤ 150°, and the length of the edge line is at least greater than 30 pixels.

[0007] Step 2: Construct a dataset for training the YOLO model to identify the structured feature region. The images in the training dataset are drawn according to the requirements of the first type or the second type of structured feature region, and the labels of the images are the first type or the second type of structured feature region.

[0008] Step 3: Use the constructed training set to train the YOLO model, obtain the optimal model, and save the weight file obtained from the training.

[0009] Step 4: Convert the image to be detected into a grayscale image and input it into the trained YOLO model for detecting the structured feature region, and calculate the edge line intersection points within each detected target region. The target region is detected as the first type or the second type of structured feature region. Set the proportionality coefficient k. According to the category of the structured feature region to which the target region belongs, multiply the height or width of the target region by k as the edge detection interval, and traverse the target region along the vertical direction or the horizontal direction at this interval to detect edge points and calculate the edge lines and edge line intersection points.

[0010] Step 5: In the neighborhood of the edge line intersection point, use the Harris corner detection algorithm to locate the corner points in the neighborhood, and select the point with the largest response value as the corner point.

[0011] In the above Step 4, if the target region is detected as the first type of structured feature region, then perform the following steps:

[0012] Step 411: Set the vertical edge detection interval Δy = k·H according to the height H of the target region;

[0013] Step 412: Traverse the target region image along the vertical direction with a step size of Δy, and detect edge points for each row, including: calculating the gray difference between adjacent pixels point by point for the current row, and let ΔG i represent the gray difference between the i-th pixel and the (i + 1)-th pixel; if the detected gray difference ΔG i satisfies the following conditions, then the i-th pixel is considered an edge point, and at this time, stop searching for the current row;

[0014]

[0015] where W is the width of the target area;

[0016] Step 413: Arrange all the edge points found by traversal search in ascending order of the ordinate, find the extreme points of the abscissas of all the edge points as the demarcation points, divide all the edge points with ordinates less than the ordinate of the demarcation point into one group, and divide all the edge points with ordinates greater than the ordinate of the demarcation point into another group;

[0017] Step 414: Use the least squares method to fit the two groups of edge points respectively, obtain two edge lines, and calculate the intersection point of the edge lines.

[0018] In the fourth step described above, if the target area is detected as the second type of structured feature area, the following steps are executed at this time:

[0019] Step 421: Set the horizontal interval Δx = k·W for edge detection according to the width W of the target area;

[0020] Step 422: Perform column traversal on the target area image along the horizontal direction with a step size of Δx, and perform edge detection on each column, including: calculating the gray difference between adjacent pixels point by point for the current column. Let ΔG j represent the gray difference between the j-th pixel and the (j + 1)-th pixel of the current column; if the detected gray difference ΔG j satisfies the following conditions, then the j-th pixel is considered an edge point, and the search for the current column stops at this time;

[0021]

[0022] where H is the height of the target area;

[0023] Step 423: Arrange all the edge points found by traversal search in ascending order of the abscissa, find the position of the extreme point of the ordinate of all the edge points as the demarcation point, divide all the edge points with abscissas less than the abscissa of the demarcation point into one group, and divide all the edge points with abscissas greater than the abscissa of the demarcation point into another group;

[0024] Step 424: Use the least squares method to fit the two groups of edge points respectively, obtain two edge lines, and calculate the intersection point of the edge lines.

[0025] Compared with the prior art, the advantages and positive effects of the present invention are as follows: The method of the present invention uses the edge intersection points as the structured feature objects, pre - defines and classifies the structured feature regions in the image, and constructs a training data set based on this to train a YOLO model for recognizing the structured feature regions, ensuring that in subsequent actual target recognition, the structured feature regions in the image can be accurately located and classified, laying a foundation for the subsequent steps of the robust corner detection method. The method of the present invention combines deep learning and the gradient information within the region, can quickly and robustly identify and locate the robust corner points in the image, avoids the problem of generating too many redundant feature points when currently identifying image feature points, and can achieve the purpose of better completing visual tasks using a limited number of feature points. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 FIG. is a schematic diagram of two types of structured feature regions to be recognized by the present invention, where (a) is an example diagram of the first type of structured feature region and (b) is an example diagram of the second type of structured feature region;

[0027] Figure 2 FIG. is a schematic diagram of an implementation process for quickly detecting and locating the robust corner points of the present invention;

[0028] Figure 3 FIG. is a schematic diagram of the structured feature region generated in the embodiment of the present invention;

[0029] Figure 4 FIG. is a schematic diagram of calculating the intersection points of the edge lines of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The present invention will be further described below in conjunction with specific drawings and embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the embodiments cited do not limit the present invention.

[0031] Due to the existence of constraints such as contours and edges, the structured features have good robustness and position stability. Based on this, the method of the present invention uses the edge intersection points as the structured feature objects, and uses YOLO to recognize the structured feature regions in the image, and quickly locates the robust corner points within the regions. The method of the present invention first constructs a data set of structured feature regions, then conducts model training, obtains a weight file and conducts target region detection to obtain the structured feature regions where the corner points are located, and finally applies different corner point positioning methods within different category target regions, and outputs the recognized and located corner point information for subsequent visual tasks.

[0032] First, the structured feature regions to be recognized by the method of the present invention are described. In the structured feature regions of the present invention, there are and only two edge lines that intersect at one point. The structured feature regions can be divided into two categories according to the opening direction of the included angle formed by the two edge lines, as Figure 1As shown below. Two types of structured feature regions are described as follows.

[0033] ① The first type of structured feature region: The opening direction of the included angle faces left or right, the included angle θ satisfies 30° ≤ θ ≤ 150°, and the length of the edge line is at least greater than 30 pixels.

[0034] ② The second type of structured feature region: The opening direction of the included angle faces up or down, the included angle θ satisfies 30° ≤ θ ≤ 150°, and the length of the edge line is at least greater than 30 pixels.

[0035] In the application scenario of the embodiment of the present invention, the camera is installed on the drone. The drone identifies the robust corner points in the image captured by the camera, enabling it to complete subsequent vision tasks. As Figure 1 shown, the fast detection method of robust corner points based on YOLO in the embodiment of the present invention mainly includes the following four steps.

[0036] Step 1, prepare the training dataset for the YOLO model. The YOLO model is used to identify two types of structured feature regions.

[0037] (11) Draw the structured feature region images according to the requirements of the structured feature regions.

[0038] In the embodiment of the present invention, MATLAB is used to generate the structured feature region images, as Figure 3 shown. The images should contain robust corner points with various opening directions (left, right, up, down), the length of the two edge lines is at least 30 pixels, the included angle ranges from 30 degrees to 150 degrees, and a set of images is generated every 5 degrees during generation. To ensure the diversity of the image data and enhance the generalization ability of the model, the following operations ① to ④ are performed on each generated image.

[0039] ① Apply different gray levels to each image, multiply the image gray level by coefficients 0.5, 0.75, and 1.0 respectively to simulate different lighting conditions.

[0040] ② Use the imnoise function in MATLAB to add Gaussian noise to the images to simulate the interference in the real environment.

[0041] ③ Rotate the images slightly appropriately, rotate each image by -15°, -10°, -5°, 5°, 10°, 15°.

[0042] ④ Scale the images to generate images with different scaling ratios, and the scaling ratios are 0.5, 0.75, 1.0, 1.25, and 1.5 respectively.

[0043] (12) Image annotation.

[0044] Use the image annotation tool LabelImg to annotate the images, and assign two labels according to the opening direction. The labels are the first type or the second type of structured feature regions. During annotation, if the region is a left or right opening angle, the label is the first type of structured feature region; if the region is an upper or lower opening angle, the label is the second type of structured feature region.

[0045] (13) Dataset division.

[0046] Divide the annotated dataset into a training set and a validation set to ensure the diversity and coverage of the dataset.

[0047] Step 2, YOLO model training. The input of the YOLO model is an image, and it outputs all the structured feature regions detected in the image.

[0048] Use the above dataset to train the YOLO model, and adjust the hyperparameters to obtain the best model performance. During training, use the validation set to evaluate the model's performance and adjust the model parameters as needed. After training, save the trained model weight file for subsequent target region detection. In the embodiment of the present invention, when performing target detection, the input is an image captured by a drone camera.

[0049] Step 3, Calculation of the intersection points of the edge lines of the target region.

[0050] In the embodiment of the present invention, use the trained weight file to load the YOLO model, detect the image stream collected by the drone camera, and detect all the structured feature regions in the image. According to the region label, distinguish the types of feature points. For different types of feature points, apply the corresponding edge line intersection calculation method to calculate the intersection points of the edge lines in the region. The calculation of the intersection points of the edge lines is as Figure 4 shown.

[0051] (31) If the detected target region is the first type of structured feature region, then as Figure 4 (a) shows, perform the following calculations:

[0052] (311) Determine the vertical interval Δy for edge detection according to the height of the target region. Let the height of the target region be H and the width be W, then the vertical interval Δy = k·H, where k is an appropriate proportionality coefficient that can be adjusted according to actual needs to balance the calculation efficiency and edge detection accuracy. In the embodiment of the present invention, for both types of target regions, the proportionality coefficient k is set to 0.1.

[0053] (312) Traverse the target region image vertically with a step size of Δy, perform edge detection on each row, and determine the edge points. The steps for calculating the edge points are as follows:

[0054] ① Calculate the gray difference ΔG between adjacent pixels point by point for the current rowi As follows:

[0055] ΔG i = |G i+1 - G i |, 1 ≤ i ≤ W - 1;

[0056] Wherein, G i+1 and G i are the gray values of the (i + 1)-th and i-th pixels in this row, respectively.

[0057] ② Edge point determination rule: When the gray difference ΔG i of a certain point satisfies the following conditions, it is considered that this point, that is, the i-th pixel, is on the edge, and subsequent searches are stopped.

[0058]

[0059] The above ΔG i is the gray difference between the (i + 1)-th and i-th pixels in this row. For example, ΔG i+1 is the gray difference between the (i + 2)-th and (i + 1)-th pixels in this row, and ΔG i-1 is the gray difference between the i-th and (i - 1)-th pixels in this row.

[0060] (313) Classification of edge points.

[0061] All the edge points found through traversal search are recorded in the order of increasing ordinate of the edge points. Then the variation law of its abscissa is to increase first and then decrease, or to decrease first and then increase. Find the extreme point positions of the abscissas of all edge points, and use this as the demarcation point. Divide all the edge points with ordinates less than the ordinate of the demarcation point into one category, and all the edge points with ordinates greater than the ordinate of the demarcation point into another category for subsequent calculation of two edge lines.

[0062] (314) Calculation of the intersection point of edge lines. For the two categories of edge points in the above step 313, fit two edge lines respectively using the least squares method. Calculate the intersection point of the edge lines from the two fitted edge lines.

[0063] For a set of points (x m , y m ), m = 1, 2... n, the fitted straight line equation is y = ax + b. The goal of the least squares method is to find the parameters a and b to minimize the sum of squared errors. The sum of squared errors is expressed as follows:

[0064] ∑(y m - (ax m + b)) 2 ;

[0065] The calculation formulas for obtaining the parameters a and b are as follows:

[0066]

[0067] For the two fitted edge lines y = a1x + b1 and y = a2x + b2, the calculation formula for the intersection point (x, y) is as follows:

[0068]

[0069] (32) If the target area is a second - type structured feature area, then as shown in Figure 4 (b), perform the following calculations.

[0070] (321) Calculate the horizontal interval for edge detection according to the height of the target area. Let the width of the target area be W and the height be H, then the horizontal interval Δx = k·W, where the proportionality coefficient k is set to 0.1.

[0071] (322) Traverse the columns of the target area image horizontally with a step size of Δx, perform edge detection on each column, and determine the edge points. The steps for calculating the edge points are as follows:

[0072] ① Calculate the gray - level difference ΔG of adjacent pixels point - by - point for the current column j as follows:

[0073] ΔG j = |G j+1 - G j |, 1 ≤ j ≤ H - 1;

[0074] where G j+1 and G j are the gray - level values of the (j + 1) - th and j - th pixels in this column respectively.

[0075] ② Edge - point determination rule: When the gray - level difference ΔG of a certain point j meets the following conditions, it is considered that the j - th pixel point is on the edge, and the subsequent search is stopped.

[0076]

[0077] The above ΔG i is the gray - level difference between the (i + 1) - th and i - th pixels in this row. For example, ΔG i+1 is the gray - level difference between the (i + 2) - th and (i + 1) - th pixels in this row, and ΔG i-1 is the gray - level difference between the i - th and (i - 1) - th pixels in this row.

[0078] (323) Edge - point classification.

[0079] All the edge points found through traversal search will be recorded in the order of increasing abscissa of the edge points. Then, the variation law of their ordinates is to increase first and then decrease, or to decrease first and then increase. Locate the extreme point positions of the ordinates of all edge points, and use these as the demarcation points. Divide all edge points with abscissas less than the abscissa of the demarcation point into one category, and all edge points with abscissas greater than the abscissa of the demarcation point into another category for subsequent calculation of two edge lines.

[0080] (324) Calculation of the intersection point of the edge lines. For the above two types of edge points, respectively use the least squares method to fit two edge lines and calculate their intersection points in the same way.

[0081] Step 4, corner location.

[0082] In the neighborhood of the intersection point of the edge lines, use the Harris algorithm to locate the corners, and select the point with the largest response value as the corner.

[0083] The present invention is based on YOLO to identify the structured feature regions in the image, and finally locate the robust corners in the structured feature regions. By classifying the structured feature regions, different edge detection strategies are designed to calculate the intersection point of the two edge lines. In the neighborhood of the intersection point, the Harris algorithm is applied to obtain the location of the robust corners. This ensures that only the robust corners in the structured regions in an image are quickly located, avoiding redundant or unstable feature point detection results.

Claims

1. A fast detection method for robust corner points based on YOLO, characterized in that, It includes the following steps: Step 1: Taking the edge intersection points as the structured feature objects, set two types of structured feature regions; There are exactly two edge lines in the structured feature region, and they intersect at one point. The length of each edge line is at least greater than 30 pixels. Let the angle between the two edge lines be θ. In the first type of structured feature region, the opening direction of the angle faces left or right, and the angle θ satisfies 30° ≤ θ ≤ 150°; in the second type of structured feature region, the opening direction of the angle faces up or down, and the angle θ satisfies 30° ≤ θ ≤ 150°; Step 2: Construct a training data set for training the YOLO model to recognize the two types of structured feature regions; The images in the training data set are drawn according to the requirements of the first type or the second type of structured feature region, and the labels of the images are the first type or the second type of structured feature region; the input of the YOLO model is a grayscale image, and the output is all the structured feature regions detected in the image; Step 3: Train the YOLO model to obtain the optimal model; Step 4: Convert the image to be detected into a grayscale image and input it into the trained YOLO model for detection, and calculate the edge line intersection points in each detected target region; The target region is detected as the first type or the second type of structured feature region; Set the proportionality coefficient k. According to the category of the structured feature region to which the target region belongs, multiply the height or width of the target region by k as the edge detection interval, and traverse the target region along the vertical or horizontal direction at this interval to detect edge points, and calculate the edge lines and edge line intersection points; Step 5: In the neighborhood of the edge line intersection point, use the Harris corner detection algorithm to locate the corner points in the neighborhood, and select the point with the largest response value as the corner point.

2. The method according to claim 1, wherein In the above-mentioned Step 2, the method for constructing the training data set is as follows: Step 21: Draw structured feature region images, including: (1) Vary the included angle from 30 degrees to 150 degrees, and generate a set of structured feature region images with different opening directions at each angle. The opening directions include left, right, up, and down; (2) Perform the following operations on each generated image: a. Apply different gray levels to the image; b. Add Gaussian noise to the image; c. Apply different rotation angles to the image; d. Scale the image by different ratios; Step 22: Set labels for the images according to the opening direction of the included angle; Step 23: Divide the image data set with labels into a training set and a validation set.

3. The method according to claim 1, wherein In the above-mentioned Step 4, if the target region is the first type of structured feature region, the edge line intersection points in the region are calculated as follows: Step 411: Set the longitudinal interval Δy = k·H for edge detection according to the height H of the target region; Step 412: Traverse the target region image vertically with a step size of Δy, and perform edge point detection on each row, including: calculating the gray difference between adjacent pixels point by point for the current row, and setting ΔG i to represent the gray difference between the i-th pixel and the (i + 1)-th pixel of the current row; if it is detected that the gray difference ΔG i satisfies the following conditions, then the i-th pixel is considered an edge point, and the search for the current row stops at this time; where W is the width of the target region; Step 413: Arrange all the edge points searched by traversal in ascending order of the ordinate, find the extreme points of the abscissas of all the edge points as the demarcation points, divide all the edge points with ordinates less than the ordinate of the demarcation point into one group, and divide all the edge points with ordinates greater than the ordinate of the demarcation point into another group; Step 414: Use the least squares method to fit the two sets of edge points respectively to obtain two edge lines, and calculate the intersection point of the edge lines from the two edge lines.

4. The method according to claim 1, wherein In step 4 above, if the target area is a second type of structured feature area, the intersection point of the edge lines in the area is calculated as follows: Step 421: Set the horizontal interval Δx = k·W for edge detection according to the width W of the target area; Step 422: Traverse the target region image column by column in the horizontal direction with a step size of Δx, and perform edge detection on each column, including: calculating the gray difference between adjacent pixels point by point for the current column, and setting ΔG j to represent the gray difference between the j-th pixel and the (j + 1)-th pixel in the current column; if the detected gray difference ΔG j satisfies the following conditions, then the j-th pixel is considered an edge point, and the search for the current column stops at this time; where H is the height of the target area; Step 423: Arrange all the edge points searched by traversal in ascending order of abscissa, find the extreme point positions of the ordinates of all the edge points as the demarcation points, divide all the edge points with abscissa less than the abscissa of the demarcation point into one group, and divide all the edge points with abscissa greater than the abscissa of the demarcation point into another group; Step 424: Use the least squares method to fit the two sets of edge points respectively to obtain two edge lines, and calculate the intersection point of the edge lines from the two edge lines.

Images