Six-degree-of-freedom perspective invariance feature analysis method and device, and processing equipment

Abstract

The application relates to a six-degree-of-freedom perspective invariance feature analysis method and device and processing equipment. The method performs corner point detection on a target image to be detected based on a corner point detection algorithm to obtain candidate corner points; the candidate corner points are processed based on a space segmentation algorithm to obtain four adjacent corner points; the four adjacent corner points are sorted according to a preset rule to obtain four sorted corner points; the target image to be detected is subjected to homography processing according to the four sorted corner points to obtain a transformed image; the transformed image is subjected to DFT transformation processing to obtain six-degree-of-freedom perspective invariance feature values with high stability and feature resolution capability which can be used for target matching. In the actual measurement of six-degree-of-freedom transformation, the feature values of the corresponding feature points can be maintained below a set percentage of fluctuation, thereby improving the stability and resolution capability of the feature values.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of feature processing, in particular to a six-degree-of-freedom perspective invariance feature analysis method and device and processing equipment. BACKGROUND

[0002] Currently, SIFT (Scale-invariant feature transform), SURF (Speeded Up Robust Features) and ORB (Oriented FAST and Rotated BRIEF) algorithms are widely used in image target matching. For example, the SIFT algorithm of scale-invariant feature transform has played a huge role in target recognition, image stitching and three-dimensional reconstruction. The features detected by the SIFT algorithm are local features, the features have scale and rotation invariance, are very robust to brightness and noise, and can be correctly identified in a low probability of mismatching, and have strong distinguishability. The SURF algorithm of accelerated robust features is a local feature detection algorithm with robustness. The SURF algorithm is simplified and approximated on the premise of ensuring correctness, and the concept of integral image is used multiple times to speed up the operation rate. In addition to having a high repeatability and a good distinguishability of feature vectors, the SURF algorithm also has strong robustness and higher operation rate, and the comprehensive performance is better than that of the SIFT algorithm. The ORB algorithm frequently used in visual SLAM technology is combined from the FAST (Features from accelerated segment test) algorithm and the BRIEF (Binary Robust Independent Elementary Features) algorithm. Its significant feature is fast speed, but it does not have rotation invariance. Therefore, the ORB algorithm assigns a direction angle to the feature points detected by the FAST algorithm according to the intensity centroid method after the feature points are detected by the FAST algorithm, and rotates the descriptor created in the BRIEF algorithm to the direction angle. In addition to completely retaining the fast characteristics of the two algorithms, the ORB algorithm also realizes rotation invariance.

[0003] In the implementation process, the inventors found that at least the following problems exist in the prior art: In the existing target matching, since the features themselves do not have complete six-degree-of-freedom perspective invariance, when the camera and the target have multi-degree-of-freedom relative motion, the target matching has the phenomena of feature mismatching or feature loss. SUMMARY

[0004] Therefore, it is necessary to provide a six-degree-of-freedom perspective invariance feature analysis method, device and processing equipment with high stability and feature distinguishability for target matching in view of the problems existing in the prior art.

[0005] To achieve the above objectives, embodiments of the present invention provide a six-degree-of-freedom perspective invariance feature analysis method, comprising the following steps:

[0006] Corner detection algorithms are used to detect corners in the target image to obtain candidate corners.

[0007] The candidate corner points are processed using a spatial segmentation algorithm to obtain the four adjacent corner points;

[0008] Sort the adjacent four corner points according to a preset rule to obtain the sorted four corner points;

[0009] Based on the sorted four corner points, homography processing is performed on the target image to be tested to obtain the transformed image;

[0010] The transformed image is processed by DFT transformation to obtain six-degree-of-freedom perspective invariant eigenvalues.

[0011] In one embodiment, the step of performing homography processing on the target image to be tested based on the sorted four corner points to obtain a transformed image includes:

[0012] Based on the coordinates of the four corner points after sorting, homography processing is performed on the coordinates of the four corner points of the target image to be tested to obtain the homography matrix;

[0013] Based on the homography matrix, the target image to be tested is subjected to homography transformation to obtain the transformed image.

[0014] In one embodiment, the step of sorting adjacent corner points according to a preset rule to obtain the sorted corner points includes:

[0015] The corner point with the highest brightness among the four adjacent corner points is identified as the main point, and the average coordinate of the four adjacent corner points is identified as the center point.

[0016] Using the principal angle as the zero direction, sort the remaining three corner points among the adjacent four corner points according to the first order to obtain the sorted four corner points.

[0017] In one embodiment, the step of performing corner detection on the target image to obtain candidate corner points based on a corner detection algorithm includes:

[0018] Select the pixels of the target image to be tested, and confirm the window centered on the pixel as the selection window;

[0019] The selected window is moved along the first and second coordinate directions on the target image to obtain the change in pixel value.

[0020] The pixel value change is converted to obtain the corner response function of the corresponding preset window;

[0021] When the corner response function is greater than the first preset threshold, the pixel of the corresponding selected window is identified as a candidate corner.

[0022] In one embodiment, the pixel value change is:

[0023]

[0024] Where E(u, v) is the pixel value change, w(x, y) is the selection window, (x, y) is the pixel, u is the movement in the first coordinate direction, v is the movement in the second coordinate direction, I(x, y) is the gray level corresponding to the pixel at (x, y), and Ix and Iy are the partial derivatives of I(x, y). R is the corner response function, λ1 is the first orthogonal direction variation component, and λ2 is the second orthogonal direction variation component.

[0025] In one embodiment, the step of processing candidate corner points based on a spatial segmentation algorithm to obtain adjacent four corner points includes:

[0026] The candidate corner points are processed using an octree or quadtree algorithm to obtain the four adjacent corner points.

[0027] In one embodiment, the step of performing a DFT transformation on the transformed image to obtain six-DOF perspective invariant eigenvalues ​​includes:

[0028] Perform one-dimensional or two-dimensional DFT transformation on the transformed image to obtain six-degree-of-freedom perspective invariant eigenvalues.

[0029] In one embodiment, the step of performing a DFT transformation on the transformed image to obtain six-DOF perspective invariant eigenvalues ​​includes:

[0030] Perform DFT transformation on the transformed image to obtain the transformed function;

[0031] The transformed function is subjected to threshold transformation processing, and based on the processing result, the eigenvalues ​​whose real part of the transformed function is greater than the second preset threshold are identified as six-degree-of-freedom perspective-invariant eigenvalues; or, the eigenvalues ​​whose imaginary part of the transformed function is greater than the third preset threshold are identified as six-degree-of-freedom perspective-invariant eigenvalues.

[0032] On the other hand, embodiments of the present invention also provide a six-degree-of-freedom perspective invariance feature analysis device, comprising:

[0033] The candidate corner point selection unit is used to perform corner point detection on the target image under test based on the corner point detection algorithm to obtain candidate corner points;

[0034] The adjacent four-corner point acquisition unit is used to process the candidate corner points based on the spatial segmentation algorithm to obtain the adjacent four-corner points;

[0035] The four-corner point sorting unit is used to sort adjacent four-corner points according to preset rules to obtain sorted four-corner points;

[0036] The homography processing unit is used to perform homography processing on the target image to be tested based on the sorted four corner points to obtain a transformed image;

[0037] The DFT transform unit is used to perform DFT transform processing on the transformed image to obtain six-degree-of-freedom perspective invariant eigenvalues.

[0038] On the other hand, embodiments of the present invention also provide a processing device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the six-degree-of-freedom perspective invariance feature analysis method described above.

[0039] In the embodiments of the above-mentioned six-degree-of-freedom perspective invariant feature analysis method, corner detection is performed on the target image to obtain candidate corner points based on a corner detection algorithm; the candidate corner points are processed based on a spatial segmentation algorithm to obtain adjacent four corner points; the adjacent four corner points are sorted according to a preset rule to obtain sorted four corner points; homography processing is performed on the target image to obtain a transformed image based on the sorted four corner points; DFT transformation is performed on the transformed image to obtain six-degree-of-freedom perspective invariant feature values ​​with high stability and feature resolution that can be used for target matching. This application performs corner detection on the target image to be tested using a corner detection algorithm. During six-degree-of-freedom transformation, the same corner points can always be detected, achieving strong six-degree-of-freedom transformation stability for the obtained corner points; by performing DFT transformation on the transformed image, the transformation speed for obtaining feature values ​​is improved; under actual testing conditions of six-degree-of-freedom transformation, the feature values ​​of the corresponding feature points can remain below a set percentage (e.g., 20%), improving the stability and resolution of the feature values. Attached Figure Description

[0040] Figure 1 This is a schematic diagram illustrating the application environment of a six-degree-of-freedom perspective invariance feature analysis method in one embodiment.

[0041] Figure 2 This is a schematic diagram of the first process of a six-degree-of-freedom perspective invariance feature analysis method in one embodiment;

[0042] Figure 3 This is a flowchart illustrating the corner detection steps in one embodiment;

[0043] Figure 4 This is a schematic diagram of the second process of a six-degree-of-freedom perspective invariance feature analysis method in one embodiment;

[0044] Figure 5 This is a flowchart illustrating the corner sorting process in one embodiment;

[0045] Figure 6 This is a flowchart illustrating the homography processing steps in one embodiment;

[0046] Figure 7 This is a schematic diagram of the third process of a six-degree-of-freedom perspective invariance feature analysis method in one embodiment;

[0047] Figure 8 This is a block diagram of a six-degree-of-freedom perspective invariance feature analysis device in one embodiment. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0049] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] In addition, the term "multiple" should mean two or more.

[0051] The six-degree-of-freedom perspective invariance characteristic analysis method provided in this application can be applied to, for example... Figure 1In the application environment shown, the processing device may include a processor 102 and a memory 104. The memory 104 can be used to store data such as candidate corner points, adjacent corner points, sorted corner points, transformed images, and six-degree-of-freedom perspective invariant feature values. The processor 102 can be used to perform corner point detection on the target image under test based on a corner point detection algorithm to obtain candidate corner points; process the candidate corner points based on a spatial segmentation algorithm to obtain adjacent corner points; sort the adjacent corner points according to a preset rule to obtain sorted corner points; perform homography processing on the target image under test based on the sorted corner points to obtain a transformed image; and perform DFT transformation processing on the transformed image to obtain six-degree-of-freedom perspective invariant feature values. The processing device may also include a display 106, which can display the processed target image, transformed image, and six-degree-of-freedom perspective invariant feature values ​​through a graphical interface. The processing device may be, but is not limited to, mobile robots such as sweeping robots, mopping robots, and cleaning robots.

[0052] To address the problems existing in current target matching, in one embodiment, such as Figure 2 As shown, a six-degree-of-freedom perspective invariance feature analysis method is provided, which is then applied to... Figure 1 Taking processor 102 as an example, the following steps are included:

[0053] Step S210: Perform corner detection on the target image to be tested based on the corner detection algorithm to obtain candidate corner points.

[0054] Corner detection algorithms can be categorized into grayscale image-based, binary image-based, and contour curve-based algorithms. For example, template-based corner detection algorithms may include, but are not limited to, the Kitchen-Rosenfeld, Harris, KLT, or Susan algorithms. Corners are crucial image features. For instance, template-based corner detection algorithms primarily consider grayscale variations in neighboring pixels, i.e., image brightness variations, defining points with sufficiently high brightness contrast to their neighbors as corners. It's important to note that this application uses the Harris corner detection algorithm as an example; other types of corner detection algorithms can also be used to detect corners in the target image, and will not be elaborated upon here.

[0055] For example, corner detection is performed on the target image under test based on the Harris corner detection algorithm to obtain candidate corners.

[0056] Step S220: Process the candidate corner points based on the spatial segmentation algorithm to obtain the four adjacent corner points.

[0057] The spatial segmentation algorithm divides the entire virtual space into equally sized cells, thereby segmenting objects in the scene into smaller groups, and only performs intersection tests on geometric objects occupying the same or adjacent cells. For example, the spatial segmentation algorithm can be, but is not limited to, octree and quadtree algorithms. It should be noted that this application uses the octree algorithm as an example; other types of spatial segmentation algorithms can also process candidate corner points to obtain adjacent four-corner points, and will not be elaborated upon here.

[0058] For example, an octree algorithm can be used to process the candidate corner points to obtain the four adjacent corner points. Here, the four adjacent corner points refer to four adjacent corner points.

[0059] Step S230: Sort the adjacent four corner points according to a preset rule to obtain the sorted four corner points.

[0060] In this arrangement, the four corner points of an adjacent set of four corner points are arranged in a circular pattern. The preset rule order can be, but is not limited to, a clockwise or counterclockwise order based on the brightness of the corner points. The four corner points after sorting refer to the four corner points arranged in a circular pattern.

[0061] For example, adjacent corner points can be sorted in a clockwise order based on their brightness to obtain the sorted corner points.

[0062] Step S240: Based on the sorted four corner points, homography processing is performed on the target image to be tested to obtain the transformed image.

[0063] Homography is defined as the mapping between two planar projections of an image. The homography transformation describes the positional mapping between an object in the world coordinate system and the pixel coordinate system; the corresponding transformation matrix is ​​called the homography matrix. The homography matrix constrains the 2D homogeneous coordinates of the same 3D point in two pixel planes.

[0064] For example, the target image to be tested is a 16*16 target image. Based on the sorted four corner points, the target image to be tested is subjected to homography processing to obtain a 16*16 transformed image.

[0065] Step S250: Perform DFT transformation on the transformed image to obtain six-degree-of-freedom perspective invariant eigenvalues.

[0066] The Discrete Fourier Transform (DFT) can be either a one-dimensional or two-dimensional DFT. Six degrees of freedom refer to the translational degrees of freedom along the x, y, and z Cartesian axes and the rotational degrees of freedom about the x, y, and z Cartesian axes. The six-degree-of-freedom perspective invariance characteristic is based on the perspective invariance characteristics of these six degrees of freedom.

[0067] For example, by performing DFT transformation on a 16*16 transformed image, 16*16 Boolean type feature values ​​are obtained, and these feature values ​​are used as the final feature values ​​of a 256-bit Boolean vector, which is the corresponding six-degree-of-freedom perspective invariant feature value.

[0068] In the above embodiments, corner detection is performed on the target image to obtain candidate corner points based on a corner detection algorithm; the candidate corner points are processed based on a spatial segmentation algorithm to obtain four adjacent corner points; the four adjacent corner points are sorted according to a preset rule to obtain sorted four corner points; homography processing is performed on the target image to obtain a transformed image based on the sorted four corner points; DFT transformation is performed on the transformed image to obtain six-degree-of-freedom perspective-invariant feature values ​​with high stability and feature resolution that can be used for target matching. This application uses a corner detection algorithm to detect corner points in the target image to obtain corner points. During six-degree-of-freedom transformation, the same corner points can always be detected, achieving strong six-degree-of-freedom transformation stability for the obtained corner points; by performing DFT transformation on the transformed image, the transformation speed for obtaining feature values ​​is improved; under actual testing conditions of six-degree-of-freedom transformation, the feature values ​​of the corresponding feature points can remain below a set percentage (e.g., 20%), improving the stability and resolution of the feature values.

[0069] In one embodiment, such as Figure 3 As shown, the steps for detecting corners in the target image based on the corner detection algorithm to obtain candidate corners include:

[0070] Step S310: Select the pixels of the target image to be tested, and confirm the window centered on the pixel as the selection window.

[0071] For example, if the selected pixel of the target image to be tested is (x, y), then a window centered on the pixel (x, y) is w(x, y), and w(x, y) is then confirmed as the selection window.

[0072] Step S320: Move the selected window along the first coordinate direction and the second coordinate direction on the target image to be tested to obtain the change in pixel value.

[0073] For example, the pixel value change is:

[0074] E(u, v) = ∑ (x，y) w(x,y)[I(x+u,y+v)-I(x,y)] 2 .

[0075] Where E(u, v) is the pixel value change, w(x, y) is the selection window, (x, y) is the pixel, u is the movement amount in the first coordinate direction, v is the movement amount in the second coordinate direction, and I(x, y) is the gray level corresponding to the (x, y) pixel. x and I y Let I(x, y) be the partial derivative. R is the corner response function, λ1 is the first orthogonal direction variation component, and λ2 is the second orthogonal direction variation component.

[0076] Step S330: Convert the pixel value change to obtain the corner response function of the corresponding preset window.

[0077] For example, when the selection window moves simultaneously in both the first coordinate direction (such as the x-axis direction) and the second coordinate direction (such as the y-axis direction), the change in pixel value E(u, v) inside the selection window is calculated, where u and v are the movement amounts in the x and y directions, respectively.

[0078] For example, the derivation process of E(u, v) is as follows: First, the selection window of the target image to be measured is shifted by (u, v) to generate the autocorrelation function of grayscale change (Equation 1):

[0079] E(u, v) = ∑ (x，y) w(x,y)[I(x+u,y+v)-I(x,y)] 2 .

[0080] Where I(x, y) is the gray level of the image at the pixel coordinates (x, y).

[0081] The translated expression is expanded using Taylor series and approximated as follows (Equation 2): I(x+u,y+v)≈I(x,y)+uI x +vI y .

[0082] Among them, I x and I y For the partial derivative of I(x, y), the gradient map in the target image, i.e., along the x-axis and y-axis directions:

[0083] Substituting equation two into equation one, we get:

[0084]

[0085] Wherein, matrix M is:

[0086] The above describes the diagonalization of a real symmetric matrix M, represented by eigenvalues ​​λ1, λ2, and corner response functions R (i.e., eigenvectors). R can be considered a rotation factor, which does not affect the variation components in the two orthogonal directions. After diagonalization, the variation components in the two orthogonal directions are extracted, namely λ1 and λ2. Matrix M is also called the Harris matrix. The width of w(x, y) determines the region of interest around the pixel (x, y). Because the eigenvalues ​​λ1 and λ2 determine the value of R, and R is only related to the eigenvalues ​​of M, the eigenvalues ​​can be used to determine whether a selection window is a plane, an edge, or a corner.

[0087] For example, when the selected window is a plane, the grayscale value within the selected window remains essentially unchanged as it slides across the flat area. Therefore, the |R| value is very small, with minimal changes in both the horizontal and vertical directions. x and I y If both are small, then λ1 and λ2 are also small; when the window is selected as the edge, the R value is negative, and there is only a large change in the horizontal or vertical direction, i.e., I x and I y There is only one larger value, i.e., λ1>>λ2 or λ2>>λ1; when the window is selected as a corner point, the R value is very large, and the point with large variations in both the horizontal and vertical directions is I. x and I y Both are relatively large, meaning that λ1 and λ2 are both very large.

[0088] It should be noted that for each pixel (x, y) in the target image to be tested, there is a corresponding window w(x, y) centered on that pixel. The pixel is translated (u, v) to obtain a new pixel (x+u, y+v), and E(u, v) is the weighted sum of all pixels in the selected window multiplied by the gray level difference of pixels at different positions.

[0089] Step S340: When the corner response function is greater than the first preset threshold, the pixel of the corresponding selected window is confirmed as a candidate corner.

[0090] Thresholding is applied to the corner response function. If R > the first preset threshold, it means that the selected window corresponds to a corner feature, and the pixel of the selected window is then identified as a candidate corner.

[0091] In the above embodiments, the Harris corner detection algorithm is used to detect corners in the target image under test. During the six-degree-of-freedom transformation, the same corners can always be detected, so the candidate corners have strong six-degree-of-freedom transformation stability.

[0092] In one embodiment, such as Figure 4As shown, a six-degree-of-freedom perspective invariance feature analysis method is provided, which is then applied to... Figure 1 Taking processor 102 as an example, the following steps are included:

[0093] Step S410: Perform corner detection on the target image to be tested based on the corner detection algorithm to obtain candidate corner points.

[0094] For a detailed explanation of step S410, please refer to the description of the above embodiments, which will not be repeated here.

[0095] Step S420: Process the candidate corner points based on the octree algorithm or the quadtree algorithm to obtain the adjacent four corner points.

[0096] An octree is a tree-like data structure used to describe three-dimensional space. Each node in an octree represents the volume element of a cube, and each node has eight child nodes. The sum of the volumes of the eight child nodes equals the volume of the parent node. A quadtree is a tree-like data structure where each node has four sub-blocks. Quadtrees are commonly used for the analysis and classification of two-dimensional spatial data. They divide the data into four quadrants. The data range can be square, rectangular, or any other arbitrary shape.

[0097] By processing the candidate corner points using the octree or quadtree algorithm, four adjacent corner points can be extracted from the candidate corner point, thus obtaining the four adjacent corner points.

[0098] For example, let's illustrate the process of using an octree algorithm to process candidate corner points. In a complete octree, each intermediate node has eight child nodes, all leaf nodes have the same tree depth D, and the number of leaf nodes is 8 raised to the power of D. The specific process of searching for the target coordinates to be N adjacent nodes using an octree is as follows:

[0099] Step 1: Search for the target node L, which has child nodes containing non-empty nodes, denoted as Branch. L,，i Each child node has a surrounding square. Let the target coordinates be Target.

[0100] Step 2: Determine which node's enclosing square the Target falls within, and denote it as the central enclosing square. Let R be the distance from the center point of the central enclosing square to its corner point. Using the center of the central enclosing square as the center, draw a sphere with radius R. Check if the total number of non-empty leaf nodes in the enclosing squares intersecting with this sphere is greater than N. If not, draw a sphere with radius 2R and check if the total number of non-empty leaf nodes in the enclosing squares intersecting with this sphere is greater than N. Continue this process until the smallest range containing all child nodes is found, denoted as Set.

[0101] Step 3: For the found Set range, perform Step 2 on the child nodes of each of its child nodes. Then perform Step 2 on the newly found Set, and so on, until the final N leaf nodes adjacent to the target are found.

[0102] Step S430: Sort the adjacent four corner points according to a preset rule to obtain the sorted four corner points.

[0103] For a detailed explanation of step S430, please refer to the description of the above embodiments, which will not be repeated here.

[0104] Step S440: Based on the sorted four corner points, homography processing is performed on the target image to be tested to obtain the transformed image.

[0105] For a detailed explanation of step S440, please refer to the description of the above embodiments, which will not be repeated here.

[0106] Step S450: Perform DFT transformation on the transformed image to obtain six-degree-of-freedom perspective invariant eigenvalues.

[0107] For a detailed explanation of step S450, please refer to the description of the above embodiments, which will not be repeated here.

[0108] In the above embodiments, corner detection algorithms are used to detect corners in the target image under test. During six-degree-of-freedom transformation, the same corners can always be detected, resulting in corners with strong six-degree-of-freedom transformation stability. Candidate corners are processed using octree or quadtree algorithms to obtain adjacent four-corner points. These adjacent four-corner points are then sorted according to a preset rule to obtain sorted four-corner points. Based on the sorted four-corner points, homography processing is performed on the target image under test to obtain a transformed image. DFT transformation is performed on the transformed image to improve the transformation speed for obtaining feature values. In actual six-degree-of-freedom transformation tests, the feature values ​​of the corresponding feature points can remain below a set percentage (e.g., 20%), improving the stability and resolution of the feature values.

[0109] In one embodiment, such as Figure 5 As shown, the steps to sort adjacent corner points according to a preset rule to obtain the sorted corner points include:

[0110] Step S510: The corner point with the highest brightness among the four adjacent corner points is identified as the main point, and the average coordinate of the four adjacent corner points is identified as the center point.

[0111] Step S520: Using the principal angle as the zero direction, sort the remaining three corner points among the adjacent four corner points according to the first order to obtain the sorted four corner points.

[0112] Specifically, for each of the four adjacent corner points, the corner point with the highest brightness is selected as the main point. The average coordinate of the four adjacent corner points is used as the center point. With the main point as the zero direction, the other three corner points among the four adjacent corner points are sorted clockwise, thus obtaining the four ordered corner points, which are the sorted four corner points.

[0113] For example, let's define the four corner points of the four adjacent corner points as C... i i = 1 to 4, set the center point coordinates as (M x M y ),but

[0114] Let the coordinates of the four corner points of the adjacent four corner points be (C i，x C i，y atan2(y, x) is the arctangent function that transforms a planar vector into radians, and the angle between the corner point and the center point is set to C. i，o Then: C i，o =atan2(C i，y One M y C i，x One M x ).

[0115] Let the target image be Image(x, y), and the protagonist point be C. M The Max(Image(x, y)) function is used to find the corner point with the maximum brightness of Image(x, y). Therefore:

[0116] MAx(Image(C i，x C i，y ))=C M .

[0117] Set sort() as the sorting function, C M，o Let C(j) be the angle value of the main point, and C(j) be the sorted corner point. Then: C(j) = sort(Cj) i，o One C M，o ).

[0118] In one embodiment, such as Figure 6 As shown, the steps for performing homography processing on the target image to obtain the transformed image based on the sorted four corner points include:

[0119] Step S610: Based on the coordinates of the four corner points after sorting, homography processing is performed on the coordinates of the four corner points of the target image to be tested to obtain the homography matrix.

[0120] Specifically, based on the coordinates of the four clockwise corner points, homography processing is performed on the coordinates of the four clockwise corner points of the 16*16 target image to obtain the homography matrix.

[0121] For example, define a 3x3 square matrix. Then homography transformation can be expressed as:

[0122] Where (x, y) are the input coordinates and (x′, y′) are the output coordinates.

[0123] Step S620: Based on the homography matrix, perform homography transformation on the target image to be tested to obtain the transformed image.

[0124] By performing homography transformation on the 16*16 target image under test according to the obtained homography matrix, a 16*16 transformed image is obtained.

[0125] In one embodiment, such as Figure 7 As shown, a six-degree-of-freedom perspective invariance feature analysis method is provided, which is then applied to... Figure 1 Taking processor 102 as an example, the following steps are included:

[0126] Step S710: Perform corner detection on the target image to be tested based on the corner detection algorithm to obtain candidate corner points.

[0127] For a detailed explanation of step S710, please refer to the description of the above embodiments, which will not be repeated here.

[0128] Step S720: Process the candidate corner points based on the spatial segmentation algorithm to obtain the four adjacent corner points.

[0129] For a detailed explanation of step S720, please refer to the description of the above embodiments, which will not be repeated here.

[0130] Step S730: Sort the adjacent four corner points according to a preset rule to obtain the sorted four corner points.

[0131] For a detailed explanation of step S730, please refer to the description of the above embodiments, which will not be repeated here.

[0132] Step S740: Based on the sorted four corner points, homography processing is performed on the target image to be tested to obtain the transformed image.

[0133] For a detailed explanation of step S740, please refer to the description of the above embodiments, which will not be repeated here.

[0134] Step S750: Perform one-dimensional DFT or two-dimensional DFT transformation on the transformed image to obtain six-degree-of-freedom perspective invariant eigenvalues.

[0135] For example, a one-dimensional DFT transformation is:

[0136]

[0137] Where f(x) is the original function and F(u) is the Fourier series after DFT transformation.

[0138] If we express the above process in the form of matrix calculation, then:

[0139]

[0140] Similar to the one-dimensional DFT, the following two-dimensional discrete Fourier transform pair can also be obtained. Here, f(x, y) is the two-dimensional antiderivative, and F(u, v) is the transformed two-dimensional Fourier series.

[0141]

[0142] Simplifying this transformation, we get:

[0143]

[0144] In one example, the steps for performing a DFT transform on a transformed image to obtain six-DOF perspective-invariant eigenvalues ​​include:

[0145] Perform DFT transformation on the transformed image to obtain the transformed function;

[0146] The transformed function is subjected to threshold transformation processing, and based on the processing result, the eigenvalues ​​whose real part of the transformed function is greater than the second preset threshold are identified as six-degree-of-freedom perspective-invariant eigenvalues; or, the eigenvalues ​​whose imaginary part of the transformed function is greater than the third preset threshold are identified as six-degree-of-freedom perspective-invariant eigenvalues.

[0147] The process involves performing a DFT transformation on the transformed image to obtain the transformed function; then, by performing a threshold transformation on the transformed function, such as performing a greater-than-zero transformation on the real or imaginary part of the transformed function, 16*16 Boolean type eigenvalues ​​are obtained. These eigenvalues ​​are then used as the final eigenvalues ​​of a 256-bit Boolean vector, thus yielding the six-degree-of-freedom perspective invariant eigenvalues.

[0148] In the above embodiments, corner detection is performed on the target image to be tested based on a corner detection algorithm to obtain candidate corner points; the candidate corner points are processed based on a spatial segmentation algorithm to obtain adjacent four corner points; the adjacent four corner points are sorted according to a preset rule to obtain sorted four corner points; based on the sorted four corner points, homography processing is performed on the target image to be tested to obtain a transformed image; the transformed image is processed by a one-dimensional DFT or two-dimensional DFT, and the real part of the transformed function is processed to be greater than zero to obtain 16*16 Boolean type feature values, which are used as the final feature values ​​of a 256-bit Boolean vector, thereby obtaining six-degree-of-freedom perspective invariant feature values ​​with high stability and feature resolution that can be used for target matching. This application uses a corner detection algorithm to detect corners in the target image under test. During six-degree-of-freedom transformation, the same corners can always be detected, resulting in corners with strong six-degree-of-freedom transformation stability. By performing DFT transformation on the transformed image, the transformation speed of obtaining feature values ​​is improved. Under the actual test of six-degree-of-freedom transformation, the feature values ​​of the corresponding feature points can remain below a set percentage (e.g., 20%), improving the stability and resolution of feature values.

[0149] It should be noted that during object detection, each feature correspondence yields a six-degree-of-freedom transformation matrix, ensuring that the features are not merely point-to-point but face-to-face, thus providing additional cue parameters. Furthermore, since ORB features are also 256-bit Boolean feature values, the six-degree-of-freedom invariant features proposed in this application can be embedded into any ORB series algorithm.

[0150] In one example, the specific verification process of the six-degree-of-freedom perspective invariance feature analysis method is as follows: In order to verify the effectiveness of the algorithm of this application, a verification test was conducted on the same target under random parameter perspective changes.

[0151] Using a single image, a homography transformation is performed according to random parameters to extract and compare the six-degree-of-freedom invariant feature values ​​of the corresponding positions of random targets. All feature extraction results are categorized into two cases: stable corner points applicable to this application (where Harris corner detection can determine stable corner point positions); and other cases (where at least one of the four corner points appears in other locations).

[0152] With stable corner points, the feature value remains stable within 20% fluctuation, indicating strong stability of this feature. It should be noted that other situations may occur during testing because different perspective transformations cause variations in the sharpness of the corner point edges, making the corner point position unstable.

[0153] It should be understood that, although Figures 2-7The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2-7 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0154] In one embodiment, such as Figure 8 As shown, a six-degree-of-freedom perspective invariance feature analysis device is also provided, comprising:

[0155] The candidate corner point selection unit 810 is used to perform corner point detection on the target image under test based on the corner point detection algorithm to obtain candidate corner points.

[0156] The adjacent four-corner point acquisition unit 820 is used to process the candidate corner points based on the spatial segmentation algorithm to obtain the adjacent four-corner points.

[0157] The four-corner point sorting unit 830 is used to sort adjacent four-corner points according to preset rules to obtain sorted four-corner points.

[0158] The homography processing unit 840 is used to perform homography processing on the target image to be tested based on the sorted four corner points to obtain a transformed image.

[0159] The DFT transform unit 850 is used to perform DFT transform processing on the transformed image to obtain six-degree-of-freedom perspective invariant eigenvalues.

[0160] Specific limitations regarding the six-degree-of-freedom perspective invariance feature analysis device can be found in the limitations of the six-degree-of-freedom perspective invariance feature analysis method described above, and will not be repeated here. Each module in the aforementioned six-degree-of-freedom perspective invariance feature analysis device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the controller in the processing device, or stored in software in the memory of the processing device, so that the controller can call and execute the corresponding operations of each module.

[0161] In one embodiment, a processing device is also provided, including a processor and a memory, the memory storing a computer program, the processor executing the computer program to implement the six-degree-of-freedom perspective invariant feature analysis method described above.

[0162] The processing equipment may include, but is not limited to, mobile robots such as sweeping robots, mopping robots, and cleaning robots.

[0163] The processor is used to perform the following steps of a six-degree-of-freedom perspective-invariant eigenvalue analysis method:

[0164] The corner detection algorithm is used to detect corners in the target image to obtain candidate corners. The candidate corners are then processed using a spatial segmentation algorithm to obtain four adjacent corners. These four adjacent corners are then sorted according to a preset rule to obtain sorted corners. Based on the sorted corners, homography processing is performed on the target image to obtain a transformed image. The transformed image is then subjected to DFT transformation to obtain six-degree-of-freedom perspective-invariant feature values ​​with high stability and feature resolution that can be used for target matching.

[0165] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the six-degree-of-freedom perspective invariant feature analysis method described above.

[0166] In one example, when a computer program is executed by a processor, it performs the following steps:

[0167] The corner detection algorithm is used to detect corners in the target image to obtain candidate corners. The candidate corners are then processed using a spatial segmentation algorithm to obtain four adjacent corners. These four adjacent corners are then sorted according to a preset rule to obtain sorted corners. Based on the sorted corners, homography processing is performed on the target image to obtain a transformed image. The transformed image is then subjected to DFT transformation to obtain six-degree-of-freedom perspective-invariant feature values ​​with high stability and feature resolution that can be used for target matching.

[0168] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0169] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for analyzing six-degree-of-freedom perspective invariance features, characterized in that, Includes the following steps: Corner detection algorithms are used to detect corners in the target image to obtain candidate corners. The candidate corner points are processed based on a spatial segmentation algorithm to obtain four adjacent corner points; The adjacent four corner points are sorted according to a preset rule to obtain the sorted four corner points; Based on the sorted four corner points, homography processing is performed on the target image to be tested to obtain a transformed image; The transformed image is subjected to DFT transformation to obtain six-DOF perspective invariant eigenvalues. The steps include: Perform a one-dimensional DFT or two-dimensional DFT transformation on the transformed image to obtain the transformed function; The transformed function is subjected to threshold transformation processing, and based on the processing result, the feature value whose real part of the transformed function is greater than a second preset threshold is identified as the six-degree-of-freedom perspective invariant feature value; or, the feature value whose imaginary part of the transformed function is greater than a third preset threshold is identified as the six-degree-of-freedom perspective invariant feature value.

2. The six-degree-of-freedom perspective invariance feature analysis method according to claim 1, characterized in that, The step of performing homography processing on the target image to be tested based on the sorted four corner points to obtain the transformed image includes: Based on the coordinates of the four corner points after sorting, homography processing is performed on the coordinates of the four corner points of the target image to be tested to obtain a homography matrix; Based on the homography matrix, the target image to be tested is subjected to homography transformation processing to obtain the transformed image.

3. The six-degree-of-freedom perspective invariance feature analysis method according to claim 1, characterized in that, The step of sorting the adjacent four corner points according to a preset rule to obtain the sorted four corner points includes: The corner point with the highest brightness among the four adjacent corner points is identified as the main point, and the average coordinate of the four adjacent corner points is identified as the center point. Taking the main point as the zero direction, the remaining three corner points among the adjacent four corner points are sorted according to the first order to obtain the sorted four corner points.

4. The six-degree-of-freedom perspective invariance feature analysis method according to claim 1, characterized in that, The steps for detecting corners in the target image based on the corner detection algorithm to obtain candidate corners include: Select a pixel in the target image to be tested, and confirm the window centered on the pixel as the selection window; The selected window is moved along the first coordinate direction and the second coordinate direction on the target image to be tested to obtain the change in pixel value; The pixel value change is converted to obtain the corner response function corresponding to the selected window; When the corner response function is greater than a first preset threshold, the pixel corresponding to the selected window is identified as the candidate corner.

5. The six-degree-of-freedom perspective invariance feature analysis method according to claim 4, characterized in that, The change in pixel value is: Where E(u, v) is the pixel value change, w(x, y) is the selection window, (x, y) is the pixel, u is the movement amount in the first coordinate direction, v is the movement amount in the second coordinate direction, and I(x, y) is the gray level corresponding to the (x, y) pixel. x and I y Let I(x, y) be the partial derivative. R is the corner response function, λ1 is the first orthogonal direction variation component, and λ2 is the second orthogonal direction variation component.

6. The six-degree-of-freedom perspective invariance feature analysis method according to claim 1, characterized in that, The step of processing the candidate corner points based on the spatial segmentation algorithm to obtain the adjacent four corner points includes: The candidate corner points are processed using an octree algorithm or a quadtree algorithm to obtain the adjacent four corner points.

Images