Guided data sampling method and system based on neighborhood consistency and residual ordering

Abstract

The application relates to a guiding data sampling method based on neighborhood consistency and residual ordering, which comprises the following steps: step S1: randomly selecting a data from input data, and then judging whether the data point is an inner point by calculating neighborhood consistency information of the selected data point; if the selected data point is an inner point, the data point is selected as a seed data, otherwise, step S1 is repeatedly executed until an inner point is selected or the maximum number of repetitions is reached, if no inner point is found after the above steps are iteratively executed, a data point is randomly selected as the seed data; step S2: after the seed data is selected, the sampling weight of the seed data and input data is calculated through residual index; and step S3: other data of the data subset is selected from the input data except the seed data by using the sampling weight. The application can be used in robust model fitting, and thus can be applied to computer vision tasks such as video coding transform estimation, augmented reality, 3D rigid registration and plane detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a guidance data sampling method and system based on neighborhood consistency and residual ranking. BACKGROUND

[0002] Robust model fitting is a fundamental task in artificial intelligence and has been used in various artificial intelligence applications, such as video coding motion estimation, augmented reality, 3D rigid registration, and plane detection. The purpose of robust model fitting is to fit the geometric model hypothesis in the data and further recover meaningful structures, such as object shapes and moving objects. The data is extracted from the input image and contains outliers and noise. The success of robust model fitting techniques depends on the accurate hypothesis generated by its sampling method. Most sampling methods are committed to sampling a clean minimal subset to generate a more accurate model hypothesis.

[0003] Residual ranking-based sampling algorithms are one of the most promising guidance sampling algorithms because they are good at finding inliers belonging to the same model instance (i.e., structure). However, it is relatively time-consuming to calculate the sampling probability from residual ranking, and most existing residual ranking-based sampling algorithms need to calculate the sampling probability of all input data p-1 times to sample a minimal subset including p data points. Recently, we proposed an improved algorithm (i.e., AGS) that only calculates the sampling probability once for sampling a minimal subset. Specifically, AGS first randomly selects a data point from the input data as most existing residual ranking-based sampling algorithms do, then selects meaningful data by using the sampling probability calculated by residual ranking and information theory principles, and finally samples a minimal subset from the selected meaningful data. However, if the first data selected by these sampling algorithms is an outlier, the sampled data subset will be invalid. At the same time, the higher the proportion of outliers contained in the data, the more likely it is that the first randomly selected data is an outlier. Therefore, these algorithms cannot effectively handle multi-structure data containing a high proportion of outliers. SUMMARY

[0004] Therefore, the present application aims to provide a guidance data sampling method based on neighborhood consistency and residual ranking, which can be used in robust model fitting, and thus applied to computer vision tasks such as video coding motion estimation, augmented reality, 3D rigid registration, and plane detection.

[0005] To achieve the above object, the present application adopts the following technical scheme:

[0006] A guidance data sampling method based on neighborhood consistency and residual ranking comprises the following steps:

[0007] Step S1: randomly select a data from the input data, and then determine whether the selected data is an inlier by calculating the neighborhood consistency information of the selected data point; if the selected data is an inlier, select the data as a seed data, otherwise, repeat the step S1 until an inlier is selected or the maximum number of iterations is reached, if no inlier is found after the iteration of the above steps, randomly select a data point as a seed data;

[0008] Step S2: after the seed data is selected, the sampling weight of the seed data and the input data is calculated by residual index;

[0009] Step S3: other data of the data subset is selected from the input data except the seed data by using the sampling weight.

[0010] Further, the step S1 is specifically:

[0011] (1): the input data X, the sampling number Z, the initial model hypothesis number L, the Markov chain Monte Carlo parameter β, the seed data selection maximum number G and the batch size b are given.

[0012] (2): determine whether the current sampling number τ is less than or equal to Z, if less, execute steps (3) to (8); otherwise, end the program.

[0013] (3): determine whether the current sampling number τ is less than L, if less, execute step (4); otherwise, execute step (5);

[0014] (4): a minimum data subset S is sampled from the input data by using a random sampling algorithm τ ;

[0015] (5): determine whether the generated random number is less than β, if less, execute step (6); otherwise, execute step (7);

[0016] (6): the parameter G and the neighborhood consistency information are used to try to select the first data of the data subset, i.e. the seed data;

[0017] (7): determine whether the seed data is found, if not, execute step (8);

[0018] (8): randomly select a data as the seed data.

[0019] Further, the step (6) is specifically:

[0020] Step S61: a data x i ={u i ,v i} is randomly selected from the input data;

[0021] Step S62: judging x with formula (1) i whether it is an inlier

[0022]

[0023] where α is the number of scales, K j is the number of nearest neighbors, denotes the index of the nearest K i nearest neighbors of feature point u j , and denotes the number of same nearest neighbors of feature points u i and v i . If p i is greater than a specified threshold, x i is determined to be an inlier; otherwise, it is an outlier;

[0024] Step S63: if x i is an inlier, set x i as a seed data and end the subprogram; otherwise, re-execute steps S61 to S63.

[0025] Further, the residual index, specifically:

[0026] (1) the residual vector of the ith data point x i and the N model hypotheses contained in Θ is denoted as r , where is the residual of x i and the jth model hypothesis;

[0027] (2) the elements in r i are arranged in non-decreasing order to obtain the residual index vector

[0028] (3) the correlation between different data is calculated from the residual index vector, the greater the correlation value, the more likely to come from the same structure, the correlation between x i and x j is calculated as follows where denotes the first h elements in κ i , and denotes the same index number of and .

[0029] A guidance data sampling system based on neighborhood consistency and residual sorting includes a processor, a memory, and a computer program stored on the memory, wherein the processor executes the computer program to specifically execute the steps in the guidance data sampling method based on neighborhood consistency and residual sorting as described above.

[0030] Compared with the prior art, the present application has the following advantages:

[0031] 1. The present application effectively combines the advantages of residual ordering and neighborhood consistency information to perform efficient data sampling.

[0032] 2. The sampling method of the present application can be used in robust model fitting, thereby being applied to computer vision tasks such as video coding transform estimation, augmented reality, 3D rigid registration and plane detection. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is an algorithm flowchart in an embodiment of the present application.

[0034] Figure 2 is an example of three-dimensional moving object segmentation using the method of the present embodiment. DETAILED DESCRIPTION

[0035] The present application will be further described below in conjunction with the drawings and embodiments.

[0036] Please refer to Figures 1-2 The present application provides a guidance data sampling method based on neighborhood consistency and residual ordering, comprising the following steps:

[0037] Step S1: randomly select a data from the input data, and then determine whether the selected data point is an inlier by calculating the neighborhood consistency information of the selected data point; if the selected data point is an inlier, select the data point as a seed data, otherwise, repeat step S1 until an inlier is selected or the maximum number of iterations is reached; if no inlier is found after the above steps are iteratively performed, randomly select a data point as a seed data.

[0038] Step S2: after the seed data is selected, calculate the sampling weight of the seed data and the input data by residual index.

[0039] Step S3: use the sampling weight to select other data of the data subset from the input data except the seed data.

[0040] Reference Figure 1 In the present embodiment, the guidance data sampling method based on neighborhood consistency and residual ordering is specifically:

[0041] (1) Given input data X, sampling number Z, number of initial model hypotheses L, Markov chain Monte Carlo parameter β, maximum number of seed data selection G and batch size b.

[0042] (2): Determine whether the current sampling number τ is less than or equal to Z. If it is less than Z, then execute steps (3) to (8); otherwise, end the program.

[0043] (3): Determine whether the current sampling number τ is less than L. If it is less, then execute step (4); otherwise, execute step (5).

[0044] (4): Use a random sampling algorithm to sample a minimum data subset S from the input data. τ ;

[0045] (5): Determine whether the generated random number is less than β. If it is less, then execute step (6); otherwise, execute step (7).

[0046] (6): Use parameter G and domain consistency information to attempt to select the first data in the data subset, i.e., the seed data;

[0047] (7): Determine if seed data has been found. If not, proceed to step (8).

[0048] (8): Randomly select one data point as the seed data;

[0049] (9): Sample a subset S of data using seed data and a residual sorting-based sampling algorithm (such as AGS). τ .

[0050] (10): Using S τ Generate a model hypothesis θ τ And make Θ=Θ∪{θ τ};

[0051] (11): If τ is greater than or equal to L and τ divides b, then execute (12);

[0052] (12): Update the residual index.

[0053] Preferably, in this embodiment, step (6) specifically includes:

[0054] Step S61: Randomly select a data point x from the input data. i ={u i ,v i}

[0055] Step S62: Use formula (I) to determine x i Is it an interior point?

[0056]

[0057] Where α is the number of scales, K j It is the number of neighbors. Representation and feature point ui Recent K j index of neighbors, while representing feature point u i and v i of the same number of neighbors. If p i is greater than a specified threshold, x i is determined to be an inlier; otherwise, an outlier.

[0058] Step S63: If x i is an inlier, set x i as a seed data and end the subroutine; otherwise, re-execute steps S61 to S63.

[0059] In the present embodiment, the residual index, specifically:

[0060] (1) The residual vector of the ith data point x i and the N model hypotheses contained in Θ is denoted as wherein is the residual of x i and the jth model hypothesis;

[0061] (2) The elements in r i are arranged in non-decreasing order to obtain the residual index vector

[0062] (3) The correlation between different data is calculated from the residual index vector, and the greater the correlation value, the more likely it comes from the same structure. The correlation between x i and x j is calculated as follows wherein denotes the first h elements in κ i , and denotes the same index number of and .

[0063] Referring to Figure 2 , in the present embodiment, four scene images in Figure 2 are respectively taken as input image data of the method of the present embodiment, and after the model parameters outputted after the steps of the present embodiment are fitted, the data in the images are segmented into points belonging to different moving objects, and the points of different objects are marked with different shapes.

[0064] The above only describes the preferred embodiments of the present application, and any equivalent changes and modifications made within the scope of the patent application of the present application shall be covered by the present application.

Claims

1. A method for guided data sampling based on neighborhood agreement and residual ordering, the method comprising: The method is applied to computer vision tasks including video coding transform estimation, augmented reality, 3D rigid registration and plane detection; the method comprises the following steps: Step S1: randomly selecting a data from input image data, and then judging whether the selected data point is an inlier by calculating the neighborhood consistency information of the selected data point; if the selected data point is an inlier, the selected data point is selected as a seed data, otherwise, step S1 is repeatedly executed until an inlier is selected or the maximum number of iterations is reached, if no inlier is found after the above steps are iteratively executed, a data point is randomly selected as a seed data; Step S2: after the seed data is selected, the sampling weight of the seed data and the input data is calculated by residual index; Step S3: using the sampling weight to select other data of the data subset from the input data except the seed data; The residual index is specifically: (1) the ith data point x i The residual vector of x where is the residual of x i with the jth model hypothesis; (2) arranging the elements in r in non-decreasing order to obtain a residual index vector i ​ (3) Compute the correlation between different data from the residual index vector, the larger the correlation value, the more likely to come from the same structure, x i Correlation with x j is calculated as follows where denotes the first h elements in K i , and denotes and the same index number.

2. The guided data sampling method based on neighborhood consensus and residual ordering of claim 1, wherein, The step S1 is specifically: (1) giving input data X, sampling number Z, initial model hypothesis number L, Markov chain Monte Carlo parameter β, seed data selection maximum number G and batch size b; (2) judging whether the current sampling number τ is less than or equal to Z, if less, steps (3) to (8) are executed; otherwise, the program is ended; (3) judging whether the current sampling number τ is less than L, if less, step (4) is executed; otherwise, step (5) is executed; (4): sampling a minimal data subset S from the input data using a random sampling algorithm τ ; (5) judging whether the generated random number is less than β, if less, step (6) is executed; otherwise, step (7) is executed; (6) using parameters G and neighborhood consistency information to try to select the first data of the data subset, i.e. the seed data; (7) judging whether the seed data is found, if not, step (8) is executed; (8) randomly selecting a data as the seed data.

3. The guided data sampling method based on neighborhood consensus and residual ordering of claim 2, wherein, The step (6) is specifically: Step S61 : randomly select one data x from the input data i = {u i , v i}; Step S62: Determine x with formula (1) i whether it is an interior point where a is the number of dimensions, K j is the number of nearest neighbors, denotes the same feature point u i of the K j nearest neighbors, and denotes the same feature point u i and v i ; If p i is greater than a specified threshold, determine x i is an inlier; otherwise, an outlier. Step S63: If x i is an interior point, set x i as the seed data and end the subroutine; otherwise, re-execute steps S61 to S63.

4. A guided data sampling system based on neighborhood agreement and residual ordering, characterized by, The device comprises a processor, a memory and a computer program stored in the memory, and when the processor executes the computer program, the steps of the guidance data sampling method based on neighborhood consistency and residual ordering in any one of claims 1-3 are specifically executed.

Images