A feature description method based on illumination invariant scale space

Abstract

The present invention discloses a feature description method based on illumination invariant scale space, which belongs to the field of computer vision technology and includes the following steps: S1, extracting illumination component T from input image; S2, calculating illumination invariant component R of input image; S3, constructing illumination invariant scale space; S4, detecting and screening extreme points in illumination invariant scale space to obtain feature points; S5, constructing a descriptor for each feature point in step S4. The present invention can extract sufficient number of highly robust features from low-illuminance images for subsequent matching, while improving the number and accuracy of matching. The precise positioning and matching of these features between views increases the number of reconstruction points for sparse reconstruction of spatial targets and improves point cloud coverage.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically, to a feature description method based on illumination invariant scale space. Background Art

[0002] In the aerospace field of the 21st century, the study of three-dimensional reconstruction is also of particular significance. With the development of space technology, space has become a valuable resource that countries compete for. my country's space station has begun to take shape, and more and more spacecraft are entering space to perform different tasks. With this comes higher requirements for space on-orbit services and operation technologies. In order to improve the accuracy of remote operation of space target objects, it is necessary to obtain the three-dimensional structure information of the target before performing the mission.

[0003] Compared with the active method of obtaining the three-dimensional structure information of space targets using devices such as laser radar, the passive method of three-dimensional reconstruction of space targets based on sequence images collected by devices such as cameras has been more widely used due to its various advantages. The passive method is small in size, high in efficiency, low in platform resource requirements, long in life, good in stability, and high in precision. It can not only restore the shape and size of space targets, but also estimate their relative position and attitude information. The technology is mature, with higher flexibility and wide practicality.

[0004] Robust feature description construction is the core issue of 3D reconstruction of space targets based on monocular multi-view sequence images. Although many excellent feature description algorithms are effective in actual applications on the ground, they cannot be well applied in space. The low illumination of the space environment leads to concentrated grayscale distribution of the target image and unclear image features. In addition, low illumination will also blur some texture details of space targets, such as solar panels and metal foils. These problems make it impossible for feature detection and description algorithms to effectively extract features, which brings difficulties to subsequent feature matching and pose estimation.

[0005] At present, image gradient-based feature description algorithms are often used in ground 3D reconstruction applications for image feature extraction. The most famous one is the scale-invariant feature transform (SIFT) algorithm. Image gradient-based feature description algorithms usually find local response extreme points by image grayscale difference or differential operation, and construct descriptors through local image gradient information. After the descriptor is formed, vector normalization is often performed to eliminate the impact of illumination changes on image pixels. The advantage of this image differentiation and vector normalization operation mode is that it is insensitive to the stretching and contraction of image grayscale and the increase and decrease of brightness. In other words, this method is only robust to affine transformations of image brightness and contrast, but complex brightness changes are difficult to avoid in space. Due to changes in the properties and position of the light source (relative to the object reflecting the light source), the geometry of the space target, and the perspective, the same physical point may have different pixel intensities. When an object moves in and out of the shadow, the brightness may change suddenly between frames. In addition, changes in capture parameters such as gain and exposure time, coupled with nonlinear processing of pixel intensity in the imaging pipeline, cause the brightness to undergo complex nonlinear changes. Existing image gradient-based feature description algorithms are simply unable to cope with these problems. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a feature description method based on illumination invariant scale space to solve the problem in the background, so as to extract sufficient and robust features in the low-light environment of space for subsequent applications.

[0007] The object of the present invention is achieved through the following solutions:

[0008] A feature description method based on illumination invariant scale space comprises the following steps:

[0009] S1, extracting the illumination component T of the input image;

[0010] S2, calculate the illumination invariant component R of the input image;

[0011] S3, constructing an illumination invariant scale space;

[0012] S4, detecting and screening extreme points in the illumination invariant scale space to obtain feature points;

[0013] S5. Construct a descriptor for each feature point in step S4.

[0014] Furthermore, the light component T in step S1 is extracted by the following calculation equation:

[0015]

[0016] in, represents the minimum function with independent variable T, represents the initial estimate of the illumination component T, represents the F-norm; is the coefficient; x represents the pixel on the image; d represents the direction of solving the differential, including horizontal and vertical directions, where h represents the horizontal direction and v represents the vertical direction; is the first-order differential of T, is a positive number, W is the weight matrix, c represents the RGB channels of the image, max represents the maximum value function, and I represents the input image.

[0017] Furthermore, the calculation equation of the illumination invariant component R described in step S2 is:

[0018]

[0019] in, is a positive number and I is the input image.

[0020] Furthermore, the illumination invariant scale space described in step S3 is constructed by the following formula:

[0021]

[0022] Where k is a constant coefficient, (x, y) represents the coordinates of the pixel, and * represents the convolution operation. is the Gaussian filter function with varying scale; is the scale-variable illumination invariant component obtained in step S2; is the scale change factor, expressed as:

[0023]

[0024] In the formula, is a constant, is the base layer scale factor, o is the octave group index, s is the layer index within the group, and S represents the number of images contained in each pyramid group.

[0025] Furthermore, step S4 includes the following sub-steps:

[0026] Traverse the pixel points of each layer in the scale space, compare them with the 8 neighbors in the neighborhood and the 9 neighbors in the upper and lower scale layers, and select the largest or smallest point as the extreme point; filter out low-contrast points with an absolute value of the scale space function less than 0.03 and points with excessive edge response, and finally retain the points as feature points.

[0027] Further, step S5 includes the following sub-steps: constructing a descriptor for each feature point, and for the feature point detected in the IIS space, collecting the gradient magnitude and direction distribution characteristics of the pixels in the neighborhood window of the IIS pyramid image where the feature point is located; the calculation formula of the gradient magnitude and direction is as follows:

[0028]

[0029] in, represents the gradient amplitude, Indicates the gradient direction.

[0030] Furthermore, the weight matrix W is set as:

[0031]

[0032] in, represents the image pixel coordinates, Represents the Gaussian blur template scope, and t represents the pixels in the scope; is a Gaussian kernel function with standard deviation σ;

[0033] It is expressed as: ;

[0034] represents the weight matrix in the horizontal direction in step S2; represents the weight matrix in the vertical direction in step S2; dist() represents the Euclidean distance weighting function.

[0035] Further, for the setting of parameters: , , , , that is, the generated IIS scale space has 5 groups, each group has 7 layers, and the images of the next group are obtained by downsampling the images of the previous group.

[0036] Furthermore, the step of collecting the gradient magnitude and direction distribution characteristics of pixels in the neighborhood window of the IIS pyramid image includes the following sub-steps:

[0037] The histogram is used to count the gradient and direction of pixels in the neighborhood of the feature point. The histogram divides 360 degrees into 36 intervals, each interval is 10 degrees, and the gradient amplitude in each interval is counted. The peak direction of the histogram represents the main direction of the feature point.

[0038] The direction with a peak value greater than 80% of the main direction peak is retained as the auxiliary direction of the feature point. The descriptor uses the gradient information of 8 directions calculated in a 4*4 window in the feature point scale space, with a total of 4*4*8=128-dimensional vector representation.

[0039] Further, for the setting of parameters: , , is a variable scale, and its expression is:

[0040]

[0041] In the formula, , , , .

[0042] The beneficial effects of the present invention include:

[0043] The present invention proposes a feature description method based on illumination invariant scale space, which can extract sufficient and robust features in low-illuminance images for subsequent matching, while improving the number and accuracy of matches. The precise positioning and matching of these features between views increases the number of reconstruction points for sparse reconstruction of spatial targets and improves point cloud coverage. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 Images used for the experiment;

[0046] Figure 2 This is a schematic diagram of the Retinex model;

[0047] Figure 3 Schematic diagram of the illumination invariant scale space construction process;

[0048] Figure 4 It is a flow chart of the feature description method based on illumination invariant scale space of the present invention;

[0049] Figure 5 1 is a comparison of the reconstructed point cloud renderings of an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The invention is further described below in conjunction with the accompanying drawings and embodiments. All features disclosed in all embodiments in this specification, or steps in all methods or processes disclosed implicitly, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner. Example

[0051] The technical concept of the present invention is as follows: In order to solve the problem that the gradient-based feature description algorithm cannot extract stable features in low-light environment in space, resulting in poor quality of subsequent three-dimensional reconstruction, a feature description method based on illumination-invariant scale space is proposed to ensure that a considerable number of evenly distributed feature points and robust descriptors are extracted in low-light environment for subsequent three-dimensional reconstruction.

[0052] The present invention proposes an illumination invariant and proves it theoretically, then introduces the concept of Gaussian difference pyramid, merges the illumination invariant with the pyramid to form an illumination invariant scale space, and performs feature point extraction and descriptor construction on it, which is more robust to illumination changes of the image.

[0053] In the embodiment, the present invention aims to solve the problem that the gradient-based feature descriptor is not robust to complex illumination changes and cannot extract stable features in low-illumination environments in space, which leads to poor quality of subsequent three-dimensional reconstruction. Figure 4 As shown, the specific steps are as follows:

[0054] Step 1: According to the LIME model, the input image Estimate the illumination component T. The input image dataset is as follows Figure 1 As shown. The illumination component T can be obtained by solving the following optimization equation:

[0055]

[0056] Where x represents a pixel on the image, Represents the initial estimate of the illumination component T, whose value uses the maximum value of the three color channels (R, G, B) of the input image as the initial value. represents the F norm. The first term of the optimization function In order to minimize the initial The difference between the optimized T and the original image makes the optimized T consistent with the original image in terms of pixel color distribution. The second term considers the structural perception ability of the illumination component T, and the coefficient It is used to control the degree of structural perception. d represents the horizontal and vertical directions. is the first-order differential of T, including (horizontal) and (vertical) two directions, is a very small positive number, which is set to avoid the denominator being zero. W is the weight matrix, which is critical to the structural perception of the illumination layer and is set as:

[0057]

[0058] in, represents the image pixel coordinates, Represents the Gaussian blur template scope, and t represents the pixels in the scope; is a Gaussian kernel function with standard deviation σ, It is expressed as:

[0059]

[0060] For parameter settings, , , is a variable scale, and its expression is:

[0061]

[0062] In the formula, , , , .

[0063] Step 2: Figure 2 The Retinex model is shown in Figure 1. This model indicates that the color information of an object that can be seen by an observer is determined by two factors: the illumination component T around the object and the illumination invariant component R of the object itself. That is, the original image is obtained by multiplying the illumination invariant component R and the illumination component T. The reflective property of an object itself is the ability of an object to reflect different light waves. It determines the color of the object and is an inherent property of the object. It does not change with changes in the outside world. The light intensity determines the dynamic range of all pixels in the captured image. Calculate the input image according to the Retinex model The illumination invariant component R.

[0064]

[0065] in, is a positive number to avoid zero denominator.

[0066] Step 3: Figure 3As shown, it is a schematic diagram of the illumination invariant scale space. The scale space contains multiple groups of pyramids with decreasing scales, each group of pyramids contains multiple images with continuously decreasing scales, and the first layer of the next group of pyramids is obtained by downsampling the last layer of the previous group of pyramids. The input image is subjected to the following steps in the order shown in the figure: first, the illumination invariant component R of the input image with continuously changing scale is extracted, and then a Gaussian blur operation of the corresponding scale is performed on each layer of the image, and finally a difference operation is performed on the two adjacent layers of images in the group to obtain a difference image. The final extreme point selection is performed on the three adjacent layers of difference images. Each sample point is compared with the 8 neighbors in its neighborhood and the 9 neighbors in the upper and lower scale layers. It will only be selected as an extreme point if it is larger or smaller than all neighbors. In a specific implementation, an illumination invariant scale space (IIS) space is constructed, and the formula is as follows:

[0067]

[0068] Where k is a constant coefficient, (x, y) represents the coordinates of the pixel, and * represents the convolution operation. is a Gaussian filter function with varying scale.

[0069] is the illumination invariant component of the scale change obtained in step 1. is the scale change factor, which can be expressed as:

[0070]

[0071] In the formula, is a constant, is the base layer scale factor, o is the octave group index, s is the layer index within the group, and S represents the number of images contained in each pyramid group. For parameter settings: , , , , that is, the generated IIS scale space has 5 groups, each group has 7 layers, and the images of the next group are obtained by downsampling the images of the previous group.

[0072] Step 4: Detect and screen extreme points in the illumination-invariant scale space to obtain feature points. The scale space is a Gaussian difference pyramid of the illumination-invariant component R. The pixel points of each layer in the pyramid are compared with the 8 neighbors in its neighborhood and the 9 neighbors in the upper and lower scale layers, and the largest or smallest point is selected as the extreme point. At the same time, low-contrast points and points with excessive edge response are filtered out, and the points that are finally retained are used as feature points.

[0073] Step 5: Construct a descriptor for each feature point. For the feature points detected in the IIS space, collect the IIS pyramid image where they are located. The gradient magnitude and direction distribution characteristics of pixels in the neighborhood window. The calculation formula of gradient magnitude and direction is as follows:

[0074]

[0075] in, represents the gradient amplitude, Indicates the gradient direction.

[0076] To determine the direction of the feature point, a histogram is used to count the gradient and direction of the pixels in the neighborhood of the point. The histogram divides 360 degrees into 36 intervals, each interval is 10 degrees, and the gradient amplitude falling in each interval is counted. The peak direction of the histogram represents the main direction of the feature point. In order to enhance the robustness of the matching, the direction with a peak value greater than 80% of the peak value of the main direction is retained as the auxiliary direction of the feature point. The descriptor uses the gradient information of 8 directions calculated in a 4*4 window in the feature point scale space, and a total of 4*4*8=128-dimensional vector representation.

[0077] The technical effects of the present invention are further described below in conjunction with simulation experiments.

[0078] 1. Simulation experiment conditions:

[0079] The hardware platform of the simulation experiment of the present invention is: Inter(R) Core(TM) i5-9400 CPU @ 2.90GHz, CPU memory is 8GB;

[0080] The software platform of the simulation experiment of the present invention is: Ubuntu16.04 operating system and Matlab.

[0081] The data of the simulation experiment of the present invention is a set of data of the satellite model under low light conditions. Figure 1 shown.

[0082] 2. Simulation content and results analysis:

[0083] The simulation experiment of the present invention is to Figure 1 The data is used to perform 3D reconstruction experiments to obtain the point cloud effect diagram of the target to be reconstructed, such as Figure 5 shown.

[0084] In the simulation experiment, "MVE" is an open source 3D reconstruction system, and the corresponding paper for its key 3D reconstruction technology is: A Multi-View Reconstruction Environment (Eurographics Workshop on Graphicsand Cultural Heritage, Fuhrmann S, Langguth F, Goesele M); "VisualSFM" is an open source 3D reconstruction system, and the corresponding paper for its key 3D reconstruction technology is: Towards Linear-Time Incremental Structure from Motion (In 2013 International Conference on 3DVision (3DV), Wu C); "COLMAP" is an open source 3D reconstruction system, and its corresponding paper is: Structure-from-Motion Revisited (IEEE Conference on Computer Vision & Pattern Recognition, Schonberger JL, Frahm JM).

[0085] Combine the following Figure 5 The simulation diagram of the present invention is further described.

[0086] In terms of point cloud completeness and coverage, the proposed method performs best, perfectly recovering the shape, structure and texture details of the satellite in low-light environments, and the point cloud is dense. The target 3D structure point cloud recovered by other 3D reconstruction systems in low-light environments is very sparse. In terms of robustness, COLMAP fails to reconstruct the input data set with model ambiguity, and the wings are cross-distributed. The other 3D reconstruction systems (such as Figure 5 The MVE and VisualSFM in the image processing have different degrees of deviation at the wings. The method proposed in the present invention performs well on the input data set, indicating that the algorithm proposed in the present invention can still maintain high robustness in low-light environments.

[0087] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0088] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0089] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0090] The parts not involved in the present invention are the same as the prior art or can be implemented by using the prior art.

[0091] The above technical solution is only one implementation mode of the present invention. For those skilled in the art, it is easy to make various types of improvements or modifications based on the application methods and principles disclosed in the present invention, and it is not limited to the method described in the above specific implementation mode of the present invention. Therefore, the method described above is only preferred and does not have a restrictive meaning.

[0092] In addition to the above examples, those skilled in the art may obtain other embodiments based on the above disclosure or by using the knowledge or technology in the relevant field to make changes. The features of each embodiment may be interchangeable or replaced. The changes and modifications made by those skilled in the art do not depart from the spirit and scope of the present invention and should be within the scope of protection of the claims attached to the present invention.

Claims

1. A feature description method based on illumination invariant scale space, characterized in that: Includes steps: S1. Extract the illumination component T from the input image. The extraction calculation equation of the illumination component T in step S1 is: in, represents the minimum function with independent variable T, represents the initial estimate of the illumination component T, represents the F-norm; is the coefficient; x represents the pixel x on the image; d represents the direction of solving the differential, including horizontal and vertical directions, where h represents the horizontal direction and v represents the vertical direction; is the first-order differential of T, is a positive number, W is the weight matrix, c represents the RGB channels of the image, max represents the maximum value function, and I represents the input image; S2, calculating the illumination invariant component R of the input image; the illumination invariant component R described in step S2 is calculated as follows: in, is a positive number, I is the input image; S3, constructing an illumination invariant scale space; S4, detecting and screening extreme points in the illumination invariant scale space to obtain feature points; S5. Construct a descriptor for each feature point in step S4.

2. The feature description method based on illumination invariant scale space according to claim 1, characterized in that: The illumination invariant scale space described in step S3 is constructed by the following formula: Among them, (x, y) represents the coordinates of the pixel, * represents the convolution operation, is the Gaussian filter function with varying scale; is the scale-variable illumination invariant component obtained in step S2; is the scale change factor, expressed as: In the formula, is the base layer scale factor, o is the octave group index, and s is the layer index within the group.

3. The feature description method based on illumination invariant scale space according to claim 1, characterized in that: Step S4 includes the following sub-steps: Traverse the pixel points of each layer in the scale space, compare it with the 8 neighbors in the neighborhood and the 9 neighbors in the upper and lower scale layers, and select the largest or smallest point as the extreme point; Low-contrast points with absolute values ​​of scale space functions less than 0.03 and points with excessive edge responses are filtered out, and the remaining points are used as feature points.

4. The feature description method based on illumination invariant scale space according to claim 1, characterized in that: Step S5 includes the following sub-steps: constructing a descriptor for each feature point, and for the feature point detected in the IIS space, collecting the gradient magnitude and direction distribution characteristics of the pixels in the neighborhood window of the IIS pyramid image where the feature point is located; the calculation formula of the gradient magnitude and direction is as follows: in, represents the gradient amplitude, Indicates the gradient direction.

5. The feature description method based on illumination invariant scale space according to claim 1, characterized in that: The weight matrix W is set as: in, Represents the Gaussian blur template scope, and y represents the pixels in the scope; is a Gaussian kernel function with standard deviation σ; It is expressed as: ; represents the weight matrix in the horizontal direction in step S2; represents the weight matrix in the vertical direction in step S2; dist() represents the Euclidean distance weighting function.

6. The feature description method based on illumination invariant scale space according to claim 2, characterized in that: For parameter settings, , , , , that is, the generated IIS scale space has 5 groups, each group has 7 layers, and the images of the next group are obtained by downsampling the images of the previous group.

7. The feature description method based on illumination invariant scale space according to claim 4, characterized in that: The step of collecting the gradient magnitude and direction distribution characteristics of pixels in the neighborhood window of the IIS pyramid image includes the following sub-steps: The histogram is used to count the gradient and direction of pixels in the neighborhood of the feature point. The histogram divides 360 degrees into 36 intervals, each interval is 10 degrees, and the gradient amplitude in each interval is counted. The peak direction of the histogram represents the main direction of the feature point. The direction with a peak value greater than 80% of the main direction peak is retained as the auxiliary direction of the feature point. The descriptor uses the gradient information of 8 directions calculated in a 4*4 window in the feature point scale space, with a total of 4*4*8=128-dimensional vector representation.

8. The feature description method based on illumination invariant scale space according to claim 5, characterized in that: For parameter settings: , , is a variable scale, and its expression is: In the formula, , S=4, , .

Images