A grayscale-based image matching method, system, device, product and medium

Abstract

The present invention relates to the field of image matching technology, and provides a gray-scale based image matching method, system, device, product and medium, including obtaining an input image, determining an image scaling ratio and a target scaling number of times, and performing scaling to obtain a scaled image layer; respectively calculating an inner gray-scale gradient value of an inner-layer image and an outer gray-scale gradient value of an outer-layer image, thereby determining whether the scaled image layer needs to be padded to obtain an image to be screened; generating a screening grid, dividing the image to be screened through the screening grid to obtain gray-scale regions, and calculating a gray-scale extraction threshold in the gray-scale regions; performing pixel point extraction to obtain feature pixel points; obtaining target feature pixel points and constructing a multi-dimensional descriptor, obtaining a matching multi-dimensional descriptor, matching the multi-dimensional descriptor and the matching multi-dimensional descriptor to obtain a matching descriptor pair, and completing image matching through the matching descriptor pair. The present invention effectively improves the accuracy of image matching.

Description

Technical Field

[0001] The present invention relates to the technical field of image matching, and particularly to a gray-scale based image matching method, system, device, product and medium. Background Art

[0002] SLAM (Simultaneous Localization and Mapping) has important research value in the fields of intelligent robots, driverless vehicles, and drones. Visual SLAM has been studied in depth due to the richness of the data it collects and the wide range of application scenarios. Among them, the extraction and matching of feature points in images is one of the key technologies in the image processing part of the visual SLAM system, and its effect directly determines the final performance of the system. Therefore, it is crucial to carry out research on this technology, which can provide strong support for the research and development of target tracking, visual navigation, and map construction.

[0003] With the update and iteration of computer vision technology, algorithms for image feature point extraction and matching have also been studied in depth. In 1988, the Harris algorithm determined the position of feature points by calculating the curvature and gradient of pixel points; in 2004, the SIFT (Scale-Invariant Feature Transform) algorithm established a difference of Gaussian space for the image so that it could still be matched to feature points after rotation and scale transformation, but its real-time performance was poor; in 2006, the SURF (Speeded Up Robust Features) algorithm simplified the calculation of descriptors in the SIFT algorithm, and its real-time performance was improved but still difficult to meet the requirements; subsequently, in 2009, the FAST (Features from Accelerated Segment Test) algorithm was proposed, which showed excellent real-time performance but could not guarantee scale invariance; until 2011, the ORB (Oriented FAST and Rotated BRIEF) algorithm came into being and gradually became the mainstream algorithm due to its stability and real-time performance. However, when the ORB algorithm uses the FAST algorithm for image matching, it uses a fixed threshold. On the one hand, when facing different types of images, the extraction effect is unstable, resulting in the phenomenon of feature point aggregation, and it may also be unable to extract feature points in weak texture areas, thus affecting the matching effect. On the other hand, in practical applications, it is easily affected by light, resulting in problems such as low algorithm matching rate and poor adaptability. Therefore, the direct setting of the threshold of the traditional ORB algorithm not only affects the algorithm performance, but also limits its application range. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the related art. For this purpose, the present invention provides a gray-scale based image matching method, which can achieve image matching with good illumination robustness and accurate positioning and tracking.

[0005] The present invention provides a gray-scale based image matching method, including:

[0006] S1: Obtain an input image, determine the image scaling ratio and the target scaling times, and scale the input image according to the image scaling ratio and the target scaling times to obtain a scaled layer image;

[0007] S2: Divide the scaled layer image into an inner layer image and an outer layer image, calculate the inner layer gray-scale gradient value of the inner layer image and the outer layer gray-scale gradient value of the outer layer image respectively, and determine whether the scaled layer image needs to be padded based on the inner layer gray-scale gradient value and the outer layer gray-scale gradient value; if so, pad the outer layer image to obtain an image to be screened, otherwise use the scaled layer image as the image to be screened;

[0008] S3: Generate a screening grid, divide the image to be screened through the screening grid to obtain gray-scale regions, and calculate a gray-scale extraction threshold in the gray-scale regions;

[0009] S4: Obtain the inner layer image gray-scale value and the outer layer image gray-scale value, and extract pixel points of the image to be screened based on the inner layer image gray-scale value, the outer layer image gray-scale value and the gray-scale extraction threshold to obtain characteristic pixel points;

[0010] S5: Screen the characteristic pixel points through a quadtree method to obtain target characteristic pixel points, construct a multi-dimensional descriptor through the target characteristic pixel points, select a comparison image, obtain a matching multi-dimensional descriptor through the comparison image, use the GMS algorithm to match the multi-dimensional descriptor and the matching multi-dimensional descriptor to obtain a matching descriptor pair, and complete image matching through the matching descriptor pair.

[0011] According to the gray-scale based image matching method provided by the present invention, step S2 further includes:

[0012] S21: Divide the scaled layer image into an inner layer image and an outer layer image according to the size of the scaled layer image;

[0013] S22: Obtain the pixel point gray-scale value of the scaled layer image, traverse the pixel points of the inner layer image and obtain the inner layer gray-scale gradient value according to the pixel point gray-scale value; traverse the pixel points of the outer layer image and obtain the outer layer gray-scale gradient value according to the pixel point gray-scale value;

[0014] S23: Obtain the grayscale difference threshold, calculate the internal and external grayscale difference from the inner grayscale gradient value and the outer grayscale gradient value. If the internal and external grayscale difference is greater than the grayscale difference threshold, it is necessary to perform edge padding on the outer image and enter S24; otherwise, use the scaled layer image as the image to be screened.

[0015] S24: Calculate the pixel grayscale mean value of the outer image, generate a padded image from the pixel grayscale mean value and the outer image, splice the padded image and the outer image to obtain the image to be screened.

[0016] According to an image matching method based on grayscale provided by the present invention, in step S3, the method for calculating the grayscale extraction threshold in the grayscale region is:

[0017]

[0018] where, is an adaptive parameter, is the grayscale extraction threshold in the th grayscale region, is the grayscale value of the pixel at the nd row and th column position in the grayscale region, is the average value of the region pixels, is the area of the th grayscale region, is the th grayscale region.

[0019] According to an image matching method based on grayscale provided by the present invention,

[0020] In step S4, if the image to be screened has undergone edge padding, use the circumferential point sampling method, and extract pixel points from the image to be screened according to the inner image grayscale value, the outer image grayscale value, and the grayscale extraction threshold to obtain the characteristic pixel points;

[0021] If the image to be screened has not undergone edge padding, use the circumferential point sampling method, and extract pixel points from the inner image according to the inner image grayscale value and the grayscale extraction threshold to obtain inner characteristic pixel points, use the coordinate system point sampling method, and extract pixel points from the outer image according to the outer image grayscale value and the grayscale extraction threshold to obtain outer characteristic pixel points, and merge the inner characteristic pixel points and the outer characteristic pixel points to obtain the characteristic pixel points.

[0022] According to an image matching method based on grayscale provided by the present invention, in step S4, the minimum number of feature pixels is calculated. If the number of the feature pixels is lower than the minimum number of feature pixels, the grayscale extraction threshold is updated and pixel extraction is performed again until the number of the feature pixels is higher than the minimum number of feature pixels.

[0023] According to an image matching method based on grayscale provided by the present invention, in step S5, after obtaining the target feature pixels, the grayscale centroid and geometric center of the image to be screened are determined. Multidimensional description point pairs are obtained through the feature pixels, and a multidimensional descriptor is obtained through the grayscale centroid, the geometric center, and the multidimensional description point pairs.

[0024] The present invention also provides an image matching system based on grayscale, including:

[0025] An image scaling module: used to obtain an input image, determine an image scaling ratio and a target number of scaling times, and scale the input image according to the image scaling ratio and the target number of scaling times to obtain a scaled image layer;

[0026] An image border padding module: used to divide the scaled image layer into an inner image and an outer image, calculate the inner grayscale gradient value of the inner image and the outer grayscale gradient value of the outer image respectively, and determine whether the scaled image layer needs border padding through the inner grayscale gradient value and the outer grayscale gradient value; if so, perform border padding on the outer image to obtain an image to be screened, otherwise use the scaled image layer as the image to be screened;

[0027] A grayscale extraction threshold module: used to generate a screening grid, divide the image to be screened through the screening grid to obtain grayscale regions, and calculate a grayscale extraction threshold in the grayscale regions;

[0028] A pixel extraction module: used to obtain the grayscale value of the inner image and the grayscale value of the outer image, and perform pixel extraction on the image to be screened according to the grayscale value of the inner image, the grayscale value of the outer image, and the grayscale extraction threshold to obtain feature pixels;

[0029] An image comparison module: used to screen the feature pixels through a quadtree method to obtain target feature pixels, construct a multidimensional descriptor through the target feature pixels, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

[0030] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of a grayscale-based image matching method as described in any one of the above are implemented.

[0031] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a grayscale-based image matching method as described in any one of the above are implemented.

[0032] The present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the steps of a grayscale-based image matching method as described in any one of the above.

[0033] One or more of the above technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0034] A grayscale-based image matching method, system, device, product, and medium provided by the present invention maximize the utilization rate of images by selectively performing edge filling operations, and facilitate subsequent targeted selection of different feature pixel point extraction schemes to accelerate the determination speed of feature pixel points. Moreover, by using a screening grid and a grayscale extraction threshold, the present invention fully considers the differences brought by illumination changes to different regions of the image, greatly improving the illumination robustness of the system, providing more high-quality feature points for image matching, and finally achieving accurate and efficient image matching. The present invention has significant advantages in feature point tracking and provides a good foundation for the development of technologies such as environmental perception and positioning navigation.

[0035] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 It is a schematic flowchart of a grayscale-based image matching method provided by the present invention.

[0038] Figure 2It is a schematic structural diagram of an image matching system based on grayscale provided by the present invention.

[0039] Figure 3 It is a schematic structural diagram of an image matching device based on grayscale provided by the present invention.

[0040] Reference numerals:

[0041] 100, image scaling module; 200, image border padding module; 300, grayscale extraction threshold module; 400, pixel point extraction module; 500, image comparison module; 810, processor; 820, communication interface; 830, memory; 840, communication bus. Detailed implementation manners

[0042] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0043] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0044] The following is combined with Figures 1 to 3 to describe the implementation scheme of the present invention:

[0045] Figure 1Schematic flow chart of an image matching method based on grayscale provided by the present invention. First, an input image is obtained, and then the input image is scaled to obtain a scaled-layer image. The inner and outer grayscale gradient values are calculated in the scaled-layer image, that is, the inner grayscale gradient value and the outer grayscale gradient value are calculated respectively. Subsequently, it is determined whether padding is required. If it is determined that padding is required, the image to be screened is obtained through padding. If it is determined that no padding is required, the image to be screened is directly obtained from the scaled-layer image. Then, a screening network can be generated and the grayscale extraction threshold can be calculated to perform pixel extraction to obtain feature pixels. The feature pixels are screened by the quadtree method to obtain a multi-dimensional descriptor. Finally, the GMS algorithm is used for matching to obtain a matching descriptor pair to complete the matching.

[0046] An image matching method based on grayscale provided by the present invention includes:

[0047] S1: Obtain an input image, determine the image scaling ratio and the target number of scaling times, and scale the input image according to the image scaling ratio and the target number of scaling times to obtain a scaled-layer image;

[0048] Further, the purpose of this stage is to scale the input image to obtain a scaled-layer image. The specific implementation method is as follows: First, a grayscale image is obtained as the input image, and the pixel value in the grayscale image is equal to the grayscale value. Then, the image scaling ratio and the target number of scaling times are determined, and the input image is scaled multiple times according to the image scaling ratio until the number of scaling times is equal to the target number of scaling times, and the scaled-layer image can be obtained. Here, a scaled-layer image can be obtained each time scaling is performed, and subsequent steps such as padding, grayscale extraction threshold calculation, pixel extraction, and quadtree screening are all performed on a single scaled-layer image.

[0049] S2: Divide the scaled-layer image into an inner image and an outer image, calculate the inner grayscale gradient value of the inner image and the outer grayscale gradient value of the outer image respectively, and determine whether the scaled-layer image needs to be padded based on the inner grayscale gradient value and the outer grayscale gradient value; if so, pad the outer image to obtain the image to be screened, otherwise use the scaled-layer image as the image to be screened;

[0050] Further, the purpose of this stage is to divide the scaled-layer image and obtain the inner grayscale gradient value of the inner image and the outer grayscale gradient value of the outer image, so as to determine whether the scaled-layer image needs to be padded, and then obtain the image to be screened. Step S2 specifically includes: S21: Divide the scaled-layer image into an inner image and an outer image according to the size of the scaled-layer image;

[0051] S22: Obtain the gray value of the pixel points of the scaled layer image, traverse the pixel points of the inner layer image and obtain the inner layer gray gradient value according to the pixel gray value; traverse the pixel points of the outer layer image and obtain the outer layer gray gradient value according to the pixel gray value;

[0052] S23: Obtain the gray difference threshold, obtain the internal and external gray difference through the inner layer gray gradient value and the outer layer gray gradient value. If the internal and external gray difference is greater than the gray difference threshold, it is necessary to perform edge padding on the outer layer image and enter S24. Otherwise, use the scaled layer image as the image to be screened;

[0053] S24: Calculate the average pixel gray value of the outer layer image, generate a padding image through the average pixel gray value and the outer layer image, splice the padding image and the outer layer image to obtain the image to be screened.

[0054] For the above steps, the specific implementation in this embodiment is as follows:

[0055] First, since the gray gradient values of the pixels at the image edge are usually relatively high, it is necessary to crop the outermost circle of the image with a width of one pixel to avoid its influence on the judgment of whether edge padding is required. Then divide the scaled layer image into an inner layer image and an outer layer image according to the size of the scaled layer image. In this embodiment, if the length and width of the scaled layer image are and , the image center of the inner layer image is located inside the scaled layer image and has the same position as the image center of the scaled layer image. The length and width of the inner layer image are and respectively, and the remaining area of the scaled layer image is the outer layer image.

[0056] Subsequently, obtain the gray value of the pixel points of the scaled layer image, traverse all the pixel points of the inner layer image and calculate the gray gradient value of each pixel point in the inner layer image according to the pixel gray value to obtain the inner layer gray gradient value, traverse all the pixel points of the outer layer image and calculate the gray gradient value of each pixel point in the outer layer image according to the pixel gray value to obtain the outer layer gray gradient value. The calculation method of the gray gradient value is as follows:

[0057]

[0058]

[0059]

[0060] Among them, is the gray value of the pixel at the position of the th row and the th column in the image, is the grayscale value of the pixel at the th row and th column position in the image. is the grayscale value of the pixel at the th row and th column position in the image. is the grayscale gradient value of the pixel at the th row and th column position in the image in the direction, i.e., the horizontal direction. is the grayscale gradient value of the pixel at the th row and th column position in the image in the direction, i.e., the vertical direction. is the grayscale gradient value of the pixel at the th row and th column position in the image.

[0061] Next, obtain the grayscale difference threshold, and calculate the mean value of the inner-layer grayscale gradient values and the mean value of the outer-layer grayscale gradient values respectively. Calculate the difference between the mean value of the inner-layer grayscale gradient values and the mean value of the outer-layer grayscale gradient values to obtain the inner-outer grayscale difference. Determine whether the outer-layer image needs to be border-complemented based on the inner-outer grayscale difference. If the inner-outer grayscale difference is greater than the grayscale difference threshold, it is considered that the outer-layer image needs to be border-complemented.

[0062] When the outer-layer image needs to be border-complemented, calculate the mean value of the grayscale values of all pixels in the outer-layer image to obtain the pixel grayscale mean value of the outer-layer image. Next, obtain the outer-layer image near the outer edge of the outer-layer image and use it as the initial border-complemented image. Adjust the mean value of the grayscale values of the initial border-complemented image according to the pixel grayscale mean value so that the mean value of the grayscale values of the initial border-complemented image is equal to the pixel grayscale mean value of the outer-layer image, thereby obtaining the border-complemented image; splice the border-complemented image to the outer edge of the outer-layer image to obtain the image to be screened. When the outer-layer image does not need to be border-complemented, directly use the single scaled-layer image as the image to be screened.

[0063] The border-complement operation can effectively avoid missing feature pixel points near the outer edge of the outer-layer image when extracting feature pixel points subsequently. The selective border complement is to perform border complement only when the outer-layer image contains relatively rich texture features. When it is determined that border complement is not required, it means that the outer-layer image contains few texture features, and even if there are omissions, it will not affect the accuracy of image matching, thus effectively saving computing resources.

[0064] S3: Generate a screening grid, divide the image to be screened through the screening grid to obtain grayscale regions, and calculate the grayscale extraction threshold in the grayscale regions;

[0065] Further, the purpose of this stage is to divide and obtain the grayscale regions, and calculate the grayscale extraction threshold for each grayscale region, so as to extract the feature pixel points in the grayscale regions subsequently. In step S3, the method for calculating the grayscale extraction threshold in the grayscale region is as follows:

[0066]

[0067] Wherein, is the adaptive parameter, is the grayscale extraction threshold in the th grayscale region, is the grayscale value of the pixel at the th row and th column position in the grayscale region, is the average value of the region pixels, is the th area of the grayscale region, is the th grayscale region.

[0068] For the above steps, the specific implementation manner of this embodiment is as follows:

[0069] First, generate a screening grid, that is, a grid array with the same size for each grid. Cover the screening grid on the image to be screened, and the area surrounded by one grid can be used as a grayscale region, thus completing the division of the image to be screened. There are grayscale regions, is the number of grids in one row of the grid array, is the number of grids in one column of the grid array, is the length of the image to be screened, is the width of the image to be screened. In this embodiment, the value of the length of the image to be screened is equal to the number of grids in one row of the grid array, and the value of the width of the image to be screened is equal to the number of grids in one column of the grid array. Then calculate the grayscale extraction threshold for each grayscale region in the grayscale region:

[0070]

[0071]

[0072]

[0073] Wherein, is the adaptive parameter set according to experience, is the grayscale extraction threshold in the th grayscale region, is the grayscale value of the pixel at the th row and th column position in the grayscale region, is the average value of regional pixels, the area of the th gray level region,

[0074] S4: Obtain the gray level values of the inner-layer image and the outer-layer image, and extract pixel points from the image to be screened according to the gray level values of the inner-layer image, the gray level values of the outer-layer image, and the gray extraction threshold, so as to obtain characteristic pixel points;

[0075] Furthermore, the purpose of this stage is to traverse each pixel point in the image to be screened, extract pixel points from the image to be screened, and obtain characteristic pixel points. In step S4, if the image to be screened has undergone edge padding, then the method of taking points on the circumference is used, and pixel points are extracted from the image to be screened according to the gray level values of the inner-layer image, the gray level values of the outer-layer image, and the gray extraction threshold, so as to obtain the characteristic pixel points;

[0076] If the image to be screened has not undergone edge padding, then the method of taking points on the circumference is used, and pixel points are extracted from the inner-layer image according to the gray level values of the inner-layer image and the gray extraction threshold to obtain inner-layer characteristic pixel points, and the method of taking points in the coordinate system is used, and pixel points are extracted from the outer-layer image according to the gray level values of the outer-layer image and the gray extraction threshold to obtain outer-layer characteristic pixel points, and the inner-layer characteristic pixel points and the outer-layer characteristic pixel points are merged to obtain the characteristic pixel points.

[0077] In addition, in step S4, calculate the minimum number of characteristic pixel points. If the number of the characteristic pixel points is lower than the minimum number of characteristic pixel points, then update the gray extraction threshold and re-perform pixel point extraction until the number of the characteristic pixel points is higher than the minimum number of characteristic pixel points.

[0078] For the above steps, the specific implementation manners of this embodiment are as follows:

[0079] First, calculate the minimum number of characteristic pixel points of the image to be screened. The calculation method of the minimum number of characteristic pixel points N is as follows:

[0080]

[0081] where NA is the total number of characteristic points estimated according to experience, is the image scaling ratio, is the number of scaling times experienced by the image to be screened, is the target number of scaling times.

[0082] Next, obtain the gray values of the inner-layer image and the outer-layer image. The gray value of the inner-layer image is the gray value of each pixel in the inner-layer image of the image to be screened, and the gray value of the outer-layer image is the gray value of each pixel in the outer-layer image of the image to be screened. Different pixel extraction strategies are selected according to whether the outer-layer image of the image to be screened has undergone edge padding. If the outer-layer image has undergone edge padding, it indicates that the texture information contained in the outer-layer image is also relatively rich. At this time, the circular point-taking method is used to extract pixels for all pixels in the entire image to be screened.

[0083] The circular point-taking method is to draw a circle with a radius of r centered on a pixel. In this embodiment, r is the length of 3 pixels. Then, determine the gray value of each pixel in the circle according to the gray value of the inner-layer image and the gray value of the outer-layer image, and randomly select multiple pixels in the circle. Here, it is 16 pixels. Subsequently, calculate the difference between the gray value of each selected pixel and the gray value of the pixel at the center of the circle. If the difference between the gray values of multiple consecutive pixels and the gray value of the pixel at the center of the circle is greater than the gray extraction threshold in the gray area where the pixel at the center of the circle is located. Here, take 12 consecutive pixels, then the pixel at the center of the circle is considered a feature pixel. Take each pixel in the image to be screened as the center of the circle in turn to traverse all pixels in the image to be screened, and the feature pixels of the image to be screened can be obtained.

[0084] If the image to be screened has not undergone edge padding, it indicates that the texture features of the image to be screened are concentrated in the inner-layer image, and the texture features contained in the outer-layer image are limited. At this time, a simpler coordinate system point-taking method can be used to extract the outer-layer image. For the inner-layer image, the gray value of the inner-layer image and the gray extraction threshold are still used to extract pixels, and all pixels in the inner-layer image are traversed to obtain the inner-layer feature pixels. For the outer-layer image, the coordinate system point-taking method is used to extract pixels.

[0085] The coordinate system point-taking method is to establish a plane coordinate system with a pixel as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis. In this embodiment, the lengths of the x-axis and the y-axis are both 6 pixels long. Then, calculate the difference between the gray value of each pixel on the x-axis and the y-axis and the gray value of the pixel at the origin. If the gray values of multiple pixels among them, in this embodiment, 9 pixels, are all greater than the gray extraction threshold in the gray area where the pixel at the origin is located, then the pixel at the origin is considered an outer-layer feature pixel. Take each pixel in the outer-layer image as the origin in turn to traverse all pixels in the outer-layer image to obtain the outer-layer feature pixels, and merge the inner-layer feature pixels and the outer-layer feature pixels to obtain the feature pixels.

[0086] In addition, if the number of extracted feature pixels is lower than the minimum number of feature pixels, it indicates that the gray extraction thresholds of each gray region are too high during extraction. At this time, update the gray extraction thresholds of each gray region to 1 / 2 of the original gray extraction thresholds and re-extract the pixels. Repeat this process until the number of feature pixels is higher than or equal to the minimum number of feature pixels. The circumferential point-taking method can ensure a relatively high quality of the extracted feature pixels, but it also consumes a large amount of computing resources. On the contrary, the coordinate system point-taking method consumes less computing resources. Therefore, in this application, a suitable feature pixel extraction method is selected according to the amount of texture features included in the inner image and the outer image, so as to ensure both a relatively high quality of the extracted feature pixels and less consumption of computing resources.

[0087] S5: Screen the feature pixels through a quadtree method to obtain target feature pixels. Construct a multi-dimensional descriptor through the target feature pixels, select a comparison image, obtain a matching multi-dimensional descriptor through the comparison image, use the GMS algorithm to match the multi-dimensional descriptor and the matching multi-dimensional descriptor to obtain a matching descriptor pair, and complete image matching through the matching descriptor pair.

[0088] Further, the purpose of this stage is to obtain a matching multi-dimensional scan descriptor of the comparison image, obtain a multi-dimensional descriptor of the image to be screened, and use the GMS (Grid-based Motion Statistics) algorithm to match the multi-dimensional descriptor and the matching multi-dimensional descriptor, so as to complete image matching. In step S5, after obtaining the target feature pixels, determine the gray centroid and geometric center of the image to be screened, obtain a multi-dimensional description point pair through the feature pixels, and obtain a multi-dimensional descriptor through the gray centroid, the geometric center, and the multi-dimensional description point pair.

[0089] For the above steps, the specific implementation manners of this embodiment are as follows:

[0090] First, screen the feature pixels through a quadtree method. That is, first use the image to be screened including the feature pixels as the root node, then divide the root node into four nodes, and judge whether each node contains multiple feature pixels. If a node does not contain any feature pixels, delete the node; if it contains only one feature pixel, retain the node; if it contains multiple feature pixels, continue to divide the node into four nodes until all nodes contain only one feature pixel or the number of nodes starts to be greater than the minimum number of feature pixels. When the input of the node starts to be greater than the minimum number of feature pixels, for the nodes that still contain multiple feature pixels at this time, only retain the feature pixel with the largest gray value. In this way, the feature pixels in the nodes can be used as the target feature pixels.

[0091] Next, after obtaining the target feature pixel points, determine the gray centroid of the image to be screened, that is, the weighted center point calculated with the gray value of each pixel as the weight in the image to be screened, and the geometric center. Take random pixel pairs around the feature pixel points to form multi-dimensional description point pairs. Draw a line from the geometric center to the gray centroid, and use the direction of the line as the main direction. Adding the main direction to the multi-dimensional description point pairs can obtain the multi-dimensional descriptor.

[0092] Obtain a comparison image whose scaling times and image scaling ratio are the same as those of the image to be screened. Use the above methods and steps on the comparison image to obtain matching multi-dimensional descriptors. Then use the GMS algorithm to match the matching multi-dimensional descriptors and the multi-dimensional descriptors, so as to establish a matching relationship between the multi-dimensional descriptors and the matching multi-dimensional descriptors, and obtain matching description pairs. However, some of the matching relationships in the matching description pairs at this time are incorrect, that is, false matches occur. In the GMS algorithm, we consider that the intersecting and chaotic matching lines in the matching description pairs belong to false matches, while the smooth matching lines that are consistent with the directions of the surrounding matching lines belong to correct matches. In addition, there will be more other matching lines around the correct matching lines. Therefore, the matching lines with the number of other matching lines around them lower than the threshold set according to experience can also be classified as false matches. After deleting the false matches, the image matching is completed. In addition, multiple scaled images with different scaling times can be respectively matched with comparison images with the same scaling times and the same image scaling ratio, and the obtained image matching results can be compared and verified with each other to further delete the parts where the matching results are inconsistent between different scaling times, and improve the accuracy of image matching.

[0093] The present invention also verifies the effectiveness of a gray-based image matching method. During the verification, the fr2_desk image set and the fr3_office image set in the TUM dataset are selected for image matching experiments under different lighting environments, and the matching rate between the same images is used as a measurement index. The method for comparison is the ORB method. The experimental results are shown in Table 1; among them, +20% means a 20% increase in brightness, and -20% means a 20% decrease in brightness.

[0094] Table 1 Experimental results of image matching under different lighting environments

[0095]

[0096] As can be seen from Table 1, the present invention can achieve a higher matching rate under different lighting environments, which shows that the method has stronger adaptability to different lighting environments and a higher matching rate.

[0097] Next, a feature point tracking experiment is carried out, that is, the feature points in the image are tracked through image matching, and the tracking error is used as a measurement index. During verification, the fr2_desk image set, fr1_room image set, and fr1_desk image set in the TUM dataset, as well as the 00 image set, 05 image set, and 07 image set in the KITTI dataset, are selected. The method for comparison is the ORB-SLAM2 method. The experimental results are shown in Table 2, and the unit of the tracking error is meters:

[0098] Table 2 Tracking experiment results in different image sets

[0099]

[0100] It can be seen from Table 2 that smaller errors can be obtained in the feature point tracking experiments of each image set, that is, better image matching results can be obtained.

[0101] Next, a gray-scale based image matching system provided by the present invention will be described. The gray-scale based image matching system described below can be correspondingly referred to the gray-scale based image matching method described above.

[0102] Figure 2 It is a structural schematic diagram of a gray-scale based image matching system, as Figure 2 shown, for executing a gray-scale based image matching method as described above, including:

[0103] Image scaling module 100: used to obtain the input image, determine the image scaling ratio and the target scaling times, and scale the input image according to the image scaling ratio and the target scaling times to obtain the scaled layer image;

[0104] Image border supplement module 200: used to divide the scaled layer image into an inner layer image and an outer layer image, calculate the inner gray-scale gradient value of the inner layer image and the outer gray-scale gradient value of the outer layer image respectively, and judge whether the scaled layer image needs to be border-supplemented through the inner gray-scale gradient value and the outer gray-scale gradient value; if so, perform border supplementation on the outer layer image to obtain the image to be screened, otherwise use the scaled layer image as the image to be screened;

[0105] Gray-scale extraction threshold module 300: used to generate a screening grid, divide the image to be screened through the screening grid to obtain gray-scale regions, and calculate the gray-scale extraction threshold in the gray-scale regions;

[0106] Pixel point extraction module 400: used to obtain the inner layer image gray-scale value and the outer layer image gray-scale value, and extract pixel points from the image to be screened according to the inner layer image gray-scale value, the outer layer image gray-scale value, and the gray-scale extraction threshold to obtain the feature pixel points;

[0107] Image comparison module 500: used to screen the feature pixel points by the quadtree method to obtain target feature pixel points, construct a multi-dimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multi-dimensional descriptor through the comparison image, use the GMS algorithm to match the multi-dimensional descriptor and the matching multi-dimensional descriptor to obtain a matching descriptor pair, and complete image matching through the matching descriptor pair.

[0108] On the other hand, Figure 3 An example of the physical structure diagram of an electronic device is shown in Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 complete communication with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute an image matching method based on grayscale. The method includes:

[0109] S1: Obtain an input image, determine the image scaling ratio and the target scaling times, and scale the input image according to the image scaling ratio and the target scaling times to obtain a scaled image layer;

[0110] S2: Divide the scaled image layer into an inner image and an outer image, calculate the inner grayscale gradient value of the inner image and the outer grayscale gradient value of the outer image respectively, and determine whether the scaled image layer needs to be padded through the inner grayscale gradient value and the outer grayscale gradient value; if so, pad the outer image to obtain an image to be screened, otherwise use the scaled image layer as the image to be screened;

[0111] S3: Generate a screening grid, divide the image to be screened through the screening grid to obtain grayscale regions, and calculate grayscale extraction thresholds in the grayscale regions;

[0112] S4: Obtain the inner image grayscale value and the outer image grayscale value, and extract pixel points from the image to be screened according to the inner image grayscale value, the outer image grayscale value, and the grayscale extraction threshold to obtain feature pixel points;

[0113] S5: Screen the feature pixel points by the quadtree method to obtain target feature pixel points, construct a multi-dimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multi-dimensional descriptor through the comparison image, use the GMS algorithm to match the multi-dimensional descriptor and the matching multi-dimensional descriptor to obtain a matching descriptor pair, and complete image matching through the matching descriptor pair.

[0114] In addition, when the logical instructions in the memory 830 described above can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0115] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a grayscale-based image matching method provided by the above-mentioned various methods. The method includes:

[0116] S1: Obtain an input image, determine the image scaling ratio and the target scaling times, and scale the input image according to the image scaling ratio and the target scaling times to obtain a scaled-layer image;

[0117] S2: Divide the scaled-layer image into an inner-layer image and an outer-layer image, respectively calculate the inner-layer grayscale gradient value of the inner-layer image and the outer-layer grayscale gradient value of the outer-layer image, and determine whether the scaled-layer image needs to be padded based on the inner-layer grayscale gradient value and the outer-layer grayscale gradient value; if so, pad the outer-layer image to obtain an image to be screened, otherwise use the scaled-layer image as the image to be screened;

[0118] S3: Generate a screening grid, divide the image to be screened through the screening grid to obtain grayscale regions, and calculate a grayscale extraction threshold in the grayscale regions;

[0119] S4: Obtain the inner-layer image grayscale value and the outer-layer image grayscale value, and extract pixel points from the image to be screened based on the inner-layer image grayscale value, the outer-layer image grayscale value, and the grayscale extraction threshold to obtain characteristic pixel points;

[0120] S5: Screen the feature pixel points through the quadtree method to obtain target feature pixel points, construct a multi-dimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multi-dimensional descriptor through the comparison image, use the GMS algorithm to match the multi-dimensional descriptor and the matching multi-dimensional descriptor to obtain matching descriptor pairs, and complete image matching through the matching descriptor pairs.

[0121] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a gray-scale based image matching method provided by the above methods. The method includes:

[0122] S1: Obtain an input image, determine the image scaling ratio and the target scaling times, and scale the input image according to the image scaling ratio and the target scaling times to obtain a scaled layer image;

[0123] S2: Divide the scaled layer image into an inner layer image and an outer layer image, calculate the inner layer gray-scale gradient value of the inner layer image and the outer layer gray-scale gradient value of the outer layer image respectively, and judge whether the scaled layer image needs to be padded based on the inner layer gray-scale gradient value and the outer layer gray-scale gradient value; if so, pad the outer layer image to obtain an image to be screened, otherwise use the scaled layer image as the image to be screened;

[0124] S3: Generate a screening grid, divide the image to be screened through the screening grid to obtain gray-scale regions, and calculate the gray-scale extraction threshold in the gray-scale regions;

[0125] S4: Obtain the inner layer image gray-scale value and the outer layer image gray-scale value, and extract pixel points from the image to be screened based on the inner layer image gray-scale value, the outer layer image gray-scale value and the gray-scale extraction threshold to obtain feature pixel points;

[0126] S5: Screen the feature pixel points through the quadtree method to obtain target feature pixel points, construct a multi-dimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multi-dimensional descriptor through the comparison image, use the GMS algorithm to match the multi-dimensional descriptor and the matching multi-dimensional descriptor to obtain matching descriptor pairs, and complete image matching through the matching descriptor pairs.

[0127] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort.

[0128] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image matching method based on grayscale, characterized in that Including: S1: Obtain an input image, determine the image scaling ratio and the target number of scaling times, and scale the input image according to the image scaling ratio and the target number of scaling times to obtain a scaled image layer; S2: Divide the scaled image layer into an inner image and an outer image, calculate the inner gray gradient value of the inner image and the outer gray gradient value of the outer image respectively, and determine whether the scaled image layer needs to be padded based on the inner gray gradient value and the outer gray gradient value; If so, pad the outer image to obtain an image to be screened, otherwise use the scaled image layer as the image to be screened; S3: Generate a screening grid, divide the image to be screened through the screening grid to obtain gray regions, and calculate a gray extraction threshold in the gray regions; S4: Obtain the inner image gray value and the outer image gray value, and extract pixel points from the image to be screened according to the inner image gray value, the outer image gray value and the gray extraction threshold to obtain characteristic pixel points; S5: Screen the characteristic pixel points through a quadtree method to obtain target characteristic pixel points, construct a multi-dimensional descriptor through the target characteristic pixel points, select a comparison image, obtain a matching multi-dimensional descriptor through the comparison image, use the GMS algorithm to match the multi-dimensional descriptor and the matching multi-dimensional descriptor to obtain a matching descriptor pair, and complete image matching through the matching descriptor pair.

2. The image matching method based on grayscale according to claim 1, characterized in that, Step S2 further includes: S21: Divide the scaled image layer into an inner image and an outer image according to the size of the scaled image layer; S22: Obtain the pixel gray value of the scaled image layer, traverse the pixel points of the inner image and obtain the inner gray gradient value according to the pixel gray value; traverse the pixel points of the outer image and obtain the outer gray gradient value according to the pixel gray value; S23: Obtain a gray difference threshold, obtain an inner-outer gray difference through the inner gray gradient value and the outer gray gradient value. If the inner-outer gray difference is greater than the gray difference threshold, the outer image needs to be padded and proceed to S24, otherwise use the scaled image layer as the image to be screened; S24: Calculate the pixel gray mean value of the outer image, generate a padded image through the pixel gray mean value and the outer image, and splice the padded image and the outer image to obtain the image to be screened.

3. A grayscale-based image matching method according to claim 1, characterized in that In step S3, the method for calculating the gray extraction threshold in the gray regions is: Among them, is an adaptive parameter, is the grayscale extraction threshold in the th grayscale region, is the grayscale value of the pixel at the th row and th column position in the grayscale region, is the average value of the region pixels, is the area of the th grayscale region, is the th grayscale region.

4. A gray-scale-based image matching method according to claim 1, characterized in that In step S4, if the image to be screened has been padded, use the circumferential point-taking method, and extract pixel points from the image to be screened according to the inner image gray value, the outer image gray value and the gray extraction threshold to obtain the characteristic pixel points; If the image to be screened has not undergone edge padding, the inner layer feature pixel points are obtained by using the circumferential point sampling method and extracting pixel points from the inner layer image according to the inner layer image gray value and the gray value extraction threshold. The outer layer feature pixel points are obtained by using the coordinate system point sampling method and extracting pixel points from the outer layer image according to the outer layer image gray value and the gray value extraction threshold. The inner layer feature pixel points and the outer layer feature pixel points are merged to obtain the feature pixel points.

5. A grayscale-based image matching method according to claim 4, characterized in that In step S4, the minimum number of feature pixel points is calculated. If the number of the feature pixel points is lower than the minimum number of feature pixel points, the gray value extraction threshold is updated and pixel point extraction is performed again until the number of the feature pixel points is higher than the minimum number of feature pixel points.

6. A method for image matching based on grayscale according to claim 1, characterized in that, In step S5, after obtaining the target feature pixel points, the gray centroid and geometric center of the image to be screened are determined. Multidimensional description point pairs are obtained through the feature pixel points, and multidimensional descriptors are obtained through the gray centroid, the geometric center, and the multidimensional description point pairs.

7. A gray-scale based image matching system for performing a gray-scale based image matching method according to any one of claims 1 to 6, characterized in that, Including: Image scaling module: used to obtain an input image, determine an image scaling ratio and a target scaling number of times, and scale the input image according to the image scaling ratio and the target scaling number of times to obtain a scaled layer image; Image edge padding module: used to divide the scaled layer image into an inner layer image and an outer layer image, calculate the inner layer gray gradient value of the inner layer image and the outer layer gray gradient value of the outer layer image respectively, and determine whether the scaled layer image needs edge padding through the inner layer gray gradient value and the outer layer gray gradient value; If so, perform edge padding on the outer layer image to obtain an image to be screened, otherwise use the scaled layer image as the image to be screened; Gray value extraction threshold module: used to generate a screening grid, divide the image to be screened through the screening grid to obtain gray regions, and calculate the gray value extraction threshold in the gray regions; Pixel point extraction module: used to obtain the inner layer image gray value and the outer layer image gray value, and extract pixel points from the image to be screened according to the inner layer image gray value, the outer layer image gray value, and the gray value extraction threshold to obtain feature pixel points; Image comparison module: used to screen the feature pixel points through the quadtree method to obtain target feature pixel points, construct multidimensional descriptors through the target feature pixel points, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching descriptor pair, and complete image matching through the matching descriptor pair.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a gray-based image matching method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of a gray-based image matching method according to any one of claims 1 to 6.

10. A computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, characterized in that, When the program instructions are executed by the computer, the computer can execute the steps of a gray-based image matching method according to any one of claims 1 to 6.

Images