Grayscale-based image matching method, system and device, product and medium

Abstract

The invention relates to the technical field of image matching, and provides a gray-scale-based image matching method, system and device, a product and a medium, and the method comprises the steps: obtaining an input image, determining an image zooming ratio and a target zooming frequency, and carrying out the zooming to obtain a zooming layer image; respectively calculating an inner-layer gray scale gradient value of the inner-layer image and an outer-layer gray scale gradient value of the outer-layer image so as to judge whether the scaling-layer image needs edge filling or not, and obtaining a to-be-screened image; generating a screening grid, dividing the to-be-screened image through the screening grid to obtain a gray region, and calculating a gray extraction threshold in the gray region; pixel point extraction is carried out to obtain feature pixel points; the method comprises the steps of obtaining target feature pixel points, constructing a multi-dimensional descriptor, obtaining a matched multi-dimensional descriptor, matching the multi-dimensional descriptor and the matched multi-dimensional descriptor to obtain a matched description pair, and completing image matching through the matched description pair. According to the invention, the image matching accuracy is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image matching technology, and in particular to a grayscale-based image matching method, system, device, product and medium. Background Art

[0002] SLAM (Simultaneous Localization and Mapping) has important research value in the fields of intelligent robots, unmanned driving and drones. Visual SLAM has been studied in depth due to the richness of its collected data and the wide range of application scenarios. Among them, feature point extraction and matching in the image is one of the key technologies in the image processing part of the visual SLAM system, and its effect directly determines the final performance characterization of the system. Therefore, it is crucial to carry out research on this technology, which can provide strong support for the research and development of target tracking, visual navigation and map construction.

[0003] With the update and iteration of computer vision technology, image feature point extraction and matching algorithms have also been studied in depth. In 1988, the Harris algorithm determined the location of feature points by calculating the curvature and gradient of pixel points. In 2004, the SIFT (Scale-Invariant Feature Transform) algorithm established a Gaussian difference space for the image so that it could still be matched to feature points after rotation and scale transformation, but its real-time performance was poor. In 2006, the SURF (Speeded Up Robust Features) algorithm simplified the calculation of descriptors in the SIFT algorithm, and its real-time performance was improved but still could not meet the demand. Then in 2009, the FAST (Features from Accelerated Segment Test) algorithm was proposed, which performed well in real-time performance, but could not guarantee scale invariance. Until 2011, the ORB (Oriented FAST and Rotated BRIEF) algorithm came into being, and gradually became the mainstream algorithm because of its stability and real-time performance. However, ORB uses a fixed threshold when using the FAST algorithm for image matching. On the one hand, the extraction effect is unstable when facing different types of images, resulting in the phenomenon of feature point aggregation. It may also fail to extract feature points at weak textures, thus affecting the matching effect. On the other hand, in actual application, it is easily affected by lighting, resulting in problems such as low algorithm matching rate and poor adaptability. Therefore, the direct setting of the threshold of the traditional ORB algorithm not only affects the performance of the algorithm, but also limits its scope of application. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a grayscale-based image matching method to achieve image matching with good illumination robustness and accurate positioning and tracking.

[0005] The present invention provides a grayscale-based image matching method, comprising: S1: Acquire an input image, determine an image zoom ratio and a target zoom number, and scale the input image according to the image zoom ratio and the target zoom number to obtain a zoom layer image; S2: Divide the zoom layer image into an inner layer image and an outer layer image, calculate the inner layer grayscale gradient value of the inner layer image and the outer layer grayscale gradient value of the outer layer image respectively, and judge whether the zoom layer image needs to be padded according to the inner layer grayscale gradient value and the outer layer grayscale gradient value; if necessary, padded the outer layer image to obtain the image to be screened, otherwise the zoom layer image is used as the image to be screened; S3: generating a screening grid, dividing the image to be screened by the screening grid to obtain a grayscale area, and calculating a grayscale extraction threshold in the grayscale area; S4: Obtaining the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extracting pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; S5: Filter the feature pixel points through a quadtree method to obtain target feature pixel points, construct a multidimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

[0006] According to a grayscale-based image matching method provided by the present invention, step S2 further comprises: S21: dividing the scaled layer image into an inner layer image and an outer layer image according to the size of the scaled layer image; S22: Obtain the grayscale value of the pixel of the zoom layer image, traverse the pixel of the inner layer image and obtain the inner layer grayscale gradient value according to the grayscale value of the pixel; traverse the pixel of the outer layer image and obtain the outer layer grayscale gradient value according to the grayscale value of the pixel; S23: Obtaining a grayscale difference threshold, obtaining the inner and outer grayscale difference through the inner grayscale gradient value and the outer grayscale gradient value, if the inner and outer grayscale difference is greater than the grayscale difference threshold, it is necessary to perform edge padding on the outer layer image, and proceed to S24, otherwise the zoom layer image is used as the image to be screened; S24: Calculate the pixel grayscale mean of the outer layer image, generate a fill-in image through the pixel grayscale mean and the outer layer image, and splice the fill-in image with the outer layer image to obtain the image to be screened.

[0007] According to a grayscale-based image matching method provided by the present invention, in step S3, the method for calculating the grayscale extraction threshold in the grayscale area is: in, is an adaptive parameter, For the The grayscale extraction threshold in the grayscale area, In the grayscale area Line The gray value of the pixel at the column position, is the average pixel value in the region, No. The area of ​​the grayscale region, For the grayscale area.

[0008] According to a grayscale-based image matching method provided by the present invention, In step S4, if the image to be screened has been edge-filled, the pixel points of the image to be screened are extracted by a circular point extraction method according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain the characteristic pixel points; If the image to be screened has not undergone edge filling, the inner layer image is subjected to pixel extraction by the circular point method according to the grayscale value of the inner layer image and the grayscale extraction threshold to obtain inner layer feature pixels, the outer layer image is subjected to pixel extraction by the coordinate system point method according to the grayscale value of the outer layer image and the grayscale extraction threshold to obtain outer layer feature pixels, and the inner layer feature pixels are merged with the outer layer feature pixels to obtain the feature pixels.

[0009] According to a grayscale-based image matching method provided by the present invention, in step S4, the minimum number of feature pixels is calculated. If the number of feature pixels is lower than the minimum number of feature pixels, the grayscale extraction threshold is updated and pixel extraction is performed again until the number of feature pixels is higher than the minimum number of feature pixels.

[0010] According to a grayscale-based image matching method provided by the present invention, in step S5, after obtaining the target feature pixel point, the grayscale centroid and geometric center of the image to be screened are determined, a multidimensional description point pair is obtained through the feature pixel point, and a multidimensional descriptor is obtained through the grayscale centroid, the geometric center and the multidimensional description point pair.

[0011] The present invention also provides a grayscale-based image matching system, comprising: Image scaling module: used to obtain an input image, determine an image scaling factor and a target scaling number, scale the input image according to the image scaling factor and the target scaling number to obtain a scaling layer image; Image edge-filling module: used for dividing the zoomed layer image into an inner layer image and an outer layer image, respectively calculating the inner layer grayscale gradient value of the inner layer image and the outer layer grayscale gradient value of the outer layer image, and judging whether the zoomed layer image needs edge-filling by the inner layer grayscale gradient value and the outer layer grayscale gradient value; if necessary, the outer layer image is edge-filled to obtain the image to be screened, otherwise the zoomed layer image is used as the image to be screened; Grayscale extraction threshold module: used to generate a screening grid, divide the image to be screened by the screening grid to obtain a grayscale area, and calculate the grayscale extraction threshold in the grayscale area; Pixel point extraction module: used to obtain the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extract the pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; Image comparison module: used to screen the feature pixels through the quadtree method to obtain target feature pixels, construct a multidimensional descriptor through the target feature pixels, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor with the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any of the grayscale-based image matching methods described above are implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the grayscale-based image matching methods described above are implemented.

[0014] The present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can perform the steps of any of the grayscale-based image matching methods described above.

[0015] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: The present invention provides a grayscale-based image matching method, system, device, product and medium, which maximize the utilization rate of the image by selectively performing edge filling operations, and provide convenience for the subsequent targeted selection of different feature pixel extraction schemes to speed up the determination of feature pixels; and the present invention uses a screening grid and a grayscale extraction threshold to fully consider the differences brought by illumination changes to different areas of the image, thereby greatly improving the illumination robustness of the system, providing more high-quality feature points for image matching, and ultimately achieving accurate and efficient image matching. The present invention has significant advantages in feature point tracking and provides a good foundation for the development of technologies such as environmental perception and positioning navigation.

[0016] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of a grayscale-based image matching method provided by the present invention.

[0019] Figure 2 It is a structural schematic diagram of a grayscale-based image matching system provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of a grayscale-based image matching device provided by the present invention.

[0021] Reference numerals: 100, image scaling module; 200, image edge filling module; 300, grayscale extraction threshold module; 400, pixel extraction module; 500, image contrast module; 810, processor; 820, communication interface; 830, memory; 840, communication bus. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme in the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0023] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0024] Combine the following Figures 1 to 3 Describing embodiments of the present invention: Figure 1 A flow chart of a grayscale-based image matching method provided by the present invention, wherein an input image is first acquired, and then the input image is scaled to obtain a scaled layer image, and inner and outer grayscale gradient values ​​are calculated in the scaled layer image, that is, the inner grayscale gradient value and the outer grayscale gradient value are calculated respectively, and then it is determined whether edge filling is required, if it is determined that edge filling is required, the image to be screened is obtained by edge filling, and if it is determined that edge filling is not required, the image to be screened is directly obtained by the scaled layer image; then a screening network can be generated and a grayscale extraction threshold can be calculated to extract pixel points and obtain feature pixel points; the feature pixel points are screened by a quadtree method to obtain a multidimensional descriptor, and finally the GMS algorithm is used for matching and a matching description pair is obtained to complete the matching.

[0025] The present invention provides a grayscale-based image matching method, comprising: S1: Acquire an input image, determine an image zoom ratio and a target zoom number, and scale the input image according to the image zoom ratio and the target zoom number to obtain a zoom layer image; Furthermore, the purpose of this stage is to scale the input image to obtain a scaled layer image. The specific implementation method is to first obtain a grayscale image as the input image, in which the pixel value is equal to the grayscale value. Then determine the image scaling factor and the target scaling times, scale the input image multiple times according to the image scaling factor until the number of scalings is equal to the target scaling times, and then obtain the scaled layer image. Here, each scaling can obtain a scaled layer image, and the subsequent steps of edge filling, grayscale extraction threshold calculation, pixel extraction, quadtree screening, etc. are all performed in a single scaled layer image.

[0026] S2: Divide the zoom layer image into an inner layer image and an outer layer image, calculate the inner layer grayscale gradient value of the inner layer image and the outer layer grayscale gradient value of the outer layer image respectively, and judge whether the zoom layer image needs to be padded according to the inner layer grayscale gradient value and the outer layer grayscale gradient value; if necessary, padded the outer layer image to obtain the image to be screened, otherwise the zoom layer image is used as the image to be screened; Furthermore, the purpose of this stage is to divide the zoom layer image and obtain the inner grayscale gradient value of the inner layer image and the outer grayscale gradient value of the outer layer image, so as to determine whether the zoom layer image needs to be padded, and then obtain the image to be screened. Step S2 specifically includes: S21: dividing the zoom layer image into an inner layer image and an outer layer image according to the size of the zoom layer image; S22: Obtain the grayscale value of the pixel of the zoom layer image, traverse the pixel of the inner layer image and obtain the inner layer grayscale gradient value according to the grayscale value of the pixel; traverse the pixel of the outer layer image and obtain the outer layer grayscale gradient value according to the grayscale value of the pixel; S23: Obtaining a grayscale difference threshold, obtaining the inner and outer grayscale difference through the inner grayscale gradient value and the outer grayscale gradient value, if the inner and outer grayscale difference is greater than the grayscale difference threshold, it is necessary to perform edge padding on the outer layer image, and proceed to S24, otherwise the zoom layer image is used as the image to be screened; S24: Calculate the pixel grayscale mean of the outer layer image, generate a fill-in image through the pixel grayscale mean and the outer layer image, and splice the fill-in image with the outer layer image to obtain the image to be screened.

[0027] With respect to the above steps, the specific implementation methods in this embodiment are as follows: First, since the grayscale gradient value of the pixels at the edge of the image is usually relatively high, it is necessary to cut off the image with a width of one pixel at the outermost edge of the image to avoid its influence on the judgment of whether to fill the edge. Then, the zoom layer image is divided into an inner layer image and an outer layer image according to the size of the zoom layer image. In this embodiment, if the length and width of the zoom layer image are respectively and , then the image center of the inner layer image is located inside the zoom layer image and has the same position as the image center of the zoom layer image. The length and width of the inner layer image are and , the remaining area of ​​the zoomed layer image is the outer layer image.

[0028] Then, the grayscale value of the pixel of the zoomed layer image is obtained, all the pixels of the inner layer image are traversed, and the grayscale gradient value of each pixel in the inner layer image is calculated according to the grayscale value of the pixel, and the inner layer grayscale gradient value is obtained. All the pixels of the outer layer image are traversed, and the grayscale gradient value of each pixel in the outer layer image is calculated according to the grayscale value of the pixel, and the outer layer grayscale gradient value is obtained. The grayscale gradient value is calculated as follows: in, For the image Line The grayscale value of the pixel at the column position, For the image Line The grayscale value of the pixel at the column position, For the image Line The grayscale value of the pixel at the column position, For the image Line The pixel at column position Direction, that is, the grayscale gradient value in the horizontal direction, For the image Line The pixel at column position Direction, that is, the grayscale gradient value in the vertical direction, For the image Line The grayscale gradient value of the pixel at the column position.

[0029] Then, the grayscale difference threshold is obtained, and the mean of the inner grayscale gradient value and the mean of the outer grayscale gradient value are calculated respectively, and the difference between the mean of the inner grayscale gradient value and the mean of the outer grayscale gradient value is calculated to obtain the inner and outer grayscale difference. The inner and outer grayscale difference is used to determine whether the outer image needs to be padded. If the inner and outer grayscale difference is greater than the grayscale difference threshold, it is considered that the outer image needs to be padded.

[0030] When the outer layer image needs to be padded, the mean of the grayscale values ​​of all pixels of the outer layer image is calculated to obtain the pixel grayscale mean of the outer layer image, then the outer layer image near the outer edge of the outer layer image is obtained and used as the initial padded image, and the grayscale mean of the initial padded image is adjusted according to the pixel grayscale mean, so that the grayscale mean of the initial padded image is equal to the pixel grayscale mean of the outer layer image, thereby obtaining the padded image; the padded image is spliced ​​to the outer edge of the outer layer image to obtain the image to be screened. When the outer layer image does not need to be padded, the single zoom layer image is directly used as the image to be screened.

[0031] The edge filling operation can effectively avoid missing feature pixels near the outer edge of the outer image when extracting feature pixels later. Selective edge filling is to fill the outer image only when it contains rich texture features. If it is judged that edge filling is not needed, it means that the outer image does not contain many texture features, and even if it is omitted, it will not affect the accuracy of image matching, thereby effectively saving computing resources.

[0032] S3: generating a screening grid, dividing the image to be screened by the screening grid to obtain a grayscale area, and calculating a grayscale extraction threshold in the grayscale area; Furthermore, the purpose of this stage is to divide the grayscale area and calculate the grayscale extraction threshold of each grayscale area so as to extract feature pixels in the grayscale area later. In step S3, the method for calculating the grayscale extraction threshold in the grayscale area is: in, is an adaptive parameter, For the The grayscale extraction threshold in the grayscale area, In the grayscale area Line The gray value of the pixel at the column position, is the average pixel value in the region, No. The area of ​​the grayscale region, For the grayscale area.

[0033] With respect to the above steps, the specific implementation of this embodiment is as follows: First, generate a screening grid, that is, a grid array with the same size of each grid. Cover the screening grid on the image to be screened, and the area surrounded by a grid can be regarded as a grayscale area, thereby completing the division of the image to be screened. indivual, is the number of grid cells in a row of the grid array, is the number of grid cells in a column of the grid array, is the length of the image to be filtered, is the width of the image to be screened. In this embodiment, the length of the image to be screened is equal to the number of grids in a row of the grid array, and the width of the image to be screened is equal to the number of grids in a column of the grid array. Then, the grayscale extraction threshold of each grayscale area is calculated in the grayscale area: in, is an adaptive parameter set based on experience. For the The grayscale extraction threshold in the grayscale area, In the grayscale area Line The gray value of the pixel at the column position, is the average pixel value in the region, No. The area of ​​the grayscale region, For the grayscale area.

[0034] S4: Obtaining the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extracting pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; Furthermore, the purpose of this stage is to traverse each pixel point in the image to be screened, extract the pixel points of the image to be screened, and obtain the characteristic pixel points. In step S4, if the image to be screened has undergone edge filling, the pixel points of the image to be screened are extracted by the circular point method according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain the characteristic pixel points; If the image to be screened has not undergone edge filling, the inner layer image is subjected to pixel extraction by the circular point method according to the grayscale value of the inner layer image and the grayscale extraction threshold to obtain inner layer feature pixels, the outer layer image is subjected to pixel extraction by the coordinate system point method according to the grayscale value of the outer layer image and the grayscale extraction threshold to obtain outer layer feature pixels, and the inner layer feature pixels are merged with the outer layer feature pixels to obtain the feature pixels.

[0035] In addition, in step S4, the minimum number of feature pixels is calculated. If the number of feature pixels is lower than the minimum number of feature pixels, the grayscale extraction threshold is updated and pixel extraction is performed again until the number of feature pixels is higher than the minimum number of feature pixels.

[0036] With respect to the above steps, the specific implementation of this embodiment is as follows: First, calculate the minimum number of characteristic pixels of the image to be screened. The calculation method of the minimum number of characteristic pixels N is as follows: Among them, NA is the total number of feature points estimated based on experience, is the image zoom factor, is the number of times the image to be filtered undergoes scaling. Zooms the target by a certain amount.

[0037] Then, the grayscale value of the inner layer image and the grayscale value of the outer layer image are obtained. The grayscale value of the inner layer image is the grayscale value of each pixel in the inner layer image of the image to be screened, and the grayscale value of the outer layer image is the grayscale value of each pixel in the outer layer image of the image to be screened. Different pixel extraction strategies are selected according to whether the outer layer image of the image to be screened has undergone edge filling. If the outer layer image has undergone edge filling, it means that the texture information contained in the outer layer image is also relatively rich. At this time, the pixel points in the entire image to be screened are extracted using the circular point method.

[0038] The circumference point method is to draw a circle with a radius of r with a pixel as the center. In this embodiment, r is the length of 3 pixels. Then, the grayscale value of each pixel in the circle is determined according to the grayscale value of the inner image and the grayscale value of the outer image, and multiple pixels are randomly selected in the circle, here 16 pixels, and then the difference between the grayscale value of each selected pixel and the grayscale value of the pixel at the center of the circle is calculated. If the difference between the grayscale value of multiple consecutive pixels and the grayscale value of the pixel at the center of the circle is greater than the grayscale extraction threshold in the grayscale area where the pixel at the center of the circle is located, here 12 consecutive pixels are taken, and the pixel at the center of the circle is considered to be a feature pixel. Each pixel in the image to be screened is taken as the center of the circle in turn to traverse all the pixels in the image to be screened, and the feature pixel of the image to be screened can be obtained.

[0039] If the image to be screened has not been edge-filled, it means that the texture features of the image to be screened are concentrated in the inner layer image, and the texture features contained in the outer layer image are limited. At this time, the outer layer image can be extracted using a simpler coordinate system point extraction method. The inner layer image grayscale value and grayscale extraction threshold are still used to extract pixel points, and all pixel points in the inner layer image are traversed to obtain the inner layer feature pixel points. For the outer layer image, the coordinate system point extraction method is used to extract pixel points.

[0040] The coordinate system point method is to take a pixel point as the origin, establish a plane coordinate system with the x-axis in the horizontal direction and the y-axis in the vertical direction, and in this embodiment, the length of the x-axis and the y-axis are both 6 pixels long, and then calculate the difference between the grayscale value of each pixel point on the x-axis and the y-axis and the grayscale value of the pixel point of the origin. If the grayscale values ​​of multiple pixels, 9 pixels in this embodiment, and the grayscale values ​​of the pixel point of the origin are both greater than the grayscale extraction threshold in the grayscale area where the pixel point of the origin is located, then the pixel point of the origin is considered to be an outer feature pixel point, and each pixel point in the outer image is taken as the origin in turn to traverse all the pixels in the outer image to obtain the outer feature pixel point, and the inner feature pixel point is merged with the outer feature pixel point to obtain the feature pixel point.

[0041] In addition, if the number of extracted feature pixels is lower than the minimum number of feature pixels, it means that the grayscale extraction threshold of each grayscale area is too high during extraction. At this time, the grayscale extraction threshold of each grayscale area is updated to 1 / 2 of the original grayscale extraction threshold and pixel extraction is performed again, and this cycle is repeated until the number of feature pixels is higher than or equal to the minimum number of feature pixels. The circular point selection method can ensure that the quality of the extracted feature pixels is high, but it also consumes more computing resources, while the coordinate system point selection rule is the opposite. Therefore, this application selects a suitable feature pixel extraction method based on the number of texture features contained in the inner and outer images to ensure that the extracted feature pixels have a high quality while also ensuring less computing resources are occupied.

[0042] S5: Filter the feature pixel points through a quadtree method to obtain target feature pixel points, construct a multidimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

[0043] Furthermore, the purpose of this stage is to obtain a matching multidimensional scanner of the comparison image, obtain a multidimensional descriptor of the image to be screened, and use the GMS (Grid-based Motion Statistics) algorithm to match the multidimensional descriptor and the matching multidimensional descriptor, thereby completing image matching. In step S5, after obtaining the target feature pixel point, the grayscale centroid and geometric center of the image to be screened are determined, and a multidimensional description point pair is obtained through the feature pixel point, and a multidimensional descriptor is obtained through the grayscale centroid, the geometric center and the multidimensional description point pair.

[0044] With respect to the above steps, the specific implementation of this embodiment is as follows: First, the feature pixels are screened by the quadtree method, that is, the image to be screened including the feature pixels is first used as the root node, and then the root node is divided into four nodes to determine whether each node contains multiple feature pixels. If the node does not contain any feature pixels, the node is deleted, and if it contains only one feature pixel, the node is retained; if it contains multiple feature pixels, the node is further divided into four nodes until all nodes contain only one feature pixel or the number of nodes begins to exceed the minimum number of feature pixels. When the input of the node begins to exceed the minimum number of feature pixels, for the nodes that still contain multiple feature pixels at this time, only the feature pixel with the largest grayscale value is retained. In this way, the feature pixels in the node can be used as the target feature pixels.

[0045] Next, after obtaining the target feature pixel point, determine the grayscale centroid of the image to be screened, that is, the weighted center point and the geometric center calculated with the grayscale value of each pixel in the image to be screened as the weight, and take random pixel pairs around the feature pixel point to form a multidimensional description point pair, draw a line from the geometric center to the grayscale centroid, and use the direction of the line as the main direction. Add the main direction to the multidimensional description point pair to obtain a multidimensional descriptor.

[0046] Obtain a comparison image with the same number of zooms and image zoom ratio as the image to be screened, and use the above method and steps on the comparison image to obtain a matching multidimensional descriptor, and then use the GMS algorithm to match the matching multidimensional descriptor with the multidimensional descriptor, so that a matching relationship is established between the multidimensional descriptor and the matching multidimensional descriptor. Obtain a matching description pair. However, some matching relationships in the matching description pair are wrong at this time, that is, a mismatch occurs. In the GMS algorithm, we believe that the crossed and chaotic matching lines in the matching description pair are mismatches, while the smooth matching lines that are consistent with the surrounding matching lines are correct matches; in addition, there will be many other matching lines around the correct matching lines, so the matching lines whose number of other matching lines around the matching lines is lower than the threshold set according to experience can also be classified as mismatches, and the image matching is completed after the mismatch is deleted. In addition, multiple zoom layer images that have been zoomed in different times can be matched with the comparison images that have been zoomed in the same number of times and the same image zoom ratio, and the obtained image matching results can be compared and verified with each other to further delete the inconsistent matching results between different zoom times and improve the accuracy of image matching.

[0047] The present invention also verifies the effectiveness of a grayscale-based image matching method. During the verification, the fr2_desk image set and the fr3_office image set in the TUM dataset are selected to perform image matching experiments under different lighting environments, and the matching rate between the same images is used as a measurement indicator. The method used for comparison is the ORB method, and the experimental results are shown in Table 1; wherein +20% indicates a 20% increase in brightness, and -20% indicates a 20% decrease in brightness.

[0048] Table 1 Image matching experimental results under different lighting environments

[0049] It can be seen from Table 1 that the present invention can achieve a higher matching rate under different lighting environments, which indicates that the present method has a stronger adaptability to different lighting environments and a higher matching rate.

[0050] Next, a feature point tracking experiment is conducted, that is, the feature points in the image are tracked through image matching, and the tracking error is used as a measurement indicator. During verification, the fr2_desk image set, fr1_room image set and fr1_desk image set in the TUM dataset and the 00 image set, 05 image set and 07 image set in the KITTI dataset are selected. The method used for comparison is the ORB-SLAM2 method. The experimental results are shown in Table 2, and the unit of the tracking error is meter: Table 2 Tracking experiment results in different image sets

[0051] It can be seen from Table 2 that smaller errors can be obtained in the feature point tracking experiments of each image set, that is, better image matching results can be obtained.

[0052] A grayscale-based image matching system provided by the present invention is described below. The grayscale-based image matching system described below and the grayscale-based image matching method described above can refer to each other.

[0053] Figure 2 It is a structural diagram of a grayscale-based image matching system. Figure 2 As shown, the method for performing the grayscale-based image matching method as described above includes: Image scaling module 100: used to obtain an input image, determine an image scaling factor and a target scaling number, scale the input image according to the image scaling factor and the target scaling number, and obtain a scaling layer image; The image edge-filling module 200 is used to divide the zoomed layer image into an inner layer image and an outer layer image, respectively calculate the inner layer grayscale gradient value of the inner layer image and the outer layer grayscale gradient value of the outer layer image, and judge whether the zoomed layer image needs edge-filling according to the inner layer grayscale gradient value and the outer layer grayscale gradient value; if necessary, the outer layer image is edge-filled to obtain the image to be screened, otherwise the zoomed layer image is used as the image to be screened; Grayscale extraction threshold module 300: used to generate a screening grid, divide the image to be screened by the screening grid to obtain grayscale areas, and calculate the grayscale extraction threshold in the grayscale areas; Pixel point extraction module 400: used to obtain the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extract the pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; Image comparison module 500: used to screen the feature pixel points through the quadtree method to obtain target feature pixel points, construct a multidimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

[0054] on the other hand, Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute a grayscale-based image matching method, which includes: S1: Acquire an input image, determine an image zoom ratio and a target zoom number, and scale the input image according to the image zoom ratio and the target zoom number to obtain a zoom layer image; S2: Divide the zoom layer image into an inner layer image and an outer layer image, calculate the inner layer grayscale gradient value of the inner layer image and the outer layer grayscale gradient value of the outer layer image respectively, and judge whether the zoom layer image needs to be padded according to the inner layer grayscale gradient value and the outer layer grayscale gradient value; if necessary, padded the outer layer image to obtain the image to be screened, otherwise the zoom layer image is used as the image to be screened; S3: generating a screening grid, dividing the image to be screened by the screening grid to obtain a grayscale area, and calculating a grayscale extraction threshold in the grayscale area; S4: Obtaining the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extracting pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; S5: Filter the feature pixel points through a quadtree method to obtain target feature pixel points, construct a multidimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

[0055] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0056] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer can execute a grayscale-based image matching method provided by the above methods, the method comprising: S1: Acquire an input image, determine an image zoom ratio and a target zoom number, and scale the input image according to the image zoom ratio and the target zoom number to obtain a zoom layer image; S2: Divide the zoom layer image into an inner layer image and an outer layer image, calculate the inner layer grayscale gradient value of the inner layer image and the outer layer grayscale gradient value of the outer layer image respectively, and judge whether the zoom layer image needs to be padded according to the inner layer grayscale gradient value and the outer layer grayscale gradient value; if necessary, padded the outer layer image to obtain the image to be screened, otherwise the zoom layer image is used as the image to be screened; S3: generating a screening grid, dividing the image to be screened by the screening grid to obtain a grayscale area, and calculating a grayscale extraction threshold in the grayscale area; S4: Obtaining the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extracting pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; S5: Filter the feature pixel points through a quadtree method to obtain target feature pixel points, construct a multidimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

[0057] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform a grayscale-based image matching method provided by the above methods, the method comprising: S1: Acquire an input image, determine an image zoom ratio and a target zoom number, and scale the input image according to the image zoom ratio and the target zoom number to obtain a zoom layer image; S2: Divide the zoom layer image into an inner layer image and an outer layer image, calculate the inner layer grayscale gradient value of the inner layer image and the outer layer grayscale gradient value of the outer layer image respectively, and judge whether the zoom layer image needs to be padded according to the inner layer grayscale gradient value and the outer layer grayscale gradient value; if necessary, padded the outer layer image to obtain the image to be screened, otherwise the zoom layer image is used as the image to be screened; S3: generating a screening grid, dividing the image to be screened by the screening grid to obtain a grayscale area, and calculating a grayscale extraction threshold in the grayscale area; S4: Obtaining the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extracting pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; S5: Filter the feature pixel points through a quadtree method to obtain target feature pixel points, construct a multidimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

[0058] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0059] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A grayscale-based image matching method, characterized in that: include: S1: Acquire an input image, determine an image zoom ratio and a target zoom number, and scale the input image according to the image zoom ratio and the target zoom number to obtain a zoom layer image; S2: dividing the zoomed layer image into an inner layer image and an outer layer image, respectively calculating an inner layer grayscale gradient value of the inner layer image and an outer layer grayscale gradient value of the outer layer image, and judging whether the zoomed layer image needs edge filling according to the inner layer grayscale gradient value and the outer layer grayscale gradient value; If necessary, the outer layer image is padded to obtain the image to be screened, otherwise the zoomed layer image is used as the image to be screened; S3: generating a screening grid, dividing the image to be screened by the screening grid to obtain a grayscale area, and calculating a grayscale extraction threshold in the grayscale area; S4: Obtaining the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extracting pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; S5: Filter the feature pixel points through a quadtree method to obtain target feature pixel points, construct a multidimensional descriptor through the target feature pixel points, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor and the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

2. The grayscale-based image matching method according to claim 1, characterized in that: Step S2 further comprises: S21: dividing the scaled layer image into an inner layer image and an outer layer image according to the size of the scaled layer image; S22: Obtain the grayscale value of the pixel of the zoom layer image, traverse the pixel of the inner layer image and obtain the inner layer grayscale gradient value according to the grayscale value of the pixel; traverse the pixel of the outer layer image and obtain the outer layer grayscale gradient value according to the grayscale value of the pixel; S23: Obtaining a grayscale difference threshold, obtaining the inner and outer grayscale difference through the inner grayscale gradient value and the outer grayscale gradient value, if the inner and outer grayscale difference is greater than the grayscale difference threshold, it is necessary to perform edge padding on the outer layer image, and proceed to S24, otherwise the zoom layer image is used as the image to be screened; S24: Calculate the pixel grayscale mean of the outer layer image, generate a fill-in image through the pixel grayscale mean and the outer layer image, and splice the fill-in image with the outer layer image to obtain the image to be screened.

3. The grayscale-based image matching method according to claim 1, characterized in that: In step S3, the method for calculating the grayscale extraction threshold in the grayscale area is: in, is an adaptive parameter, For the The grayscale extraction threshold in the grayscale area, In the grayscale area Line The gray value of the pixel at the column position, is the average pixel value in the region, No. The area of ​​the grayscale region, For the grayscale area.

4. The grayscale-based image matching method according to claim 1, characterized in that: In step S4, if the image to be screened has been edge-filled, the pixel points of the image to be screened are extracted by a circular point extraction method according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain the characteristic pixel points; If the image to be screened has not undergone edge filling, the inner layer image is subjected to pixel extraction by the circular point method according to the grayscale value of the inner layer image and the grayscale extraction threshold to obtain inner layer feature pixels, the outer layer image is subjected to pixel extraction by the coordinate system point method according to the grayscale value of the outer layer image and the grayscale extraction threshold to obtain outer layer feature pixels, and the inner layer feature pixels are merged with the outer layer feature pixels to obtain the feature pixels.

5. The grayscale-based image matching method according to claim 4, characterized in that: In step S4, the minimum number of feature pixels is calculated. If the number of feature pixels is lower than the minimum number of feature pixels, the grayscale extraction threshold is updated and pixel extraction is performed again until the number of feature pixels is higher than the minimum number of feature pixels.

6. The grayscale-based image matching method according to claim 1, characterized in that: In step S5, after obtaining the target feature pixel point, the grayscale centroid and geometric center of the image to be screened are determined, a multidimensional description point pair is obtained through the feature pixel point, and a multidimensional descriptor is obtained through the grayscale centroid, the geometric center and the multidimensional description point pair.

7. A grayscale-based image matching system, used to execute a grayscale-based image matching method according to any one of claims 1 to 6, characterized in that: include: Image scaling module: used to obtain an input image, determine an image scaling factor and a target scaling number, scale the input image according to the image scaling factor and the target scaling number to obtain a scaling layer image; Image edge filling module: used for dividing the zoom layer image into an inner layer image and an outer layer image, respectively calculating the inner layer grayscale gradient value of the inner layer image and the outer layer grayscale gradient value of the outer layer image, and judging whether the zoom layer image needs edge filling according to the inner layer grayscale gradient value and the outer layer grayscale gradient value; If necessary, the outer layer image is padded to obtain the image to be screened, otherwise the zoomed layer image is used as the image to be screened; Grayscale extraction threshold module: used to generate a screening grid, divide the image to be screened by the screening grid to obtain a grayscale area, and calculate the grayscale extraction threshold in the grayscale area; Pixel point extraction module: used to obtain the grayscale value of the inner layer image and the grayscale value of the outer layer image, and extract the pixels of the image to be screened according to the grayscale value of the inner layer image, the grayscale value of the outer layer image and the grayscale extraction threshold to obtain characteristic pixels; Image comparison module: used to screen the feature pixels through the quadtree method to obtain target feature pixels, construct a multidimensional descriptor through the target feature pixels, select a comparison image, obtain a matching multidimensional descriptor through the comparison image, use the GMS algorithm to match the multidimensional descriptor with the matching multidimensional descriptor to obtain a matching description pair, and complete image matching through the matching description pair.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the grayscale-based image matching method as described in any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the grayscale-based image matching method as described in any one of claims 1 to 6 are implemented.

10. A computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, characterized in that: When the program instructions are executed by a computer, the computer can execute the steps of the grayscale-based image matching method as described in any one of claims 1 to 6.

Images