Image registration method based on multi-channel local binary pattern and related device

Through the multi-channel local binary mode method, the symbols, amplitudes and neighborhood features of the image are extracted, and feature histograms are generated for image registration, which solves the problems of high complexity and poor matching effect of the image registration algorithm in the prior art. Efficient and robust image registration is achieved.

CN118941603APending Publication Date: 2024-11-12XI AN JIAOTONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411033806.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing image registration algorithms have high computational complexity and poor matching effect, especially in image scale changes and noise environments.

Method used

Using the multi-channel local binary mode method, the symbol information, amplitude information and neighborhood features of the image center pixels are extracted by the rotation of the unchanged unified mode LBP, independent feature channels are constructed and feature histograms are generated, and the image is calculated using the histogram intersection distance. The similarity between the two is registered.

Benefits of technology

It reduces the complexity of the algorithm, improves the robustness and accuracy of image registration, can effectively handle rotation and scale changes, and is highly robust to noise and lighting changes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941603A_ABST
    Figure CN118941603A_ABST
Patent Text Reader

Abstract

The invention discloses an image registration method based on a multi-channel local binary pattern and a related device, and belongs to the technical field of image feature extraction and registration. The method comprises the following steps: respectively extracting symbol information LBPS, amplitude information LBPM and neighborhood feature LBPNF of an image to be registered, then constructing three independent image feature channels, and generating three feature histograms; cascading the three feature histograms to obtain a final feature histogram of the image; and calculating the similarity between the images by using the final intersection distance of the feature histograms, and carrying out image registration. The LBP algorithm used in the invention is low in calculation complexity, easy for engineering realization, and strong in robustness to external environments including noise, illumination and the like. The algorithm can be applied to an image registration algorithm, similar texture features of pictures of the same scene with obvious resolution differences are extracted, the algorithm has the characteristics of rotation invariance, multi-scale adaptability and the like, and the problems existing in a traditional image registration algorithm can be solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image feature extraction and registration, and relates to an image registration method based on multi-channel local binary patterns and a related device. Background Art

[0002] Image registration is one of the fundamental problems in computer vision and image processing. There are two main types of problems: (1) multiple images from different acquisition devices or perspectives need to be compared and the differences between the images need to be found through registration; (2) important structures are found in a known template image and then matched in another image.

[0003] The main purpose of image registration is to eliminate geometric differences, and the difference in grayscale information is not of concern. However, it is precisely because of the existence of grayscale differences in image distortion that geometric correction becomes more difficult.

[0004] At present, after decades of development, image registration algorithms can be mainly summarized into four types: image grayscale-based, transform domain-based, template-based, and feature-based image matching algorithms. Traditional image grayscale-based matching algorithms have huge computational complexity, are time-consuming, and are greatly affected by changes in image illumination. They are no longer suitable for practical applications in the current image processing field. Image registration algorithms based on deep learning are not suitable for hardware implementation because they require training and learning and a large amount of data. Image feature matching algorithms construct feature descriptors to describe the invariant information in the image and then perform matching. Compared with grayscale-based matching algorithms, feature matching algorithms use less information for matching, so the computing speed is greatly improved. Generally, the features of an image have properties such as high distinguishability and good robustness. In addition, local information such as feature points has good robustness to deformation, noise, etc. Common algorithms include SIFT, ORB, LBP, etc.

[0005] The features of an image can generally be classified into point, line, and surface features. Among them, point features are simple to extract, fast to match, and have strong robustness to image illumination and various geometric transformations. Therefore, matching algorithms based on image features usually use point features of images. Its core idea is to use the local features of certain pixels in the image and their surrounding areas to represent the entire image, thereby reducing the complexity of image registration. It has important research value and broad application prospects.

[0006] The point feature-based image registration algorithm can usually be divided into three stages: feature detection, feature description and feature matching. However, the traditional point feature-based image registration algorithm has certain problems in each stage.

[0007] Extracting image feature points is the first step in image feature matching and is crucial to the final matching performance. Currently, a variety of feature point extraction methods have been proposed, such as Moravec, Harris, and SUSAN. However, these methods perform poorly in adapting to image scale changes. In contrast, the SIFT algorithm solves this problem, but because it involves a large number of complex convolution operations and matrix operations of high-dimensional feature point descriptors, the complexity of the algorithm is greatly increased.

[0008] Subsequently, the algorithm needs to construct a feature descriptor. After obtaining the position and scale parameters of the image feature points, the traditional SIFT algorithm uses the gradient direction map of the image feature points in the surrounding area to describe the feature points, and obtains a gradient-based feature point descriptor. As the application scenarios of image feature point matching algorithms become increasingly complex, current applications have put forward higher requirements on the algorithm complexity and the storage size of feature point description vectors. The feature point descriptor based on image gradients has high dimensions and a complex generation process, which requires a lot of calculation and memory space. To meet this challenge, binary descriptors came into being. As a typical representative, BRIEF has a simple generation process, short calculation time, uses a compact binary form to describe features, and occupies little storage space. However, it is not invariant to image rotation and scale changes, and is easily interfered by noise.

[0009] At present, the optimization algorithm is developing rapidly, which helps to simplify the search process of feature space. However, even for classic image feature point matching algorithms such as SIFT, when the image undergoes significant changes, the final matching effect is not very ideal, and many mismatched feature point pairs will be generated. However, in practical applications, matching the same area of ​​two images does not require too many matching feature point pairs, and the existence of these mismatched feature point pairs is meaningless. In summary, it is urgent to develop a new image registration method based on multi-channel local binary patterns. Summary of the invention

[0010] The purpose of the present invention is to provide an image registration method and related devices based on multi-channel local binary patterns to solve the technical problems of high computational complexity and poor final matching effect of image registration in the prior art.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] In a first aspect, the present invention provides an image registration method based on multi-channel local binary patterns, comprising the following steps:

[0013] The symbol information of the central pixel of the image to be registered is extracted using the rotation invariant unified pattern LBP; the neighborhood pixels are quantized to obtain the local binary pattern of the local area according to the difference vector between the central pixel and the neighborhood pixels of the local area; the local binary pattern is calculated one by one for each central pixel of the image to be registered, and the symbol information LBP_S of the central pixel of the image to be registered is obtained;

[0014] Extract the amplitude part of the difference vector between the central pixel and the neighboring pixels, and quantize the amplitude part to obtain the amplitude information LBP_M of the central pixel;

[0015] The first-order difference and L2 norm are used to calculate the gradient of the neighborhood pixels, and the local adaptive threshold quantization method is used to encode the gradient to obtain the neighborhood feature LBP_NF;

[0016] Based on the sign information LBP_S, amplitude information LBP_M and neighborhood feature LBP_NF, three independent image feature channels are constructed respectively to generate three feature histograms. The three feature histograms are cascaded to obtain the final feature histogram of the image. The final feature histogram intersection distance is used to calculate the similarity between images for image registration.

[0017] Further, the steps of using the rotation invariant unified pattern LBP to extract the sign information of the central pixel of the image to be registered; quantizing the neighborhood pixels according to the central pixel of the local area and the neighborhood pixel difference vector to obtain the local binary pattern of the local area; obtaining the local binary pattern for each central pixel of the image to be registered one by one to obtain the sign information LBP_S of the central pixel of the image to be registered specifically include:

[0018] Calculate the difference vector between the central pixel and P neighboring pixels in the circular neighborhood template on the sampling scale R, use the central pixel as the quantization threshold of binary quantization, and quantize the neighboring pixels; when the pixel value of the neighboring pixel is greater than the pixel value of the central pixel, the neighboring pixel is quantized to 1; when the pixel value of the neighboring pixel is less than or equal to the pixel value of the central pixel, the neighboring pixel is quantized to 0; after the quantization is completed, a local binary string is obtained;

[0019] The obtained local binary string is multiplied by the weight template to be converted into a decimal value as the final local binary pattern of the local area;

[0020] The local binary pattern is calculated for each central pixel of the image to be registered one by one to obtain the symbol information LBP_S of the central pixel of the image to be registered.

[0021] Furthermore, the local binary pattern includes a unified pattern and a non-uniform pattern; the unified pattern is a local binary pattern with a U value not greater than 2; the non-uniform pattern is a local binary pattern with a U value greater than 2; and U represents the number of conversions between 0 and 1 characters in the local binary string.

[0022] Furthermore, the step of extracting the amplitude part of the difference vector between the central pixel and the neighboring pixels and quantizing the amplitude part to obtain the amplitude information LBP_M of the central pixel specifically includes:

[0023] The amplitude part of the difference vector between the central pixel and the neighboring pixels is extracted, and the amplitude part is binary quantized using the rotation-invariant unified pattern LBP. The quantization threshold is the mean of the amplitudes of all the central pixel neighborhood difference vectors in the entire image. Finally, the amplitude information LBP_M of the central pixel is quantized.

[0024] Furthermore, the step of using the first-order difference and L2 norm to calculate the gradient of the neighborhood pixels, adopting the local adaptive threshold quantization method, encoding the gradient to obtain the neighborhood feature LBP_NF specifically includes:

[0025] The neighborhood pixel gradient in the clockwise direction is defined as The neighborhood pixel gradient in the counterclockwise direction is defined as The specific formula is:

[0026]

[0027] Where P is the number of neighborhood pixels; g is the neighborhood pixel;

[0028] Neighborhood pixel gradient Defined as corresponding and The combination of the L2 norm is:

[0029]

[0030] The neighborhood pixel gradient is encoded to obtain the neighborhood feature LBP_NF.

[0031] Furthermore, the step of encoding the neighborhood pixel gradient to obtain the neighborhood feature LBP_NF specifically includes: dividing the image to be registered into N×N sub-images, using the local adaptive threshold of each sub-image to encode the gradient vector Binary quantization is performed to obtain the neighborhood feature LBP_NF.

[0032] Furthermore, the local adaptive threshold of each sub-image is The mean of .

[0033] In a second aspect, the present invention provides an image registration system based on multi-channel local binary patterns, comprising:

[0034] The symbol information module is used to extract the symbol information of the central pixel of the image to be registered using the rotation invariant unified pattern LBP; according to the central pixel of the local area and the neighborhood pixel difference vector, the neighborhood pixels are quantized to obtain the local binary pattern of the local area; the local binary pattern is calculated for each central pixel of the image to be registered one by one to obtain the symbol information LBP_S of the central pixel of the image to be registered;

[0035] Amplitude information module, used to extract the amplitude part of the difference vector between the central pixel and the neighboring pixels, and quantize the amplitude part to obtain the amplitude information LBP_M of the central pixel;

[0036] The neighborhood feature module is used to calculate the gradient of the neighborhood pixels using the first-order difference and L2 norm, and adopt the local adaptive threshold quantization method to encode the gradient to obtain the neighborhood feature LBP_NF;

[0037] The image registration module is used to construct three independent image feature channels based on the symbol information LBP_S, the amplitude information LBP_M and the neighborhood feature LBP_NF, and generate three feature histograms; the three feature histograms are cascaded to obtain the final feature histogram of the image; the final feature histogram intersection distance is used to calculate the similarity between images for image registration.

[0038] In a third aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention discloses an image registration method and related device based on multi-channel local binary pattern, which uses a local binary pattern algorithm to extract local texture features of the central pixel of the image to be registered, and generates an image feature histogram generated by the cascade of symbol information, amplitude information and neighborhood feature information, and then calculates the similarity between images based on the intersection distance of the histogram to match the images. The LBP algorithm used in the present invention has low computational complexity, is easy to implement in engineering, and has strong robustness to external environments such as noise and illumination. The algorithm can be applied in the image registration algorithm to extract similar texture features of the same scene pictures with obvious differences in resolution. It has the characteristics of rotation invariance and multi-scale adaptability, and can solve the problems existing in the traditional image registration algorithm. In addition, there is strong complementarity between the multi-channel features extracted by the algorithm of the present invention, which can fully characterize the texture features of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 is a flow chart of the method of the present invention;

[0044] Figure 2 is a schematic diagram of the system of the present invention;

[0045] Figure 3 It is a flowchart of the LBP algorithm of the present invention;

[0046] Figure 4 Schematic diagram of LBP circular neighborhood system of different scales in the present invention;

[0047] Figure 5 A schematic diagram of neighborhood pixel gradients of the present invention;

[0048] Figure 6 This is a flowchart of the image registration process of the present invention;

[0049] Figure 7 The figure shows the experimental results of the complementarity verification between LBP_M and LBP_NF;

[0050] Figure 8 The figure shows the experimental results of the complementarity verification between LBP_S and LBP_NF;

[0051] Fig. 9 This is the result of the complementary verification experiment between LBP_M and LBP_S;

[0052] Fig.10 It is a schematic diagram of the computer device structure of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0056] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0057] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0058] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0059] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0060] See also Figure 1 The embodiment of the present invention discloses an image registration method based on multi-channel local binary patterns, comprising the following steps:

[0061] Step 1, such as Figure 3 As shown, the rotation-invariant unified pattern LBP is used to extract the symbol information of the central pixel. This algorithm extracts the relative size relationship between the central pixel and its P neighboring pixels in the circular neighborhood template at the sampling scale R, see Figure 4 , schematic diagram of circular neighborhood systems of different scales, the sampling radius R and the number of neighborhood pixels P are (1,8), (2,16) and (3,24) respectively, and the central pixel is used as the quantization threshold of binary quantization to quantize the neighborhood pixels. When the pixel value of the neighborhood pixel is greater than the pixel value of the central pixel, the neighborhood pixel is quantized to 1. When the pixel value of the neighborhood pixel is less than or equal to the pixel value of the central pixel, the neighborhood pixel is quantized to 0. Subsequently, the obtained local binary string is multiplied by the weight template to convert it into a decimal value as the final local binary pattern of the local area. Local binary patterns are classified into uniform patterns and non-uniform patterns. A uniform pattern means that the U value of the pattern is not greater than 2. On the contrary, a local binary pattern with a U value greater than 2 is classified as a non-uniform pattern. U represents the number of conversions between 0 and 1 characters in the local binary string. Among them, there are P+1 uniform patterns, and there is only 1 non-uniform pattern left. Therefore, the dimension of the final LBP feature histogram is only P+2. The local binary pattern is obtained for each central pixel of the image to be registered, which is defined as the symbol information LBP_S of the central pixel of the image. The LBP_S map generated by the image is statistically analyzed, and finally the generated feature histogram is used as one of the image features.

[0062] Step 2, in order to utilize the information such as amplitude lost during LBP encoding, the amplitude part of the difference vector between the neighborhood pixels and the center pixel in the local area extracted in the first step is extracted. For the amplitude part, the same binary quantization method as LBP is adopted, and the quantization threshold is taken as the mean of the amplitude of the neighborhood difference vector of all center pixels in the entire image, and the amplitude vector is quantized. At the same time, in order to achieve rotation invariance and reduce the dimension of the feature histogram, the amplitude part also adopts the rotation invariant unified mode, and the extracted information is defined as the amplitude information of the center pixel LBP_M.

[0063] In step 3, it is proposed to calculate the gradient of the neighborhood pixels by using simple first-order differences and L2 norms to represent the neighborhood features. Subsequently, the local neighborhood features are encoded using the local adaptive threshold quantization method, which is defined as the LBP_NF feature. Experiments show that it has strong feature identification capabilities and is highly complementary to the sign information LBP_S and the amplitude information LBP_M.

[0064] Step 4, see Figure 6 , introduce a multi-channel framework, construct three independent image feature channels, which are used to transfer the LBP_S, LBP_M and LBP_NF feature information obtained in steps 1, 2 and 3, and generate three independent feature histograms for the extracted features. Subsequently, the three feature histograms are cascaded to obtain the final feature histogram of the image. Finally, the histogram intersection distance is used to calculate the similarity between images for image registration.

[0065] The embodiment of the present invention extracts different texture features LBP_S, LBP_M and LBP_NF on the circular neighborhood of the central pixel, and finds that these features are highly complementary, so multiple features can be selected to fully characterize the texture features of the image to obtain the optimal image registration result; the present invention uses the local rotation-invariant unified LBP to extract features and construct multiple feature descriptors, then establishes a multi-channel feature histogram and uses the histogram intersection distance to calculate the similarity between images, so that the proposed image registration algorithm has stronger image feature extraction capability and image registration accuracy compared to the traditional algorithm while the algorithm complexity is not significantly improved, and also has strong robustness to external environments such as noise and illumination.

[0066] In a feasible embodiment of the present invention,

[0067] The traditional local binary pattern only extracts the sign information of the difference vector between the central pixel and the neighboring pixels, and performs binary quantization to extract features, while the amplitude information of the difference vector is completely ignored. In order to further take the lost amplitude information into account, Guo et al. proposed to use the CLBP (Completed Local Binary Pattern) improved algorithm to describe the amplitude information of the neighborhood pixel difference vector. Drawing on this algorithm, LBP_S is used to describe the sign information of the difference vector, and LBP_M is used to describe the amplitude information of the difference vector, and two feature histogram channels are generated respectively.

[0068] The construction process of the first feature channel LBP_S is as follows:

[0069] First calculate the center pixel g c and its neighboring pixel g p The difference vector d of (p=0,1,...,P-1)p =[g0-g c ,g1-g c ,...,g P-1 -g c ], the sign information LBP_S of the difference vector is encoded using the rotation-invariant unified mode LBP operator, and the sign information of the difference vector is defined as:

[0070]

[0071] In the formula, The U value is used to measure whether a local binary pattern should be classified as a uniform pattern or a non-uniform pattern. It counts the number of changes of 0 / 1 or 1 / 0 in the local binary string. When U is less than or equal to 2, the pattern is a uniform pattern. When the U value is greater than 2, the pattern is a non-uniform pattern. The definition of the U value is as follows:

[0072]

[0073] The construction process of the second feature channel LBP_M is as follows:

[0074] The magnitude information of the difference vector is defined as:

[0075] m p =|g p -g c |(3)

[0076] LBP_M extracts the amplitude information of the difference vector. At the same time, in order to ensure a low feature histogram dimension and robustness to rotation changes, the same method as the rotation-invariant unified mode LBP operator is used to extract the amplitude information of the difference vector. p Perform binary quantization, defined as:

[0077]

[0078] Where t1 is the quantization threshold of binary quantization, and is the amplitude mean of the local neighborhood difference vector of all central pixels in the entire image.

[0079] However, although the amplitude information can provide local texture feature information of the image that is complementary to the sign information, it has certain limitations. First, in certain cases, if LBP_M is not effectively utilized, the amount of information it contains may be low and the feature identification ability may be poor, resulting in inefficient registration performance. Second, LBP_M has a high dimension and may contain redundant image feature information. Therefore, the LBP_NF feature is constructed.

[0080] The construction process of the third feature channel LBP_NF is as follows:

[0081] Most existing LBP variants focus on extracting neighborhood pixels gp The corresponding center pixel g c In this case, the neighborhood feature information will be completely discarded. In LBP, even if the center pixel g c Different from the two local binary patterns, it is also possible that the two local binary patterns obtain the same LBP value, although in this case the neighborhood pixels g p may not be the same. In fact, the feature information of the neighborhood pixels has a strong feature identification ability. Therefore, the present invention proposes to use the neighborhood pixel feature information to provide effective neighborhood feature information. The gradient information is considered to be an important local feature of the image because it can represent the contrast of the local image. For the high contrast area of ​​the image or the edge of the image, the gradient is larger; on the contrary, for the smooth area of ​​the image, the gradient is smaller. Therefore, the gradient is selected as the local neighborhood feature.

[0082] Strictly speaking, the calculation of gradient requires derivatives, but in order to simplify the calculation process and reduce the computational complexity, the approximate derivative of the gradient is obtained by directly calculating the first-order difference of the neighborhood pixels along the circular neighborhood arc. Figure 5 As shown, the two gradients of the defined neighborhood pixels along the clockwise and counterclockwise directions are expressed as and The formula is as follows:

[0083]

[0084] Where P is the number of neighborhood pixels.

[0085] Neighborhood pixel gradient Defined as corresponding and The combination of the L2 norm is defined as follows:

[0086]

[0087] After the above calculation, the neighborhood pixel gradient corresponding to the central pixel can be extracted locally in the image The characteristic vector of P The gradient vector G P It can concisely describe neighborhood features and has many advantages: (1) Gradient information can directly reflect the local contrast of the image; (2) No matter how the image is rotated, the difference vector between neighborhood pixels remains unchanged. Therefore, the gradient vector G P is rotation-invariant; (3) The center pixel g c may be easily contaminated by noise, in which case the difference vector d P will be completely changed, thus losing its feature identification ability. P In contrast, the gradient vector G Pis less susceptible to noise. Therefore, the gradient vector G P Than d P It has stronger noise robustness.

[0088] Gradient vector G P By continuous value Composition. And the difference vector d P Same, G. P It cannot be used directly for registration, but needs to be converted into a binary string. Therefore, the neighborhood feature mode LBP_NF is proposed, and a new encoding method is applied to the gradient vector G P to encode.

[0089] First, image segmentation is performed to obtain N×N sub-images in the entire image. Since the local region of the image has a strong correlation between features, it is proposed to use a local adaptive threshold for each sub-image to adjust the gradient vector G P Then the local adaptive threshold of each sub-image is defined as the value of all Obviously, the local adaptive threshold can better reflect the texture changes in the local area than the global threshold used in LBP and most LBP variants. In order to achieve a balance between computational complexity and feature identification ability, in the experiment, N = 4 is set to obtain 4 × 4 = 16 sub-images. Therefore, the gradient vector G in the local area P It can be encoded into the neighborhood feature pattern LBP_NF and participate in the subsequent image registration task.

[0090] Secondly, there are high-frequency and low-frequency modes in the LBP_NF mode, and their contributions to the registration are not the same. Therefore, they need to be encoded in different ways, as defined below: , the LBP_NF mode is defined as the unified LBP_NF mode; otherwise, it is the non-uniform LBP_NF mode. We choose different A series of experiments are conducted on the UIUC texture database, which contains 25 types of textures, each type contains 40 images, and the size is 640×480. The experimental results show that if you choose is less than or equal to 4, then the frequency of occurrence of non-uniform LBP_NF mode is higher than that of uniform LBP_NF mode. When the sampling radius R = 1 or R = 2, non-uniform LBP_NF patterns almost never appear, which is not a reasonable situation. When it is fixed at 4, the frequency of occurrence of non-uniform LBP_NF mode is consistent with the study of Ojala. Therefore, in the subsequent encoding process of LBP_NF mode, Set to 4.

[0091] Finally, the neighborhood feature (LBP_NF) mode is defined. Inspired by the local binary pattern (LBP), the LBP_NF mode is calculated by comparing the neighborhood pixel gradient with its corresponding local adaptive threshold in the sub-image. Therefore, we introduce the encoding strategy of the rotation-invariant unified pattern LBP operator. The LBP_NF mode is defined as follows:

[0092]

[0093] In the formula, the function s(x) is the same sign function as in formula (1), is the neighborhood pixel g p The corresponding gradient of LBP_NF, U(LBP_NF) represents the number of bit changes between “0” and “1” in the LBP_NF binary string, and the local adaptive threshold t G In the sub-image The mean of .

[0094] The following are specific invention implementation experimental results:

[0095] In order to demonstrate the effectiveness and registration performance of the image registration algorithm based on multi-channel local binary patterns proposed in this invention, we selected the representative UIUC database, CUReT database and Outex database for a series of experiments. Table 1 summarizes the characteristics of these three databases.

[0096] Table 1 Summary of database characteristics used in the experiment

[0097]

[0098] There are many ways to measure the similarity between images. The experiment uses histogram intersection distance to calculate the similarity between images. This method is one of the most commonly used similarity measurement methods in traditional grayscale-based image registration algorithms. It obtains the distance between histograms by calculating the distribution of grayscale histograms. We apply it to the feature histogram in this invention, and the formula is as follows:

[0099]

[0100] Among them, F M(i) and F N(i) are the feature histograms of texture features extracted from the training image and the test image, respectively, and L is the dimension of the two histograms.

[0101] In order to verify the complementarity between LBP_S, LBP_M and LBP_NF features, we conduct a series of experiments on the Outex image database. The Outex_TC12'h' subset in the Outex dataset is selected for testing, which contains 4320 images of 24 categories. They are taken under different lighting conditions and viewing angles.

[0102] like Figure 7 , Figure 8 and Fig. 9 As shown, LBP_S, LBP_M and LBP_NF are highly complementary. Compared with LBP_S, LBP_NF can correctly register 2494 images, while LBP_M can correctly register 2505 images. LBP_NF can register 734 images that LBP_S cannot correctly register and 639 images that LBP_M cannot correctly register. Therefore, in order to make full use of local features and improve the registration performance of the image registration algorithm, the present invention proposes to introduce the concept of a multi-channel framework into image registration, and combine LBP_S, LBP_M and LBP_NF to jointly extract image features.

[0103] The UIUC database contains 40 high-definition images of 25 categories, all of which are taken in real-world environments with dramatic changes in viewing angles. The experiment randomly selects 40-N images as test images and N images as training images. In the experiment, N is 5, 10, 15, and 20 respectively. To eliminate randomness, each experiment is repeated 100 times, and the average accuracy of the registration algorithm is taken as the final registration result.

[0104] Table 2 below lists the registration accuracy of the traditional registration algorithm based on image grayscale histogram (Histogram), LBP algorithm (LBP) and our proposed image registration algorithm based on multi-channel local binary pattern (Proposed). In Table 1, 20, 15, 10 and 5 represent the number of training images N, R is the sampling scale, and P is the number of neighborhood pixels. Regardless of how R and P are selected, we observe that the registration performance of our proposed method is better than other methods. The results in Table 2 show that the proposed method achieves the highest registration accuracy of 82.33%, 79.58%, 74.12% and 63.31% when (R, P) = (3, 24). Compared with LBP, the registration accuracy is 21.68%, 23.22%, 23.12% and 21.89% higher, respectively, and 38.72%, 38.53%, 36.91% and 31.86% higher than the traditional algorithm based on image grayscale histogram. The comparison of registration accuracy proves that the registration algorithm proposed in the present invention has excellent registration performance.

[0105] Table 2 Image registration accuracy on UIUC database

[0106]

[0107]

[0108] The CUReT database contains 92 images of 61 categories, and the images have changes in illumination, perspective, and rotation. The experiment randomly selects 92-N images as test images and N images as training images. In the experiment, N is 46, 23, 12, and 6 respectively. Each experiment is repeated 100 times, and the average image registration accuracy is taken as the final registration result. The registration accuracy of the algorithm is shown in Table 3. In Table 3, R is the sampling scale and P is the number of pixels in the sampling neighborhood.

[0109] On the CUReT database, see Table 3 below, the registration algorithm of the present invention is significantly superior to LBP and the traditional registration algorithm based on the image grayscale histogram in terms of accuracy. When (R, P) = (1, 8), the registration algorithm of the present invention achieves registration accuracies of 89.32%, 83.24%, 75.27% and 65.26%, while the traditional registration algorithm based on the image grayscale histogram only achieves accuracies of 47.95%, 41.26%, 34.86% and 28.05%. Similarly, for (R, P) = (3, 24), the image registration algorithm based on multi-channel local binary patterns can achieve registration accuracies of 92.25%, 86.80%, 79.38% and 69.09%, which are 4.85%, 4.96%, 4.47% and 3.3% higher than LBP. Therefore, the image registration algorithm proposed in the present invention exhibits excellent registration performance.

[0110] Table 3 Image registration accuracy on the CUReT database

[0111]

[0112] The Outex database contains two image subsets: Outex_TC10 and Outex_TC12. Both subsets contain 24 categories of images, each category contains 9 rotation angle changes (0°, 5°, 10°, 15°, 30°, 45°, 60°, 75° and 90°), and each contains 20 images. Outex_TC10 was acquired under the "inca" lighting condition, while Outex_TC12 was acquired under the "tl84" and "horizon" lighting conditions. In the experiment, 24×20 images with a rotation angle of 0° in the Outex_TC10 image subset were selected as training images, and the remaining 24×20×8 images (other 8 rotation angles) were selected as test images. For the Outex_TC12 image subset, the same training images as Outex_TC10 were selected, and all their own images were set as test images. In Table 4, R is the sampling scale and P is the number of neighborhood pixels. "h" stands for the lighting condition "horizon" and "t" stands for the lighting condition "tl84".

[0113] The registration accuracy results in Table 4 show that the image registration algorithm based on multi-channel local binary patterns is superior to other algorithms in registration performance. Compared with the LBP algorithm and the traditional registration algorithm based on image grayscale histogram, the image registration algorithm proposed in this invention has excellent registration accuracy on all Outex subsets and all (R, P), showing excellent image registration performance.

[0114] Table 4 Image registration accuracy on Outex database

[0115]

[0116] See also Figure 2 The embodiment of the present invention discloses an image registration system based on multi-channel local binary patterns, comprising:

[0117] The symbol information module uses the rotation-invariant uniform pattern LBP to extract the symbol information of the central pixel. The algorithm extracts the relative size relationship of the pixel values ​​between the central pixel and its P neighboring pixels in the circular neighborhood template on the sampling scale R, and uses the central pixel as the quantization threshold for binary quantization to quantize the neighborhood pixels. When the pixel value of the neighborhood pixel is greater than the pixel value of the central pixel, the neighborhood pixel is quantized to 1. When the pixel value of the neighborhood pixel is less than or equal to the pixel value of the central pixel, the neighborhood pixel is quantized to 0. Subsequently, the obtained local binary string is multiplied by the weight template to convert it into a decimal value as the final local binary pattern of the local area. The local binary pattern is classified into a uniform pattern and a non-uniform pattern. The uniform pattern means that the U value of the pattern is not greater than 2. On the contrary, the local binary pattern with a U value greater than 2 is classified as a non-uniform pattern. U represents the number of conversions between 0 and 1 characters in the local binary string. Among them, there are P+1 uniform patterns, and there is only 1 non-uniform pattern left. Therefore, the final LBP feature histogram dimension is only P+2. The local binary pattern is obtained for each central pixel of the image to be registered, which is defined as the symbol information LBP_S of the central pixel of the image. The LBP_S map generated by the image is statistically analyzed, and finally the generated feature histogram is used as one of the image features.

[0118] In order to utilize the information such as amplitude lost during LBP encoding, the amplitude information module extracts the amplitude part of the difference vector between the neighborhood pixels and the center pixel in the local area extracted in the first step. For the amplitude part, the same binary quantization method as LBP is adopted, and the quantization threshold is set as the mean of the amplitude of all the center pixel neighborhood difference vectors in the entire image to quantize the amplitude vector. At the same time, in order to achieve rotation invariance and reduce the dimension of the feature histogram, the amplitude part also adopts the rotation invariant unified mode, and the extracted information is defined as the amplitude information of the center pixel LBP_M.

[0119] The neighborhood feature module proposes to represent the neighborhood features by calculating the gradient of the neighborhood pixels using simple first-order differences and L2 norms. Subsequently, the local neighborhood features are encoded using the local adaptive threshold quantization method, which is defined as the LBP_NF feature. Experiments show that it has strong feature discrimination capabilities and is highly complementary to the sign information LBP_S and the amplitude information LBP_M.

[0120] The image registration module introduces a multi-channel framework and constructs three independent image feature channels to transmit LBP_S, LBP_M and LBP_NF feature information. The extracted features are respectively generated into three independent feature histograms. Subsequently, the three feature histograms are cascaded to obtain the final feature histogram of the image. Finally, the histogram intersection distance is used to calculate the similarity between images for image registration.

[0121] In one embodiment of the present invention, see Fig.10 , a computer device is provided, the computer device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the image registration method based on multi-channel local binary patterns.

[0122] The present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the image registration method based on multi-channel local binary patterns in the above embodiment.

[0123] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An image registration method based on multi-channel local binary patterns, characterized in that: The following steps are involved: The symbol information of the central pixel of the image to be registered is extracted using the rotation invariant unified pattern LBP; the neighborhood pixels are quantized to obtain the local binary pattern of the local area according to the difference vector between the central pixel and the neighborhood pixels of the local area; the local binary pattern is calculated one by one for each central pixel of the image to be registered, and the symbol information LBP_S of the central pixel of the image to be registered is obtained; Extract the amplitude part of the difference vector between the central pixel and the neighboring pixels, and quantize the amplitude part to obtain the amplitude information LBP_M of the central pixel; The first-order difference and L2 norm are used to calculate the gradient of the neighborhood pixels, and the local adaptive threshold quantization method is used to encode the gradient to obtain the neighborhood feature LBP_NF; Based on the sign information LBP_S, amplitude information LBP_M and neighborhood feature LBP_NF, three independent image feature channels are constructed respectively to generate three feature histograms. The three feature histograms are cascaded to obtain the final feature histogram of the image. The final feature histogram intersection distance is used to calculate the similarity between images for image registration.

2. The image registration method based on multi-channel local binary patterns according to claim 1, characterized in that: The steps of using the rotation invariant unified pattern LBP to extract the symbol information of the central pixel of the image to be registered; quantizing the neighboring pixels according to the central pixel of the local area and the neighborhood pixel difference vector to obtain the local binary pattern of the local area; obtaining the local binary pattern of each central pixel of the image to be registered one by one to obtain the symbol information LBP_S of the central pixel of the image to be registered specifically include: Calculate the difference vector between the central pixel and P neighboring pixels in the circular neighborhood template on the sampling scale R, use the central pixel as the quantization threshold of binary quantization, and quantize the neighboring pixels; when the pixel value of the neighboring pixel is greater than the pixel value of the central pixel, the neighboring pixel is quantized to 1; when the pixel value of the neighboring pixel is less than or equal to the pixel value of the central pixel, the neighboring pixel is quantized to 0; after the quantization is completed, a local binary string is obtained; The obtained local binary string is multiplied by the weight template to be converted into a decimal value as the final local binary pattern of the local area; The local binary pattern is calculated for each central pixel of the image to be registered one by one to obtain the symbol information LBP_S of the central pixel of the image to be registered.

3. The image registration method based on multi-channel local binary patterns according to claim 2, characterized in that: The local binary pattern includes a uniform pattern and a non-uniform pattern; the uniform pattern is a local binary pattern with a U value not greater than 2; the non-uniform pattern is a local binary pattern with a U value greater than 2; and U represents the number of conversions between 0 and 1 characters in the local binary string.

4. The image registration method based on multi-channel local binary patterns according to claim 1, characterized in that: The step of extracting the amplitude part of the difference vector between the central pixel and the neighboring pixels and quantizing the amplitude part to obtain the amplitude information LBP_M of the central pixel specifically includes: The amplitude part of the difference vector between the central pixel and the neighboring pixels is extracted, and the amplitude part is binary quantized using the rotation-invariant unified pattern LBP. The quantization threshold is the mean of the amplitudes of all the central pixel neighborhood difference vectors in the entire image. Finally, the amplitude information LBP_M of the central pixel is quantized.

5. The image registration method based on multi-channel local binary patterns according to claim 1, characterized in that: The step of using the first-order difference and the L2 norm to calculate the gradient of the neighborhood pixels, and using the local adaptive threshold quantization method to encode the gradient to obtain the neighborhood feature LBP_NF specifically includes: The neighborhood pixel gradient in the clockwise direction is defined as The neighborhood pixel gradient in the counterclockwise direction is defined as The specific formula is: Where P is the number of neighborhood pixels; g is the neighborhood pixel; Neighborhood pixel gradient Defined as corresponding and The combination of the L2 norm is: The neighborhood pixel gradients are encoded to obtain the neighborhood feature LBP_NF.

6. The image registration method based on multi-channel local binary patterns according to claim 5, characterized in that: The step of encoding the neighborhood pixel gradient to obtain the neighborhood feature LBP_NF specifically includes: dividing the image to be registered into N×N sub-images, using the local adaptive threshold of each sub-image to encode the gradient vector Binary quantization is performed to obtain the neighborhood feature LBP_NF.

7. The image registration method based on multi-channel local binary patterns according to claim 6, characterized in that: The local adaptive threshold of each sub-image is the value of all The mean of .

8. An image registration system based on multi-channel local binary patterns, characterized in that: include: A symbol information module is used to extract the symbol information of the central pixel of the image to be registered using a rotation-invariant unified pattern LBP; According to the difference vector between the central pixel and the neighboring pixels of the local area, the neighboring pixels are quantized to obtain the local binary pattern of the local area; the local binary pattern is calculated one by one for each central pixel of the image to be registered, and the symbol information LBP_S of the central pixel of the image to be registered is obtained; Amplitude information module, used to extract the amplitude part of the difference vector between the central pixel and the neighboring pixels, and quantize the amplitude part to obtain the amplitude information LBP_M of the central pixel; The neighborhood feature module is used to calculate the gradient of the neighborhood pixels using the first-order difference and L2 norm, and adopt the local adaptive threshold quantization method to encode the gradient to obtain the neighborhood feature LBP_NF; The image registration module is used to construct three independent image feature channels based on the symbol information LBP_S, the amplitude information LBP_M and the neighborhood feature LBP_NF, and generate three feature histograms; the three feature histograms are cascaded to obtain the final feature histogram of the image; the final feature histogram intersection distance is used to calculate the similarity between images for image registration.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.