Digital image processing method, hardware circuit and feature vector storage and matching method

Abstract

The invention relates to a digital image processing method, a hardware circuit and a feature vector storage and matching method in the field of integrated circuit design. The digital image processing method comprises the following steps: in a digital image, defining a next circular area by taking each feature point as a center and taking an r-grid pixel as a radius; 8 fan-shaped areas are divided every 45 degrees from the central angles of 0 degree and-22.5 degrees, and cross coding is carried out in sequence; carrying out 1-m sequential coding on m amplitude storage positions of the storage content of the same storage unit; rotating the digital image to enable the main direction angle to be 0 degree; and changing the information storage position of the storage information of the fan-shaped area before rotation and the amplitude storage positions of the corresponding m amplitudes. According to the method, 16 fan-shaped regions with an overlapping relation are sketched on the basis of delimiting the circular region for each feature point, so that after rotation according to the main direction angle, the circular region does not need to be counted again, only the stored sequence needs to be changed, the feature vector generation process is simplified, and hardware implementation is facilitated.

Description

Technical Field

[0001] The present invention relates to a digital image processing method and hardware circuit, a feature vector storage and matching method in the field of integrated circuit design, and in particular to a digital image processing method that is easy to implement with hardware, a hardware circuit for implementing information storage position transformation in the digital image processing method, another hardware circuit for implementing amplitude storage position transformation in the digital image processing method, a feature vector storage method for reducing the area consumption of storing feature vectors, and a feature vector matching method using the feature vector storage method. Background Art

[0002] SIFT (Scale-Invariant Feature Transform) is an algorithm for image feature extraction, proposed by David Lowe in 1999 and improved in 2004. SIFT detects local feature points in an image and generates feature descriptors with scale invariance and rotation invariance for them, so that the algorithm can recognize the same object or scene under different scales, rotations, and lighting changes. SIFT is an important breakthrough in the fields of image matching, splicing, and recognition, and has become a classic algorithm in the fields of computer vision and image processing.

[0003] SIFT can be used to extract feature vectors for feature comparison from each feature point in a digital image. The feature vectors generally have 8 direction angles and 8 amplitudes corresponding to the 8 direction angles. Feature comparison is used to identify whether the image is the same or whether the same feature part exists. However, in order to improve the accuracy of feature matching, the current digital image processing methods are relatively complex in hardware implementation. Summary of the invention

[0004] Based on this, in order to solve the technical problem that the existing digital image processing methods are difficult to implement in hardware, the present invention provides a digital image processing method that is easy to implement in hardware, a hardware circuit for implementing information storage position transformation in the digital image processing method, another hardware circuit for implementing amplitude storage position transformation in the digital image processing method, a feature vector storage method for reducing the area consumption for storing feature vectors, and a feature vector matching method using the feature vector storage method.

[0005] The objective of the present invention is achieved through the following technical solution: a digital image processing method that is easy to implement with hardware, which is used to extract a feature vector for feature comparison from each feature point in the digital image, wherein the feature vector has m direction angles and m amplitudes corresponding to the m direction angles one by one; the digital image processing method comprises the following steps:

[0006] In the digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer; for the circular area, 8 sector-shaped areas are divided every 45° from the center angle of 0°, and the storage information of the 8 sector-shaped areas is stored in 8 storage units coded by 1, 3, 5, 7, 9, 11, 13, and 15 in sequence; for the circular area, 8 sector-shaped areas are re-divided every 45° from the center angle of -22.5°, and the storage information of the re-divided 8 sector-shaped areas is stored in another 8 storage units coded by 2, 4, 6, 8, 10, 12, 14, and 16 in sequence; the mutual positions of the 16 stored information are the information storage positions;

[0007] For the storage content of the same storage unit, m amplitudes S n,1 ~S n,m The amplitude storage position of S is encoded sequentially from 1 to m; n,m Represents the amplitude at the mth direction angle in the nth sector area, m amplitudes S n,1 ~S n,m That is the storage information of the nth sector area;

[0008] Rotating the digital image so that the main direction angle of the circular area is 0°;

[0009] The storage information on the 16 storage units sequentially coded from 1 to 16 is sequentially updated according to the rotation angle of the digital image, and the information storage position of the storage information of the fan-shaped area before the rotation is changed;

[0010] The stored contents at the m signal positions encoded sequentially from 1 to m are also updated sequentially according to the rotation angle of the digital image, and the m amplitudes S of the stored information in the fan-shaped area before the rotation are changed. n,1 ~S n,m The amplitude storage location of

[0011] The storage information sequentially output by the 16 storage units is used as the feature vector.

[0012] As a further improvement of the above solution, the digital image processing method further includes the following steps:

[0013] The eigenvectors are subjected to sequential amplitude binarization processing:

[0014] For the circular area, 8 sector-shaped areas are divided every 45° from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, 15 are sequentially encoded to define 8 corresponding odd sub-areas;

[0015] For the circular area, starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and even-numbered codes of 2, 4, 6, 8, 10, 12, 14, and 16 are performed in sequence, thereby defining corresponding 8 even sub-areas;

[0016] Starting from the sector area coded 1, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes to form a feature vector after amplitude binarization in sequence for feature comparison.

[0017] As a further improvement of the above solution, the digital image processing method further includes the following steps:

[0018] Perform skip-order amplitude binarization processing on the feature vector:

[0019] For the circular area, 8 sector-shaped areas are divided every 45° from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, 15 are sequentially encoded to define 8 corresponding odd sub-areas;

[0020] For the circular area, starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and even-numbered codes of 2, 4, 6, 8, 10, 12, 14, and 16 are performed in sequence, thereby defining corresponding 8 even sub-areas;

[0021] Starting from the fan-shaped area with code 1, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code of m amplitudes of the corresponding fan-shaped area is obtained in sequence;

[0022] Starting from the fan-shaped area of ​​code 2, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area with the same even number code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code of m amplitudes of the corresponding fan-shaped area is obtained in sequence;

[0023] The 16 binary codes of the 16 sector-shaped areas are serially concatenated according to the codes to form a jump sequence amplitude binarization feature vector which is also used for feature comparison.

[0024] As a further improvement of the above scheme, r=9.

[0025] As a further improvement of the above solution, m=8, and the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° respectively.

[0026] The present invention also provides a hardware circuit for changing storage positions, which is used to implement the information storage position change in any of the above digital image processing methods that are easy to implement with hardware. The hardware circuit includes 16 sector-shaped area information storage circuits corresponding to the 16 sector-shaped areas one by one, and each sector-shaped area information storage circuit includes:

[0027] Adder 1, one of the addend input terminals of the adder 1 receives an addend: K1-1, K1 is the sector area number value corresponding to the sector area information storage circuit, and another addend input terminal of the adder 1 receives an addend: 2(L-1), L is the encoding value of the amplitude of the main direction angle in the m amplitude storage positions;

[0028] A multiplexer 1, whose input is the storage information of the sector area corresponding to the sector area information storage circuit, and whose selection control end is the output of the adder 1, controls the output end of the multiplexer 1 to select the storage information of the corresponding sector area from the 16 sector areas.

[0029] As a further improvement of the above solution, the hardware circuit further includes m amplitude storage circuits for realizing the change of amplitude storage position in the same sector area and corresponding to the m amplitudes one by one, and each amplitude storage circuit includes:

[0030] Adder 2, one of the addend input terminals of the adder 2 receives an addend: K2-1, K2 is an amplitude coding value corresponding to the amplitude storage circuit, and another addend input terminal receives an augend: L-1;

[0031] The input of the multiplexer 2 is the m amplitudes corresponding to the amplitude storage circuit, and the selection control terminal is the output of the adder 2, which controls the output terminal of the multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

[0032] The present invention also provides another hardware circuit for changing storage positions, which is used to implement the amplitude storage position change in any of the above digital image processing methods that are easy to implement with hardware. The hardware circuit includes m amplitude storage circuits corresponding to m amplitudes one by one, and each amplitude storage circuit includes:

[0033] Adder 2, one of the addend input terminals of the adder 2 receives an addend: K2-1, K2 is an amplitude coding value corresponding to the amplitude storage circuit, and another addend input terminal receives an augend: L-1;

[0034] The input of the multiplexer 2 is the m amplitudes corresponding to the amplitude storage circuit, and the selection control terminal is the output of the adder 2, which controls the output terminal of the multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

[0035] The present invention also provides a feature vector storage method for reducing the area consumption of storing feature vectors, which comprises the following steps:

[0036] In the digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer and satisfies; for the circular area, starting from the central angle of 0°, 8 sector areas are divided every 45°, and odd codes of 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially performed to define 8 corresponding odd sub-areas; starting from the central angle of -22.5°, 8 sector areas are re-divided every 45°, and even codes of 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially performed to define 8 corresponding even sub-areas; for each sub-area of ​​the same feature point, there are corresponding m amplitudes at m direction angles;

[0037] Perform sequential amplitude binarization processing: starting from the fan-shaped area of ​​code 1, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area of ​​code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of m amplitudes of the corresponding fan-shaped area is obtained in sequence. The 16 binary codes of the 16 fan-shaped areas are serially connected according to the codes as the feature vector after sequential amplitude binarization;

[0038] Perform skip sequence amplitude binarization processing: starting from the fan-shaped area of ​​code 1, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area with the same odd number code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0, and a binary code about m amplitudes of the corresponding fan-shaped area is obtained in sequence; starting from the fan-shaped area of ​​code 2, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area with the same even number code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0, and a binary code about m amplitudes of the corresponding fan-shaped area is obtained in sequence; 16 binary codes of 16 fan-shaped areas are serially connected according to the codes as the feature vector after skip sequence amplitude binarization;

[0039] The sequential amplitude binarization feature vectors and the skip-sequence amplitude binarization feature vectors are stored as feature vectors for feature comparison of the corresponding feature points.

[0040] As a further improvement of the above scheme, r=9.

[0041] As a further improvement of the above solution, m=8, and the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° respectively.

[0042] The present invention also provides a feature vector matching method, which extracts a feature vector of each feature point from multiple feature points in two different digital images for feature comparison; wherein the feature vector matching method uses any of the above-mentioned digital image processing methods that are easy to implement in hardware to extract the feature vector.

[0043] Compared with the prior art, the digital image processing method of the present invention, which is easy to implement in hardware, outlines 16 overlapping sector-shaped areas for each feature point on the basis of demarcating a circular area. Therefore, after rotating according to the main direction angle, the circular area does not need to re-count the rotated diamond pixel area into a square pixel area. It only needs to change the storage order. In combination with the 16 overlapping sector-shaped areas, the feature vector generation process can be greatly simplified, which is conducive to hardware implementation.

[0044] The hardware circuit of the present invention, based on "drawing out 16 overlapping sector-shaped areas on the basis of demarcated circular areas", can easily realize the specific hardware circuits of "information storage position transformation between different sector-shaped areas" and "amplitude storage position transformation within the same sector-shaped area". The specific hardware circuits are so simple that only a few adders and a few multiplexers are needed. Combined with a specific control method, it can be easily realized.

[0045] The feature vector storage method for reducing the area consumption of storing feature vectors of the present invention can perform sequential amplitude binarization processing and jump sequence amplitude binarization processing on the basis of "delimiting the circular area and outlining 16 overlapping fan-shaped areas", which can reduce the area consumption of storing feature vectors. For example, each element of the original feature vector is implemented as a 10-bit fixed-point number in hardware, and each feature vector contains 128 elements, that is, each feature vector requires 1.25kb to store. However, the present invention only contains 256 binary numbers, and the storage of each feature vector only requires 256 bits, and the storage consumption is reduced by 80%.

[0046] The feature vector matching method of the present invention is based on "outlining 16 overlapping sector-shaped areas on the basis of demarcated circular areas". By using the method of area overlap, the amplitude and angle of pixels in different divided areas are repeatedly counted, thereby ensuring the stability of feature vector matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The present invention provides a flowchart of a digital image processing method.

[0048] Figure 2 for Figure 1 Schematic diagram of the circular area in the middle divided into 8 odd sub-areas.

[0049] Figure 3 for Figure 1 Schematic diagram of dividing the circular area into 8 even sub-areas.

[0050] Figure 4 for Figure 2 , Figure 3 Schematic diagram of the 16 sub-regions composed of odd sub-regions and even sub-regions.

[0051] Figure 5 for Figure 3 Schematic diagram of the middle circular area after rotation according to the main direction angle, in which the eight direction angle amplitudes of the fan-shaped area numbered 4 are used as an example for illustration.

[0052] Figure 6 for Figure 5 Hardware implementation diagram of the transformation of neutron region information storage location.

[0053] Figure 7 for Figure 5 Hardware implementation diagram of the transformation of the amplitude storage position of all direction angles in each sub-region.

[0054] Figure 8 for Figure 5 The example diagram of the sequential amplitude binarization of the sector area numbered 4 in the figure.

[0055] Fig. 9 for Figure 5 An example diagram of the binarization of the jump sequence amplitude in the sector area numbered 4.

[0056] Fig.10 for Figure 5 Medium circular area Figure 8 , Fig. 9 The matching effect diagram of the binary feature vector obtained after binarization. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] The digital image processing method provided in this embodiment is easy to implement by hardware, and is used to extract a feature vector for feature comparison of each feature point in the digital image, wherein the feature vector has m direction angles and m amplitudes corresponding to the m direction angles. In this embodiment, m is 8 according to the current existing standard, and the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, respectively.

[0059] See also Figure 1 , which is a flow chart of a digital image processing method according to Embodiment 1 of the present invention. The digital image processing method comprises the following steps: S1 to S6.

[0060] Step S1, in the digital image, with each feature point as the center and r pixels as the radius, a circular area is defined, where r is a positive integer and satisfies; for the circular area, starting from a central angle of 0°, 8 sector-shaped areas are divided every 45°, and the storage information of the 8 sector-shaped areas is stored in 8 storage units encoded by 1, 3, 5, 7, 9, 11, 13, and 15 in sequence; for the circular area, starting from a central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and the storage information of the re-divided 8 sector-shaped areas is stored in another 8 storage units encoded by 2, 4, 6, 8, 10, 12, 14, and 16 in sequence; the relative positions of the 16 stored information are the information storage positions.

[0061] In this example, see Figure 2 As shown, according to the current standard, r is set to 9, because the radius of 9 pixels can basically cover the effective pixel area centered on the feature point, so the value of r needs to satisfy the requirement that the covered pixel area is the effective pixel area centered on the feature point. The circular domain with a circular radius of 9 pixels is divided into odd sub-areas and even sub-areas. Starting from 0°, a sector area is divided every 45°. They are numbered 1, 3, 5, 7, 9, 11, 13, and 15, and these areas are called odd sub-areas. Please combine Figure 3 Starting from -22.5°, a sector is divided every 45 degrees, and they are numbered 16, 2, 4, 6, 8, 10, 12, and 14. These areas are called even sub-areas. The odd sub-areas and even sub-areas together constitute the characteristic domain of this article. The odd sub-areas and even sub-areas obtained in the end have an overlapping relationship, such as Figure 4 shown.

[0062] Step S2, for the storage content of the same storage unit, m amplitudes S n,1 ~S n,m The amplitude storage position of S is encoded sequentially from 1 to m; n,m Represents the amplitude at the mth direction angle in the nth sector area, m amplitudes Sn,1 ~S n,m That is the storage information of the nth sector area.

[0063] That is to say, for each sector area of ​​the same feature point, the corresponding m amplitudes S at m direction angles are counted. n,1 ~S n,m , S n,m Represents the amplitude at the mth direction angle in the nth sector area, m amplitudes S n,1 ~S n,m That is, the storage information of the corresponding sector area, m amplitudes S n,1 ~S n,m The relative position of is the amplitude storage position; for the storage content S of the same storage unit n,1 ~S n,m The m amplitude storage locations are also encoded sequentially from 1 to m.

[0064] like Figure 5 The fan-shaped area (even area) numbered 4 has amplitudes at 8 direction angles. The 8 direction angles of 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° are numbered 1, 2, 3, 4, 5, 6, 7, and 8 in sequence.

[0065] Step S3, rotating the digital image so that the main direction angle of the circular area is 0°.

[0066] At the same direction angle, the amplitudes corresponding to the corresponding direction angles in all fan-shaped areas are accumulated, the direction angle with the largest amplitude is selected as the main direction angle of the circular area, and the digital image is rotated so that the main direction angle is 0°.

[0067] Step S4, sequentially updating the storage information on the 16 storage units sequentially encoded from 1 to 16 according to the rotation angle of the digital image, and changing the information storage position of the storage information in the fan-shaped area before rotation.

[0068] Step S5, the stored contents at the m signal positions encoded sequentially from 1 to m are also updated sequentially according to the rotation angle of the digital image, and the m amplitudes S of the stored information in the fan-shaped area before the rotation are changed. n,1 ~S n,m The amplitude storage location.

[0069] from Figure 5 In the example of the original sector area numbered 4, it can be seen that the storage information of S4 is stored in the sector area numbered 16, realizing the change of the information storage location. The 8 amplitudes of the original sector area numbered 4 also have their amplitude storage locations changed.

[0070] When the main direction angle is 90°, for sector area 4, the information stored therein is placed in the storage position of sector area 16. The storage of other areas changes accordingly: 1->13, 2->14, 3->15, 5->1, 6->2, 7->3, 8->4, 9->5, 10->6, 11->7, 12->8, 13->9, 14->10, 15->11, 16->12. At the same time, the stored direction angle amplitude will also change the storage position accordingly. For example, the direction angle with the longest amplitude in the original 4 areas is 90°. After rotation, its amplitude information is stored in the amplitude position corresponding to the 0° angle of area 16.

[0071] Step S6, taking the storage information sequentially output by the 16 storage units as the feature vector.

[0072] In the above steps of the digital image processing method, the information storage position conversion of step S4 and the amplitude storage position conversion of step S5 can be implemented by the following hardware circuit for converting storage positions. The hardware circuit may include 16 sector area information storage circuits corresponding to the 16 sector areas one by one, and may also include m amplitude storage circuits corresponding to the m amplitudes one by one.

[0073] Each sector area information storage circuit includes: adder 1 and multiplexer 1. Each amplitude storage circuit includes: adder 2 and multiplexer 2.

[0074] One of the addend inputs of adder 1 receives the addend: K1-1, where K1 is the sector area number value corresponding to the sector area information storage circuit. Another addend input of adder 1 receives the addend: 2(L-1), where L is the encoding value of the amplitude of the main direction angle in the m amplitude storage positions. The input of multiplexer 1 is the storage information of the sector area corresponding to the sector area information storage circuit, and its selection control end is the output of adder 1, which controls the output of multiplexer 1 to select the storage information of the corresponding sector area from the 16 sector areas.

[0075] In this embodiment, the hardware implementation of the regional storage information rotation is shown in FIG. Figure 6, when implemented, the detected main direction is divided into 8 bins, that is, the main direction has only 8 values, 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°, corresponding to main_ori_bin values ​​0, 1, 2, 3, 4, 5, 6, 7 respectively. The direction angle amplitude information stored in the fan-shaped sub-area corresponds to S[0:15] respectively. The copy information of all sub-areas is broadcast to the input end of 16 MUXs (i.e., multiplexer 1), and the selection signal of MUX is obtained by adding twice the main_ori_bin value to the number of the sub-area. After MUX selection, the vector information S_aft_rotate stored in each sub-area after rotation is obtained.

[0076] One of the addend input terminals of the adder 2 receives the addend: K2-1, K2 is the amplitude code value corresponding to the amplitude storage circuit, and the other addend input terminal receives the augend: L-1. The input of the multiplexer 2 is the m amplitudes corresponding to the amplitude storage circuit, and its selection control terminal is the output of the adder 2, which controls the output terminal of the multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

[0077] After obtaining S_aft_rotate, you also need to rotate all the angular amplitudes in the sub-region, see Figure 7 Taking region 16 as an example, the eight azimuth amplitude information stored in region 16 corresponds to S16_aft_rotate[0:7]. S16_aft_rotate[0:7] is broadcast to the input of 8 MUXs (i.e., multiplexer 2), and the selection signal of MUX is obtained by subtracting the value of main_ori_bin from the azimuth number. After MUX selection, the final storage vector S16_final[0:7] can be obtained. The storage vectors of other sub-regions can also be obtained in the same way.

[0078] In order to facilitate the storage of digital image processing data, the digital image processing method may further include the following steps: performing sequential amplitude binarization processing and skip-sequence amplitude binarization processing on the feature vector. That is, sequential amplitude binarization processing and skip-sequence amplitude binarization processing can be added on the basis of the digital image processing method that is easy to implement in hardware in this embodiment.

[0079] Sequential amplitude binarization processing: starting from the sector area coded 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code for the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes to form a feature vector after sequential amplitude binarization for feature comparison.

[0080] In this embodiment, for the outer sector area S, each area is represented by the subscript n, and the angular amplitude in each area is represented by the subscript m. Then S n,m Represents the magnitude of the mth direction angle in the fan-shaped area n.

[0081] See also Figure 8 , is the sequential amplitude binarization abstraction method of the present invention, which compares the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area. If the amplitude is greater than, the feature vector element at that position is 1, and if it is less than, it is 0. Figure 6 In the fan-shaped area 1, the 0° angle corresponds to the amplitude S 1,1 Smaller than the corresponding amplitude S of 0° angle in sector area 2 2,1 , then Vseq 1,1 = 1. According to this method, Vseq can be obtained in turn 1,2 =1, Vseq 1,3 =1, Vseq 1,4 =0, Vseq 1,5 =1, Vseq 1,6 =1, Vseq 1,7 =0, Vseq 1,8 =0.

[0082] Binarization of jump sequence amplitude:

[0083] (1) Starting from the fan-shaped area with code 1, the amplitude of the direction angle in the fan-shaped area is compared with the amplitude of the same direction angle in the next fan-shaped area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code of m amplitudes of the corresponding fan-shaped area is obtained in sequence;

[0084] (2) Starting from the fan-shaped area with code 2, the amplitude of the direction angle in the fan-shaped area is compared with the amplitude of the same direction angle in the next fan-shaped area with the same even code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code of the m amplitudes of the corresponding fan-shaped area is obtained in sequence;

[0085] (3) The 16 binary codes of the 16 sector-shaped areas are concatenated in sequence according to the codes to form a jump sequence amplitude binarization feature vector which is also used for feature comparison.

[0086] See also Fig. 9 The jump sequence amplitude binarization abstraction method of the present invention compares the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area of ​​the same odd subdomain or even subdomain. If the amplitude is greater than, the characteristic vector element at this position is 1, and if it is less than, it is 0. As shown in the figure, the 0° angle in the fan-shaped area 1 corresponds to the amplitude S 1,1Greater than the corresponding amplitude S of 0° angle in sector area 3 3,1 , then Vuseq 1,1 = 1. According to this method, Vuseq can be obtained in turn 1,2 =1, Vuseq 1,3 =1, Vuseq 1,4 =0, Vuseq 1,5 =1, Vuseq 1,6 =1, Vuseq 1,7 =0, Vuseq 1,8 =0.

[0087] Each element of the original feature vector is implemented as a 10-bit fixed-point number in hardware, and each feature vector contains 128 elements, that is, each feature vector requires 1.25kb to store. After binarization, the feature vector only contains 256 binary numbers, and each feature vector only requires 256 bits to store. The storage consumption is reduced by 80%.

[0088] In combination with the digital image processing method that is easy to implement in hardware in this embodiment, a feature vector storage method for reducing the area consumption of storing feature vectors is also shown by adding sequential amplitude binarization processing and skipped amplitude binarization processing. The feature vector storage method includes the following steps:

[0089] In the digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer and satisfies; for the circular area, starting from the central angle of 0°, 8 sector-shaped areas are divided every 45°, and odd codes of 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially performed to define the corresponding 8 odd sub-areas; starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and even codes of 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially performed to define the corresponding 8 even sub-areas;

[0090] For each sub-region of the same feature point, the corresponding m amplitudes S at m direction angles are counted n,1 ~S n,m , S n,m Represents the amplitude at the mth direction angle in the nth sub-region;

[0091] Perform sequential amplitude binarization processing: starting from the fan-shaped area of ​​code 1, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area of ​​code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of m amplitudes of the corresponding fan-shaped area is obtained in sequence. The 16 binary codes of the 16 fan-shaped areas are serially connected according to the codes as the feature vector after sequential amplitude binarization;

[0092] Perform skip sequence amplitude binarization processing: starting from the fan-shaped area of ​​code 1, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area with the same odd number code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0, and a binary code about m amplitudes of the corresponding fan-shaped area is obtained in sequence; starting from the fan-shaped area of ​​code 2, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area with the same even number code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0, and a binary code about m amplitudes of the corresponding fan-shaped area is obtained in sequence; 16 binary codes of 16 fan-shaped areas are serially connected according to the codes as the feature vector after skip sequence amplitude binarization;

[0093] The sequential amplitude binarization feature vectors and the skip-sequence amplitude binarization feature vectors are stored as feature vectors for feature comparison of the corresponding feature points.

[0094] The original SIFT feature vector is a 128-dimensional vector. Using 8-bit fixed-point to implement each dimension also consumes a lot of storage space. The binarization method is divided into two parts: jump amplitude comparison binarization and sequential amplitude comparison binarization. Sequential amplitude comparison binarization: compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area. If the amplitude is greater, the feature vector element at that position is 1, and if it is less, it is 0. For example, S 1,1 >S 2,1 Vseq 1,1 =1. Binarization of jump sequence amplitude comparison: S n The magnitude of the direction angle in the fan-shaped area is related to S n+2 The amplitudes of the same direction angles in the fan-shaped area are compared, that is, the even sub-area and the odd sub-area compare the amplitudes of the same direction angles in the next even or odd sub-area respectively. If the amplitude is greater than, the eigenvector element at that position is 1, and if it is less than, it is 0. For example, S 1,1 >S 3,1 , then Vnseq 1,1 = 1. The method for binarizing feature vectors based on fan-shaped sub-regions proposed in the present invention greatly reduces the area consumption for storing feature vectors while ensuring matching stability.

[0095] In summary, the present invention is based on the SIFT feature point extraction algorithm. The part for generating feature vectors is a circular area, the circular area is divided into sub-areas, and the amplitudes in 8 directions are counted in each sub-area. The present invention avoids the problem of re-counting pixels in the square area after rotation, and is easier to implement using hardware. At the same time, in order to reduce the storage of feature vectors, binary abstraction of feature vectors is added later, which greatly reduces the hardware implementation area of ​​feature vectors.

[0096] A feature vector matching method is designed based on the digital image processing method that is easy to implement in hardware in this embodiment. The feature vector matching method extracts the feature vector of each feature point from multiple feature points in two different digital images for feature comparison, wherein the feature vector is extracted by the digital image processing method of the present invention, such as Fig.10 As shown, Fig.10 The binary feature vector matching result obtained in the final circular area. The red dots are the feature points of the image. The two connected red dots are the feature point pairs that are successfully matched after using the feature vector matching.

[0097] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A digital image processing method that is easy to implement with hardware, which is used to extract a feature vector for feature comparison from each feature point in the digital image, wherein the feature vector has m direction angles and m amplitudes corresponding to the m direction angles one by one; characterized in that: The digital image processing method comprises the following steps: In the digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer; for the circular area, 8 sector-shaped areas are divided every 45° from the center angle of 0°, and the storage information of the 8 sector-shaped areas is stored in 8 storage units coded by 1, 3, 5, 7, 9, 11, 13, and 15 in sequence; for the circular area, 8 sector-shaped areas are re-divided every 45° from the center angle of -22.5°, and the storage information of the re-divided 8 sector-shaped areas is stored in another 8 storage units coded by 2, 4, 6, 8, 10, 12, 14, and 16 in sequence; the mutual positions of the 16 stored information are the information storage positions; For the storage content of the same storage unit, m amplitudes S n,1 ~S n,m The amplitude storage position of S is encoded sequentially from 1 to m; n,m Represents the amplitude at the mth direction angle in the nth sector area, m amplitudes S n,1 ~S n,m That is the storage information of the nth sector area; Rotating the digital image so that the main direction angle of the circular area is 0°; The storage information on the 16 storage units sequentially coded from 1 to 16 is sequentially updated according to the rotation angle of the digital image, and the information storage position of the storage information of the fan-shaped area before the rotation is changed; The stored contents at the m signal positions encoded sequentially from 1 to m are also updated sequentially according to the rotation angle of the digital image, and the m amplitudes S of the stored information in the fan-shaped area before the rotation are changed. n,1 ~S n,m The amplitude storage location of The storage information sequentially output by the 16 storage units is used as the feature vector.

2. The digital image processing method that is easy to implement with hardware according to claim 1, characterized in that: The digital image processing method further comprises the following steps: The eigenvectors are subjected to sequential amplitude binarization processing: For the circular area, 8 sector-shaped areas are divided every 45° from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, 15 are sequentially encoded to define 8 corresponding odd sub-areas; For the circular area, starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and even-numbered codes of 2, 4, 6, 8, 10, 12, 14, and 16 are performed in sequence, thereby defining corresponding 8 even sub-areas; Starting from the sector area coded 1, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes to form a feature vector after amplitude binarization in sequence for feature comparison.

3. The digital image processing method that is easy to implement with hardware according to claim 1 or 2, characterized in that: The digital image processing method further comprises the following steps: Perform skip-order amplitude binarization processing on the feature vector: For the circular area, 8 sector-shaped areas are divided every 45° from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, 15 are sequentially encoded to define 8 corresponding odd sub-areas; For the circular area, starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and even-numbered codes of 2, 4, 6, 8, 10, 12, 14, and 16 are performed in sequence, thereby defining corresponding 8 even sub-areas; Starting from the fan-shaped area with code 1, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code of m amplitudes of the corresponding fan-shaped area is obtained in sequence; Starting from the fan-shaped area of ​​code 2, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area with the same even number code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code of m amplitudes of the corresponding fan-shaped area is obtained in sequence; The 16 binary codes of the 16 sector-shaped areas are serially concatenated according to the codes to form a jump sequence amplitude binarization feature vector which is also used for feature comparison.

4. The digital image processing method that is easy to implement with hardware according to claim 1, characterized in that: r＝9； and / or, m=8, the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° respectively; And / or, at the same direction angle, the amplitudes corresponding to the corresponding direction angles in all fan-shaped areas are accumulated, and the direction angle with the largest amplitude is selected as the main direction angle of the circular area.

5. A hardware circuit for changing a storage location, characterized in that: It is used to implement the information storage position transformation in the digital image processing method that is easy to implement in hardware according to any one of claims 1 to 4, and the hardware circuit includes 16 sector area information storage circuits corresponding to the 16 sector areas one by one, and each sector area information storage circuit includes: Adder 1, one of the addend input terminals of the adder 1 receives an addend: K1-1, K1 is the sector area number value corresponding to the sector area information storage circuit, and another addend input terminal of the adder 1 receives an addend: 2(L-1), L is the encoding value of the amplitude of the main direction angle in the m amplitude storage positions; A multiplexer 1, whose input is the storage information of the sector area corresponding to the sector area information storage circuit, and whose selection control end is the output of the adder 1, controls the output end of the multiplexer 1 to select the storage information of the corresponding sector area from the 16 sector areas.

6. The hardware circuit for changing storage positions according to claim 5, characterized in that: The hardware circuit also includes m amplitude storage circuits for realizing the change of amplitude storage position in the same sector area and corresponding to the m amplitudes one by one, each amplitude storage circuit including: Adder 2, one of the addend input terminals of the adder 2 receives an addend: K2-1, K2 is an amplitude coding value corresponding to the amplitude storage circuit, and another addend input terminal receives an augend: L-1; The input of the multiplexer 2 is the m amplitudes corresponding to the amplitude storage circuit, and the selection control terminal is the output of the adder 2, which controls the output terminal of the multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

7. A hardware circuit for changing a storage location, characterized in that: It is used to implement the amplitude storage position transformation in the digital image processing method that is easy to implement in hardware according to any one of claims 1 to 4, and the hardware circuit includes m amplitude storage circuits corresponding to m amplitudes one by one, and each amplitude storage circuit includes: Adder 2, one of the addend input terminals of the adder 2 receives an addend: K2-1, K2 is an amplitude coding value corresponding to the amplitude storage circuit, and another addend input terminal receives an augend: L-1; The input of the multiplexer 2 is the m amplitudes corresponding to the amplitude storage circuit, and the selection control terminal is the output of the adder 2, which controls the output terminal of the multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

8. A feature vector storage method for reducing the area consumption of storing feature vectors, characterized in that: It includes the following steps: In the digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer and satisfies; for the circular area, starting from the central angle of 0°, 8 sector areas are divided every 45°, and odd codes of 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially performed to define 8 corresponding odd sub-areas; starting from the central angle of -22.5°, 8 sector areas are re-divided every 45°, and even codes of 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially performed to define 8 corresponding even sub-areas; each sub-area of ​​the same feature point has corresponding m amplitudes at m direction angles; Perform sequential amplitude binarization processing: starting from the fan-shaped area of ​​code 1, compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area of ​​code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of m amplitudes of the corresponding fan-shaped area is obtained in sequence. The 16 binary codes of the 16 fan-shaped areas are serially connected according to the codes as the feature vector after sequential amplitude binarization; Perform a jump sequence amplitude binarization process: starting from the sector area with code 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next sector area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code of m amplitudes of the corresponding sector area is obtained in sequence; Starting from the sector area of ​​code 2, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next sector area with the same even number code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0, and a binary code of m amplitudes of the corresponding sector area is obtained in sequence; the 16 binary codes of the 16 sector areas are serially connected according to the codes as the feature vector after the jump sequence amplitude binarization; The sequential amplitude binarization feature vectors and the skip-sequence amplitude binarization feature vectors are stored as feature vectors for feature comparison of the corresponding feature points.

9. The feature vector storage method for reducing the area consumption of storing feature vectors according to claim 8, characterized in that: r＝9； And / or, m=8, the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° respectively.

10. A feature vector matching method, which extracts a feature vector of each feature point from multiple feature points in two different digital images for feature comparison; characterized in that: The feature vector matching method uses a digital image processing method that is easy to implement in hardware according to any one of claims 1 to 4 to extract the feature vector.

Images