Digital image processing method and hardware circuit, feature vector storage and matching method

Abstract

The present invention relates to a digital image processing method and hardware circuit, feature vector storage and matching method in the field of integrated circuit design. The digital image processing method comprises: in a digital image, taking each feature point as the center and r pixels as the radius to define a circular area; starting from the center angles of 0° and 22.5°, 8 fan-shaped areas are divided every 45°, and cross-coded in sequence; m amplitude storage locations of the storage content of the same storage unit are sequentially coded from 1 to m; rotating the digital image so that the main direction angle is 0°; changing the information storage location of the storage information of the fan-shaped area before rotation and the amplitude storage location of the corresponding m amplitudes. The present invention outlines 16 overlapping fan-shaped areas for each feature point on the basis of demarcating the circular area. Therefore, after rotating according to the main direction angle, the circular area does not need to be re-counted, and only the storage order needs to be changed, which simplifies the feature vector generation process and is conducive to hardware implementation.

Description

Technical Field

[0001] The present invention relates to a digital image processing method and hardware circuit, and a feature vector storage and matching method in the field of integrated circuit design, and in particular to a digital image processing method that is easy to implement by hardware, a hardware circuit for implementing information storage position transformation in the digital image processing method, another hardware circuit for implementing amplitude storage position transformation in the digital image processing method, a feature vector storage method for reducing area consumption for storing feature vectors, and a feature vector matching method using the feature vector storage method. Background Art

[0002] SIFT (Scale-Invariant Feature Transform) is an algorithm for image feature extraction, proposed by David Lowe in 1999 and improved in 2004. SIFT detects local feature points in an image and generates scale-invariant and rotation-invariant feature descriptors for them, enabling the algorithm to recognize the same object or scene under varying scales, rotations, and illumination. SIFT has been a major breakthrough in image matching, image stitching, and recognition, and has become a classic algorithm in computer vision and image processing.

[0003] SIFT can extract a feature vector for each feature point in a digital image, which is used for feature comparison. This feature vector generally consists of eight azimuths and eight amplitudes corresponding to each of the eight azimuths. This feature comparison allows identification of identical digital images and the presence of shared features. However, current digital image processing methods, in order to improve feature matching accuracy, are complex in hardware implementation. Summary of the Invention

[0004] Based on this, in order to solve the technical problem that existing digital image processing methods are difficult to implement in hardware, the present invention provides a digital image processing method that is easy to implement in hardware, a hardware circuit for implementing information storage position transformation in the digital image processing method, another hardware circuit for implementing amplitude storage position transformation in the digital image processing method, a feature vector storage method for reducing the area consumption of storing feature vectors, and a feature vector matching method using the feature vector storage method.

[0005] The objective of the present invention is achieved through the following technical solution: a digital image processing method that is easy to implement in hardware, which is used to extract a feature vector for feature comparison from each feature point in the digital image, wherein the feature vector has m direction angles and m amplitudes corresponding to the m direction angles respectively; the digital image processing method comprises the following steps:

[0006] In the digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer; starting from a central angle of 0°, the circular area is divided into 8 sector-shaped areas at intervals of 45°, and the storage information of the 8 sector-shaped areas is sequentially stored in 8 storage units coded 1, 3, 5, 7, 9, 11, 13, and 15; starting from a central angle of -22.5°, the circular area is re-divided into 8 sector-shaped areas at intervals of 45°, and the storage information of the re-divided 8 sector-shaped areas is sequentially stored in another 8 storage units coded 2, 4, 6, 8, 10, 12, 14, and 16; the relative positions of the 16 stored information are the information storage positions;

[0007] For the storage content of the same storage unit, m amplitudes S n,1 ~S n,m The amplitude storage position of S is sequentially encoded from 1 to m; n,m Indicates the amplitude at the mth direction angle in the nth sector area, m amplitudes S n,1 ~S n,m That is the storage information of the nth sector area;

[0008] Rotating the digital image so that the main direction angle of the circular area is 0°;

[0009] Sequentially updating the stored information on the 16 storage units sequentially coded from 1 to 16 according to the rotation angle of the digital image, thereby changing the information storage position of the stored information in the sector-shaped area before the rotation;

[0010] The stored contents at the m signal positions coded sequentially from 1 to m are also updated sequentially according to the rotation angle of the digital image, and the m amplitudes S of the stored information in the sector area before rotation are changed. n,1 ~S n,m The amplitude storage location;

[0011] The storage information sequentially output by the 16 storage units is used as the feature vector.

[0012] As a further improvement of the above solution, the digital image processing method further includes the following steps:

[0013] The eigenvectors are sequentially binarized:

[0014] For the circular area, 8 sector-shaped areas are divided every 45° starting from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially coded to define the corresponding 8 odd sub-areas;

[0015] For the circular area, starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and the even numbers 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially coded to define the corresponding 8 even sub-areas;

[0016] Starting from the sector area coded 1, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes to form a feature vector after amplitude binarization for feature comparison.

[0017] As a further improvement of the above solution, the digital image processing method further includes the following steps:

[0018] Perform jump sequence amplitude binarization processing on the eigenvector:

[0019] For the circular area, 8 sector-shaped areas are divided every 45° starting from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially coded to define the corresponding 8 odd sub-areas;

[0020] For the circular area, starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and the even numbers 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially coded to define the corresponding 8 even sub-areas;

[0021] Starting from the sector area with code 1, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next sector area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0. In this way, a binary code of the m amplitudes of the corresponding sector area is obtained in sequence.

[0022] Starting from the sector area with code 2, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next sector area with the same even code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0. In this way, a binary code of the m amplitudes of the corresponding sector area is obtained in sequence.

[0023] The 16 binary codes of the 16 sector-shaped areas are serially concatenated according to the codes to form a jump sequence amplitude binarization feature vector which is also used for feature comparison.

[0024] As a further improvement of the above solution, r=9.

[0025] As a further improvement of the above solution, m=8, and the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° respectively.

[0026] The present invention further provides a hardware circuit for changing storage locations, which is used to implement information storage location change in any of the above-mentioned digital image processing methods that are easy to implement in hardware. The hardware circuit includes 16 sector-shaped area information storage circuits corresponding one-to-one to the 16 sector-shaped areas, and each sector-shaped area information storage circuit includes:

[0027] Adder 1, wherein one addend input terminal of the adder 1 receives an addend: K1-1, where K1 is a sector area number value corresponding to the sector area information storage circuit, and another addend input terminal of the adder 1 receives an addend: 2(L-1), where L is a code value of the amplitude of the main direction angle in the m amplitude storage locations;

[0028] Multiplexer 1, whose input is the storage information of the sector area corresponding to the sector area information storage circuit, and whose selection control end is the output of adder 1, controls the output end of multiplexer 1 to select the storage information of the corresponding sector area from the 16 sector areas.

[0029] As a further improvement to the above solution, the hardware circuit further includes m amplitude storage circuits for realizing the change of amplitude storage position within the same sector area and corresponding one-to-one to the m amplitudes, each amplitude storage circuit including:

[0030] A second adder, wherein one addend input terminal of the second adder receives an addend: K2-1, K2 is an amplitude code value corresponding to the amplitude storage circuit, and another addend input terminal receives an augend: L-1;

[0031] Multiplexer 2, whose input is the m amplitudes corresponding to the amplitude storage circuit, and whose selection control terminal is the output of adder 2, controls the output terminal of multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

[0032] The present invention also provides another hardware circuit for changing storage positions, which is used to implement the amplitude storage position change in any of the above-mentioned digital image processing methods that are easy to implement in hardware. The hardware circuit includes m amplitude storage circuits corresponding to m amplitudes one by one, and each amplitude storage circuit includes:

[0033] A second adder, wherein one addend input terminal of the second adder receives an addend: K2-1, K2 is an amplitude code value corresponding to the amplitude storage circuit, and another addend input terminal receives an augend: L-1;

[0034] Multiplexer 2, whose input is the m amplitudes corresponding to the amplitude storage circuit, and whose selection control terminal is the output of adder 2, controls the output terminal of multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

[0035] The present invention also provides a feature vector storage method for reducing the area consumption of storing feature vectors, which comprises the following steps:

[0036] In a digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer and satisfies: for the circular area, eight sector-shaped areas are divided every 45° starting from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially coded to define eight corresponding odd sub-areas; eight sector-shaped areas are re-divided every 45° starting from the central angle of -22.5°, and even numbers 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially coded to define eight corresponding even sub-areas; within each sub-area of ​​the same feature point, there are m corresponding amplitudes at m direction angles;

[0037] Perform sequential amplitude binarization: starting from the sector area coded 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes as the feature vector after sequential amplitude binarization.

[0038] Perform skip sequence amplitude binarization processing: starting from the sector area of ​​code 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next sector area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code for the m amplitudes of the corresponding sector area is obtained in sequence; starting from the sector area of ​​code 2, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next sector area with the same even code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code for the m amplitudes of the corresponding sector area is obtained in sequence; the 16 binary codes of the 16 sector areas are serially concatenated according to the codes as the eigenvector after skip sequence amplitude binarization;

[0039] The sequential amplitude binarization feature vectors and the skip-sequence amplitude binarization feature vectors are stored as feature vectors for feature comparison of the corresponding feature points.

[0040] As a further improvement of the above solution, r=9.

[0041] As a further improvement of the above solution, m=8, and the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° respectively.

[0042] The present invention also provides a feature vector matching method, which extracts the feature vector of each feature point from multiple feature points in two different digital images for feature comparison; wherein, the feature vector matching method uses any of the above-mentioned digital image processing methods that are easy to implement in hardware to extract the feature vector.

[0043] Compared with the prior art, the digital image processing method of the present invention, which is easy to implement in hardware, outlines 16 overlapping sector-shaped areas for each feature point on the basis of demarcating a circular area. Therefore, after rotating according to the main direction angle, the circular area does not need to be re-counted as a square pixel area after the rotation of the diamond pixel area. It only needs to change the storage order. In combination with the 16 overlapping sector-shaped areas, the feature vector generation process can be greatly simplified, which is conducive to hardware implementation.

[0044] The hardware circuit of the present invention, based on "drawing out 16 overlapping sector-shaped areas on the basis of demarcating circular areas", can easily realize the specific hardware circuits of "transforming the information storage position between different sector-shaped areas" and "transforming the amplitude storage position within the same sector-shaped area". It is so simple that only a few adders and a few multiplexers are needed. Combined with a specific control method, it can be easily realized.

[0045] The present invention's feature vector storage method for reducing feature vector storage area consumption utilizes "delineating 16 overlapping sector-shaped regions based on demarcated circular regions" to perform sequential amplitude binarization and skipped amplitude binarization, thereby reducing feature vector storage area consumption. For example, each element of a feature vector is originally implemented in hardware as a 10-bit fixed-point number, and each feature vector contains 128 elements, requiring 1.25 kilobytes of storage. However, the present invention only contains 256 binary digits, requiring only 256 bits of storage for each feature vector, reducing storage consumption by 80%.

[0046] The feature vector matching method of the present invention is based on "outlining 16 overlapping fan-shaped areas on the basis of demarcating circular areas". By using the method of area overlap, the amplitude and angle of pixels in different divided areas are repeatedly counted, which can ensure the stability of feature vector matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flowchart of a digital image processing method provided by an embodiment of the present invention.

[0048] Figure 2 for Figure 1 Schematic diagram of the middle circular area divided into 8 odd sub-areas.

[0049] Figure 3 for Figure 1 Schematic diagram of the circular area in the middle divided into 8 even sub-areas.

[0050] Figure 4 for Figure 2 、 Figure 3 Schematic diagram of the 16 sub-regions composed of the odd and even sub-regions.

[0051] Figure 5 for Figure 3 Schematic diagram of the middle circular area after rotation according to the main direction angle. The figure uses the eight direction angle amplitudes of the fan-shaped area numbered 4 as an example for illustration.

[0052] Figure 6 for Figure 5 Hardware implementation diagram of the neutron region information storage location transformation.

[0053] Figure 7 for Figure 5 Hardware implementation diagram of the transformation of the amplitude storage position of all directional angles in each sub-region.

[0054] Figure 8 for Figure 5 An example of sequential amplitude binarization of the sector area numbered 4.

[0055] Figure 9 for Figure 5 An example diagram of performing jump sequence amplitude binarization on the sector area numbered 4.

[0056] Figure 10 for Figure 5 Medium circular area Figure 8 、 Figure 9 The matching effect diagram of the binary feature vector obtained after binarization. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] The digital image processing method provided in this embodiment is easy to implement in hardware and is used to extract a feature vector for feature comparison for each feature point in the digital image. The feature vector has m direction angles and m amplitudes corresponding to each of the m direction angles. In this embodiment, m is set to 8 according to the current standard, and the eight direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, respectively.

[0059] See also Figure 1 , which is a flow chart of a digital image processing method according to embodiment 1 of the present invention. The digital image processing method comprises the following steps: S1 to S6.

[0060] Step S1, in the digital image, with each feature point as the center and r pixels as the radius, a circular area is defined, where r is a positive integer and satisfies: for the circular area, starting from a central angle of 0°, 8 sector-shaped areas are divided every 45°, and the storage information of the 8 sector-shaped areas is sequentially stored in 8 storage units coded with 1, 3, 5, 7, 9, 11, 13, and 15; for the circular area, starting from a central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and the storage information of the re-divided 8 sector-shaped areas is sequentially stored in another 8 storage units coded with 2, 4, 6, 8, 10, 12, 14, and 16; the relative positions of the 16 stored information are the information storage positions.

[0061] In this example, see Figure 2 As shown, according to the current standard, r is set to 9, because the radius of 9 pixels can basically cover the effective pixel area centered on the feature point. Therefore, the value of r needs to satisfy the requirement that the covered pixel area is the effective pixel area centered on the feature point. The circular domain with a radius of 9 pixels is divided into odd sub-regions and even sub-regions. Starting from 0°, a sector area is divided every 45°. They are numbered 1, 3, 5, 7, 9, 11, 13, and 15, and these areas are called odd sub-regions. Please combine Figure 3 Starting from -22.5°, a sector is divided every 45 degrees, and they are numbered 16, 2, 4, 6, 8, 10, 12, and 14. These regions are called even subregions. The odd subregions and even subregions together constitute the characteristic domain of this article. The odd subregions and even subregions obtained in the end have an overlapping relationship, such as Figure 4 shown.

[0062] Step S2: m amplitude values ​​S of the storage content of the same storage unit n,1 ~S n,m The amplitude storage position of S is sequentially encoded from 1 to m; n,m Indicates the amplitude at the mth direction angle in the nth sector area, m amplitudes Sn,1 ~S n,m That is the storage information of the nth sector area.

[0063] That is to say, for each sector area of ​​the same feature point, the corresponding m amplitudes S at m direction angles are counted. n,1 ~S n,m , S n,m Indicates the amplitude at the mth direction angle in the nth sector area, m amplitudes S n,1 ~S n,m That is the storage information of the corresponding sector area, m amplitudes S n,1 ~S n,m The relative position of the amplitude is the storage position; for the storage content S of the same storage unit n,1 ~S n,m The m amplitude storage locations are also encoded sequentially from 1 to m.

[0064] like Figure 5 The sector (even sub-region) numbered 4 has amplitudes at eight angles. The eight angles, 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, are numbered 1, 2, 3, 4, 5, 6, 7, and 8, respectively.

[0065] Step S3: rotating the digital image so that the main direction angle of the circular area is 0°.

[0066] At the same direction angle, the amplitudes corresponding to the corresponding direction angles in all fan-shaped areas are accumulated, the direction angle with the largest amplitude is selected as the main direction angle of the circular area, and the digital image is rotated so that the main direction angle is 0°.

[0067] Step S4, sequentially updating the storage information on the 16 storage units coded from 1 to 16 according to the rotation angle of the digital image, and changing the information storage position of the storage information in the fan-shaped area before rotation.

[0068] Step S5, the stored contents at the m signal positions coded sequentially from 1 to m are also updated sequentially according to the rotation angle of the digital image, and the m amplitudes S of the stored information in the sector area before rotation are changed. n,1 ~S n,m The amplitude storage location.

[0069] from Figure 5 In the example of the original sector numbered 4, we can see that the information stored in S4 is stored in the sector numbered 16, thus changing the information storage location. The eight amplitudes in the original sector numbered 4 also have their amplitude storage locations changed.

[0070] When the main direction angle is 90°, the information stored in sector 4 is moved to the storage location of sector 16. The storage of the other sectors changes accordingly: 1->13, 2->14, 3->15, 5->1, 6->2, 7->3, 8->4, 9->5, 10->6, 11->7, 12->8, 13->9, 14->10, 15->11, and 16->12. At the same time, the stored direction angle amplitudes also change their storage locations accordingly. For example, if the direction angle with the longest amplitude in sector 4 was originally 90°, after rotation, its amplitude information will be stored in the amplitude location corresponding to the 0° angle in sector 16.

[0071] Step S6: taking the storage information sequentially output by the 16 storage units as the feature vector.

[0072] In the above steps of the digital image processing method, the information storage location conversion in step S4 and the amplitude value storage location conversion in step S5 can be implemented using the following hardware circuit for converting storage locations. The hardware circuit may include 16 sector-shaped region information storage circuits corresponding one-to-one to the 16 sector-shaped regions, and may also include m amplitude storage circuits corresponding one-to-one to the m amplitude values.

[0073] Each sector area information storage circuit includes: adder 1 and multiplexer 1. Each amplitude storage circuit includes: adder 2 and multiplexer 2.

[0074] One addend input of adder 1 receives an addend: K1-1, where K1 is the sector area number corresponding to the sector area information storage circuit. Another addend input of adder 1 receives a summand: 2(L-1), where L is the encoded value of the amplitude of the main direction angle in the m amplitude storage locations. Multiplexer 1 receives the stored information of the sector area corresponding to the sector area information storage circuit as input, and its selection control terminal is the output of adder 1, controlling the output of multiplexer 1 to select the stored information of the corresponding sector area from among the 16 sectors.

[0075] In this embodiment, the hardware implementation of the regional storage information rotation is shown in FIG. Figure 6During implementation, the detected main direction is divided into 8 bins, that is, the main direction has only 8 values: 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, corresponding to main_ori_bin values ​​0, 1, 2, 3, 4, 5, 6, and 7, respectively. The azimuth angle amplitude information stored in the fan-shaped sub-region corresponds to S[0:15]. The copied information of all sub-regions is broadcast to the input of 16 MUXs (i.e., multiplexer 1). The MUX selection signal is obtained by adding twice the main_ori_bin value to the sub-region number. After the MUX selection, the vector information S_aft_rotate stored in each sub-region after rotation is obtained.

[0076] One of the addend inputs of adder 2 receives an addend, K2-1, where K2 is the amplitude code value corresponding to the amplitude storage circuit. The other addend input receives a summand, L-1. Multiplexer 2 receives the m amplitudes corresponding to the amplitude storage circuit as inputs, and its selection control terminal is the output of adder 2, controlling the output of multiplexer 2 to select and output a corresponding amplitude from the m amplitudes.

[0077] After obtaining S_aft_rotate, you also need to rotate all the angular magnitudes in the sub-region, see Figure 7 Taking region 16 as an example, the eight azimuth angle values ​​stored in region 16 correspond to S16_aft_rotate[0:7]. S16_aft_rotate[0:7] is broadcast to the inputs of eight MUXs (i.e., multiplexer 2). The MUX selection signal is the azimuth angle number minus the value of main_ori_bin. After the MUX selection, the final storage vector S16_final[0:7] is obtained. The storage vectors for other subregions can be obtained in the same way.

[0078] To facilitate storage of digital image processing data, the digital image processing method may further include the following steps: performing sequential amplitude binarization and skip-sequence amplitude binarization on the feature vector. In other words, the sequential amplitude binarization and skip-sequence amplitude binarization can be added to the digital image processing method of this embodiment, which is easy to implement in hardware.

[0079] Sequential amplitude binarization processing: starting from the sector area coded 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes to form a feature vector after sequential amplitude binarization for feature comparison.

[0080] In this embodiment, for the outer sector area S, each area is represented by the subscript n, and the directional angle amplitude in each area is represented by the subscript m. Then S n,m Represents the magnitude of the mth direction angle in the sector area n.

[0081] See Figure 8 , is the sequential amplitude binarization abstraction method of the present invention, which compares the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area. If the amplitude is greater than, the eigenvector element at that position is 1, and if it is less than, it is 0. Figure 6 In the fan-shaped area 1, the 0° angle corresponds to the amplitude S 1,1 Smaller than the corresponding amplitude S of 0° in sector 2 2,1 , then Vseq 1,1 = 1. According to this method, Vseq can be obtained in sequence 1,2 =1, Vseq 1,3 =1, Vseq 1,4 =0, Vseq 1,5 =1, Vseq 1,6 =1, Vseq 1,7 =0, Vseq 1,8 =0.

[0082] Binarization processing of jump sequence amplitude:

[0083] (1) Starting from the sector area with code 1, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next sector area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0. In this way, a binary code of m amplitudes of the corresponding sector area is obtained in sequence.

[0084] (2) Starting from the sector area with code 2, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next sector area with the same even code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0. In this way, a binary code of the m amplitudes of the corresponding sector area is obtained in sequence.

[0085] (3) The 16 binary codes of the 16 sector areas are serially concatenated according to the codes to form a jump sequence amplitude binarization feature vector which is also used for feature comparison.

[0086] See Figure 9 The jump sequence amplitude binarization abstraction method of the present invention compares the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next sector area of ​​the same odd or even sub-domain. If the amplitude is greater, the eigenvector element at that position is 1, and if it is less, it is 0. As shown in the figure, the 0° angle in sector area 1 corresponds to the amplitude S 1,1Greater than the corresponding amplitude S of 0° in sector 3 3,1 , then Vuseq 1,1 = 1. According to this method, Vuseq can be obtained in sequence 1,2 =1, Vuseq 1,3 =1, Vuseq 1,4 =0, Vuseq 1,5 =1, Vuseq 1,6 =1, Vuseq 1,7 =0, Vuseq 1,8 =0.

[0087] Originally, each element of the feature vector was implemented in hardware as a 10-bit fixed-point number. Each feature vector contained 128 elements, requiring 1.25 KB of storage. However, after binarization, the feature vector only contained 256 binary digits, requiring only 256 bits of storage per feature vector. This reduced storage consumption by 80%.

[0088] In combination with the digital image processing method that is easy to implement in hardware in this embodiment, a feature vector storage method for reducing the area consumption of storing feature vectors is also demonstrated by adding sequential amplitude binarization processing and skipped amplitude binarization processing. The feature vector storage method includes the following steps:

[0089] In the digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer and satisfies the following conditions: starting from the central angle of 0°, the circular area is divided into 8 sector-shaped areas every 45°, and odd numbers 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially applied to define the corresponding 8 odd sub-areas; starting from the central angle of -22.5°, the circular area is divided into 8 sector-shaped areas every 45°, and even numbers 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially applied to define the corresponding 8 even sub-areas;

[0090] For each sub-region of the same feature point, the corresponding m amplitudes S at m direction angles are counted n,1 ~S n,m , S n,m Represents the amplitude at the mth direction angle in the nth sub-area;

[0091] Perform sequential amplitude binarization: starting from the sector area coded 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes as the feature vector after sequential amplitude binarization.

[0092] Perform skip sequence amplitude binarization processing: starting from the sector area of ​​code 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next sector area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code for the m amplitudes of the corresponding sector area is obtained in sequence; starting from the sector area of ​​code 2, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next sector area with the same even code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0, and a binary code for the m amplitudes of the corresponding sector area is obtained in sequence; the 16 binary codes of the 16 sector areas are serially concatenated according to the codes as the eigenvector after skip sequence amplitude binarization;

[0093] The sequential amplitude binarization feature vectors and the skip-sequence amplitude binarization feature vectors are stored as feature vectors for feature comparison of the corresponding feature points.

[0094] The original SIFT feature vector is a 128-dimensional vector. Using 8-bit fixed-point to implement each dimension also consumes a lot of storage space. The binarization method is divided into two parts: jump amplitude comparison binarization and sequential amplitude comparison binarization. Sequential amplitude comparison binarization: compare the amplitude of the direction angle in the fan-shaped area with the amplitude of the same direction angle in the next fan-shaped area. If the amplitude is greater, the feature vector element at that position is 1, and if it is less, it is 0. For example, S 1,1 >S 2,1 Then Vseq 1,1 = 1. Binarization of jump sequence amplitude comparison: S n The magnitude of the direction angle in the fan-shaped area is related to S n+2 The amplitudes of the same direction angles in the fan-shaped area are compared, that is, the even sub-area and odd sub-area respectively compare the amplitudes of the same direction angles in the next even or odd sub-area. If the amplitude is greater than the amplitude, the eigenvector element at that position is 1, and if it is less than the amplitude, it is 0. For example, S 1,1 >S 3,1 , then Vnseq 1,1 = 1. The method of binarizing feature vectors based on sector-shaped sub-regions proposed in the present invention greatly reduces the area consumption of storing feature vectors while ensuring matching stability.

[0095] In summary, the present invention is based on the SIFT feature point extraction algorithm. In the circular area where the feature vector is generated, the circular area is divided into sub-areas, and the amplitudes in eight directions are counted in each sub-area. The present invention avoids the problem of re-counting the pixels in the square area after rotation, and is easier to implement using hardware. At the same time, in order to reduce the storage of feature vectors, binary abstraction of the feature vectors is added later, which greatly reduces the hardware implementation area of ​​the feature vectors.

[0096] A feature vector matching method is designed based on the digital image processing method that is easy to implement in hardware in this embodiment. The feature vector matching method extracts the feature vector of each feature point from multiple feature points in two different digital images and performs feature comparison. The feature vector is extracted using the digital image processing method of the present invention, such as Figure 10 As shown, Figure 10 The final binary feature vector matching result is obtained in the circular area. The red dots are the feature points of the image. The two connected red dots are the feature point pairs that are successfully matched after feature vector matching.

[0097] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A digital image processing method that is easily implemented by hardware, for extracting a feature vector for feature comparison from each feature point in the digital image, wherein the feature vector has m direction angles and m amplitudes corresponding to the m direction angles; characterized in that: The digital image processing method comprises the following steps: In the digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer; starting from a central angle of 0°, the circular area is divided into 8 sector-shaped areas at intervals of 45°, and the storage information of the 8 sector-shaped areas is sequentially stored in 8 storage units coded 1, 3, 5, 7, 9, 11, 13, and 15; starting from a central angle of -22.5°, the circular area is re-divided into 8 sector-shaped areas at intervals of 45°, and the storage information of the re-divided 8 sector-shaped areas is sequentially stored in another 8 storage units coded 2, 4, 6, 8, 10, 12, 14, and 16; the relative positions of the 16 stored information are the information storage positions; For the storage content of the same storage unit, m amplitudes S n,1 ~S n,m The amplitude storage position of S is sequentially encoded from 1 to m; n,m Indicates the amplitude at the mth direction angle in the nth sector area, m amplitudes S n,1 ~S n,m That is the storage information of the nth sector area; Rotating the digital image so that the main direction angle of the circular area is 0°; Sequentially updating the stored information on the 16 storage units sequentially coded from 1 to 16 according to the rotation angle of the digital image, thereby changing the information storage position of the stored information in the sector-shaped area before the rotation; The stored contents at the m signal positions coded sequentially from 1 to m are also updated sequentially according to the rotation angle of the digital image, and the m amplitudes S of the stored information in the sector area before rotation are changed. n,1 ~S n,m The amplitude storage location; The storage information sequentially output by the 16 storage units is used as the feature vector.

2. The digital image processing method that is easy to implement by hardware according to claim 1, characterized in that: The digital image processing method further comprises the following steps: The eigenvectors are sequentially binarized: For the circular area, 8 sector-shaped areas are divided every 45° starting from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially coded to define the corresponding 8 odd sub-areas; For the circular area, starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and the even numbers 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially coded to define the corresponding 8 even sub-areas; Starting from the sector area coded 1, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes to form a feature vector after amplitude binarization for feature comparison.

3. The digital image processing method that is easy to implement by hardware according to claim 1 or 2, characterized in that: The digital image processing method further comprises the following steps: Perform jump sequence amplitude binarization processing on the eigenvector: For the circular area, 8 sector-shaped areas are divided every 45° starting from the central angle of 0°, and odd numbers 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially coded to define the corresponding 8 odd sub-areas; For the circular area, starting from the central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and the even numbers 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially coded to define the corresponding 8 even sub-areas; Starting from the sector area with code 1, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next sector area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0. In this way, a binary code of the m amplitudes of the corresponding sector area is obtained in sequence. Starting from the sector area with code 2, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next sector area with the same even code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0. In this way, a binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector-shaped areas are serially concatenated according to the codes to form a jump sequence amplitude binarization feature vector which is also used for feature comparison.

4. The digital image processing method that is easy to implement by hardware according to claim 1, characterized in that: r＝9； and / or, m=8, the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°; And / or, at the same direction angle, the amplitudes corresponding to the corresponding direction angles in all sector-shaped areas are accumulated, and the direction angle with the largest amplitude is selected as the main direction angle of the circular area.

5. A hardware circuit for changing a storage location, characterized in that: It is used to implement the information storage location transformation in the digital image processing method that is easy to implement in hardware according to any one of claims 1 to 4, wherein the hardware circuit includes 16 sector-shaped area information storage circuits corresponding to the 16 sector-shaped areas, and each sector-shaped area information storage circuit includes: Adder 1, wherein one addend input terminal of the adder 1 receives an addend: K1-1, where K1 is a sector area number value corresponding to the sector area information storage circuit, and another addend input terminal of the adder 1 receives an addend: 2(L-1), where L is a code value of the amplitude of the main direction angle in the m amplitude storage locations; Multiplexer 1, whose input is the storage information of the sector area corresponding to the sector area information storage circuit, and whose selection control end is the output of adder 1, controls the output end of multiplexer 1 to select the storage information of the corresponding sector area from the 16 sector areas.

6. The hardware circuit for changing storage locations according to claim 5, characterized in that: The hardware circuit further includes m amplitude storage circuits for realizing the change of amplitude storage position within the same sector area and corresponding to the m amplitudes one by one, each amplitude storage circuit including: A second adder, wherein one addend input terminal of the second adder receives an addend: K2-1, K2 is an amplitude code value corresponding to the amplitude storage circuit, and another addend input terminal receives an augend: L-1; Multiplexer 2, whose input is the m amplitudes corresponding to the amplitude storage circuit, and whose selection control terminal is the output of adder 2, controls the output terminal of multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

7. A hardware circuit for changing a storage location, characterized in that: It is used to implement the amplitude storage position transformation in the digital image processing method that is easy to implement in hardware according to any one of claims 1 to 4, and the hardware circuit includes m amplitude storage circuits corresponding to m amplitudes one by one, and each amplitude storage circuit includes: A second adder, wherein one addend input terminal of the second adder receives an addend: K2-1, K2 is an amplitude code value corresponding to the amplitude storage circuit, and another addend input terminal receives an augend: L-1; Multiplexer 2, whose input is the m amplitudes corresponding to the amplitude storage circuit, and whose selection control terminal is the output of adder 2, controls the output terminal of multiplexer 2 to select and output the corresponding amplitude from the corresponding m amplitudes.

8. A feature vector storage method for reducing the area consumption of storing feature vectors, characterized in that: It includes the following steps: In a digital image, a circular area is defined with each feature point as the center and r pixels as the radius, where r is a positive integer and satisfies the following conditions: starting from a central angle of 0°, the circular area is divided into 8 sector-shaped areas every 45°, and odd numbers 1, 3, 5, 7, 9, 11, 13, and 15 are sequentially coded to define the corresponding 8 odd sub-areas; starting from a central angle of -22.5°, 8 sector-shaped areas are re-divided every 45°, and even numbers 2, 4, 6, 8, 10, 12, 14, and 16 are sequentially coded to define the corresponding 8 even sub-areas; each sub-area of ​​the same feature point has m corresponding amplitudes at m direction angles; Perform sequential amplitude binarization: starting from the sector area coded 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next coded sector area. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitudes of the corresponding sector area is obtained in sequence. The 16 binary codes of the 16 sector areas are concatenated in sequence according to the codes as the feature vector after sequential amplitude binarization. Perform a jump sequence amplitude binarization process: starting from the sector area with code 1, compare the amplitude of the direction angle in the sector area with the amplitude of the same direction angle in the next sector area with the same odd code. If the amplitude difference is greater than zero, the eigenvector element at the corresponding position is 1, otherwise it is 0. In this way, a binary code of the m amplitudes of the corresponding sector area is obtained in sequence. Starting from the sector area with code 2, the amplitude of the direction angle in the sector area is compared with the amplitude of the same direction angle in the next sector area with the same even code. If the amplitude difference is greater than zero, the feature vector element at the corresponding position is 1, otherwise it is 0. A binary code of the m amplitude values ​​of the corresponding sector area is obtained in sequence; the 16 binary codes of the 16 sector areas are concatenated in sequence according to the code as the feature vector after the jump sequence amplitude binarization; The sequential amplitude binarization feature vectors and the skip-sequence amplitude binarization feature vectors are stored as feature vectors for feature comparison of the corresponding feature points.

9. The feature vector storage method for reducing the area consumption of storing feature vectors according to claim 8, characterized in that: r＝9； And / or, m=8, the 8 direction angles are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° respectively.

10. A feature vector matching method, which extracts the feature vector of each feature point from multiple feature points in two different digital images and performs feature comparison; characterized in that: The feature vector matching method uses the digital image processing method according to any one of claims 1 to 4, which is easy to implement in hardware, to extract the feature vector.

Images