A hardware circuit and method for principal component analysis algorithm based on singular value decomposition

Abstract

The present invention seeks to protect a hardware circuit and method for a principal component analysis algorithm based on singular value decomposition, which belongs to the hardware implementation technology of unsupervised learning dimensionality reduction algorithms. The circuit includes the following circuit modules: a control module, a Hestense module, a CORDIC module, and a Jacobi rotation module. The control module is used to control the timing and working status of each module; the Hestense module is used to calculate the parameters required by the CORDIC module and is constructed by a multiplier, an adder, a register, and a RAM; the CORDIC module is used to calculate inverse trigonometric functions and is constructed by a multiplexer, a register, and a shifter; and the Jacobi rotation module is used to calculate the orthogonalization result of the input data and is constructed by four multipliers, an adder, and a subtractor.

Description

Technical Field

[0001] The present invention belongs to the field of digital integrated circuit design, and in particular relates to the design of a digital hardware circuit for an unsupervised learning dimensionality reduction algorithm, namely, a hardware circuit for a principal component analysis algorithm based on singular value decomposition. Background Art

[0002] Data dimensionality reduction is the process of selecting a low-dimensional feature set from an initial high-dimensional feature set in order to optimally reduce the feature space based on certain evaluation criteria. It is often used as a preprocessing step in machine learning. In recent years, the number of data instances and features has increased dramatically in many applications. This massive amount of data has caused serious problems with the scalability and learning performance of machine learning algorithms. High-dimensional data sets, with hundreds or even thousands of features, contain a large amount of irrelevant and redundant information, which significantly degrades the performance of learning algorithms. Therefore, dimensionality reduction is crucial when dealing with high-dimensional data. Dimensionality reduction algorithms can be categorized as supervised or unsupervised, depending on whether they use sample labels. Supervised dimensionality reduction algorithms use sample labels, a typical example being linear discriminant analysis (LDA). Unsupervised dimensionality reduction algorithms do not use sample labels, a typical example being principal components analysis (PCA).

[0003] Principal components analysis (PCA) is a dimensionality reduction algorithm that converts numerous linearly correlated indicators into a few linearly independent comprehensive indicators through vector transformation, thereby cutting off related interference, pointing out the dominant components, and making more accurate estimates.

[0004] The core of the principal component analysis algorithm lies in finding the eigenvalues and eigenvectors of a data matrix. Current principal component analysis algorithms primarily use eigenvalue decomposition (Eigenvalue Decomposition) to implement the algorithm. This method is computationally intensive, resulting in significant hardware implementation overhead and slow computation speed. Using singular value decomposition (Singular Value Decomposition) can save hardware area and computation time.

[0005] The principal component analysis algorithm can be implemented in both software and hardware. Hardware implementations offer high speed and security. Current hardware implementations of the singular value decomposition (SVD)-based principal component analysis algorithm primarily rely on existing IP cores for SVD, which reduces the module's flexibility and portability and potentially wastes hardware resources. Summary of the Invention

[0006] The present invention aims to solve the above problems of the prior art. A hardware circuit of a principal component analysis algorithm based on singular value decomposition is proposed. The technical solution of the present invention is as follows:

[0007] A hardware circuit for a principal component analysis algorithm based on singular value decomposition includes a control module, a Henstense module, a CORDIC module, a Jacobi rotation module, and a multiplier module, wherein:

[0008] The control module is used to control the timing, data flow and working status of each module;

[0009] The Hestense module is used to calculate the parameters ERR and YX required by the CORDIC module;

[0010] The CORDIC module is used to calculate inverse trigonometric functions. It is composed of a multiplexer, registers, and shift registers. The CORDIC module receives the values of YX and ERR calculated by the Hestense module. The values enter the first CORDIC module in vector mode and calculate the corresponding A i and A j The rotation angle is input to the second CORDIC module in rotation mode, and the sine value sinθ and cosine value cosθ are output;

[0011] The Jacobi rotation module is used to calculate the orthogonalization result of the input data. It consists of four multipliers, an adder and a subtractor. The four multipliers perform parallel operations and have four input ports to calculate A i 、A j The product of their corresponding sine and cosine values is output to the adder and subtractor to obtain the corresponding orthogonalized result C i and C j .

[0012] Furthermore, the Hestense module is composed of a first multiplier, an accumulator, a register, a shifter, a first dual-port RAM, and a second dual-port RAM (for distinction). The input end of the first dual-port RAM is connected to the output end of the multiplexer, the output end of the first dual-port RAM is connected to the input end of the first multiplier, the output end of the first multiplier is connected to the accumulator, and the accumulator is connected to the adder through the second dual-port RAM. When the data is taken out from the memory in vector format and sequentially input to the input end of the Hestense module, it is recorded as A. i , at this time A j The input terminal inputs each row vector in the array in sequence, and i First, the two vectors are multiplied, that is, the elements in the two vectors are multiplied and added; second, when Ai When performing multiplication and accumulation operations with itself, the output is X, A j The output of the multiplication and accumulation operation with itself is Y, A i With A j The output of the multiplication and accumulation operation is ERR. When ERR is less than the preset value, it indicates that the orthogonalization operation is completed.

[0013] Furthermore, the Jacobi rotation module realizes the calculation of eigenvalues in the principal component analysis algorithm through the calculation method of unilateral Jacobi rotation.

[0014] Furthermore, the Jacobi rotation module implements the calculation of eigenvalues in the principal component analysis algorithm by a calculation method of unilateral Jacobi rotation, which specifically includes the following steps:

[0015] Perform rotation calculation on any matrix. Each round of iteration makes any two columns of the data set orthogonal once until the output value is orthogonal.

[0016] A=USV' (0.1)

[0017] Then there is

[0018] AV=US (0.2)

[0019] Where A is the matrix of the input data set; S is the singular value corresponding to the input matrix; U is the left singular vector; V is the right singular vector. Observe that the right side of the equation is an orthogonal matrix. Therefore, the core of the Jacobi rotation is to transform the matrix A into an orthogonal matrix through a series of linear transformations.

[0020]

[0021] Represents the orthogonalized vector obtained after calculation; represents the vector of the i or jth row of the input, θ represents the rotation angle corresponding to the orthogonalized vector, and we can get

[0022]

[0023]

[0024] Calculate sine and cosine values in CORDIC rotation mode

[0025]

[0026] x0, y0 represent the two initial vectors of input, x n 、y n Represents cosine and sine values, z n Indicates the angle after superposition (set to 0 in rotation mode), K nrepresents the scaling factor, z0 represents the rotation angle, n represents the number of rows or columns of the matrix, and i represents the number of iterations, and thus we can get

[0027]

[0028] The input data is converted into matrix calculations using dual-port RAM, and the matrix calculations use a one-sided Jacobian rotation. Rounding the eigenvalues in each iteration ensures the correctness of the calculation, and a non-restoring square root algorithm is used to ensure accuracy. After the input data is converted into matrix operations, the matrix is transposed through the Hestense module and expanded to allow matrix calculations of any size and shape. When the orthogonalization error set in Hestense reaches the predefined convergence value, it indicates that all rows in the matrix generated by the data set are orthogonal.

[0029] A hardware circuit-based implementation method comprises the following steps:

[0030] (1) The control module controls the input data parameters to be sent to the Hestense module for calculation;

[0031] (2) The Hestense module performs multiplication and accumulation operations on the input parameters;

[0032] (3) The CORDIC module calculates the corresponding sine and cosine values based on the output of the Hestense module;

[0033] (4) The Jacobi rotation module calculates the orthogonalization results corresponding to the system input parameters based on the sine and cosine values calculated by the CORDIC module, and feeds them back to the Hestense module for calculation.

[0034] The advantages and beneficial effects of the present invention are as follows:

[0035] 1. The present invention provides a hardware circuit for a principal component analysis algorithm based on singular values. By using the Verilog hardware description language to build an RTL-level model of the principal component analysis algorithm hardware circuit, a full hardware implementation architecture is provided.

[0036] 2. The hardware implementation method of the principal component analysis algorithm based on singular values of the present invention is based on the traditional principal component analysis algorithm. It adopts a unilateral Jacobi rotation calculation module composed of a Hestense circuit module, a CORDIC circuit module and a Jacobi rotation circuit module to perform a singular value decomposition operation on the matrix to obtain the eigenvalue. By applying the CORDIC module to implement the hardware structure of the unilateral Jacobi rotation algorithm, an approximation method of mathematical calculation is provided to solve the singular value decomposition, and the singular value decomposition calculation is divided into a series of addition, subtraction and shift operations. Compared with the traditional implementation method, the speed can be increased by 15% and the area can be reduced by 20%.

[0037] 3. The hardware circuit of the principal component analysis algorithm based on singular value decomposition of the present invention can input data of any dimension and size through the combination of the first dual-port RAM and the Hestense module, thereby reducing the clock cycle consumption of data transmission and calculation and improving the performance of the algorithm hardware execution.

[0038] 4. The hardware circuit of the principal component analysis algorithm based on singular value decomposition of the present invention mainly realizes the solution of singular values and eigenvalues in the principal component analysis algorithm by using unilateral Jacobi rotation through a combination of Hestense circuit module, CORDIC circuit module and Jacobi rotation circuit module. Compared with the hardware structure of the traditional principal component analysis algorithm, the number of multipliers and adders in the hardware structure is reduced, and the use of dividers is avoided, so that the calculation of the algorithm is simplified, so the calculation speed is faster and the hardware resources consumed are less.

[0039] 5. The hardware circuit of the principal component analysis algorithm based on singular value decomposition of the present invention uses a Hestense circuit module composed of a dual-port RAM, a multiplier and an accumulator, and only uses two data input ports, so that data can be reused, thereby optimizing the calculation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a hardware structure diagram of a hardware system according to a preferred embodiment of the present invention;

[0041] Figure 2 It is a hardware structure diagram of the Hestense module of the present invention;

[0042] Figure 3 It is a hardware structure diagram of the CORDIC module of the present invention;

[0043] Figure 4 It is a hardware structure diagram of the Jacobi rotation module of the present invention. DETAILED DESCRIPTION

[0044] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0045] The technical solution of the present invention to solve the above technical problems is:

[0046] A hardware implementation system for a principal component analysis algorithm based on singular value decomposition includes a control module, a storage module, a Hestense module, a CORDIC module, a Jacobi rotation module, and a multiplier module; the control module is connected to the storage module, the Hestense module, and the Jacobi rotation module, the storage module is connected to the Hestense module, the Hestense module is connected to the CORDIC module, and the CORDIC module is connected to the Jacobi rotation module;

[0047] Furthermore, the control module provides synchronization and control signals so that each module performs different tasks;

[0048] Furthermore, the storage module uses a dual-port RAM to manage and store input data sets, allowing them to be accessed and operated in the form of vectors;

[0049] Furthermore, the Hestense module includes a multiplier, an accumulator, a dual-port RAM, and an adder. It is responsible for multiplying and accumulating the input data, calculating the parameters required by the CORDIC module, and judging the feedback value of the Jacobi rotation module.

[0050] Furthermore, the CORDIC module includes a multiplier, an adder, a register, and a shifter, which uses the parameters calculated by the Hestense module to calculate the corresponding sine and cosine values for the Jacobi rotation module to perform Jacobi rotation operations;

[0051] Furthermore, the Jacobi rotation module includes four multipliers, one adder, and one subtractor. The four multipliers perform operations simultaneously and output the results to the adder or subtractor to calculate the orthogonalization results corresponding to the input parameters.

[0052] Furthermore, the orthogonalization result is judged for error in the Hestense module, and the calculated β value is compared with the defined convergence error. When the β value is less than the error, the result is output to the multiplier and multiplied with the data set to obtain the principal component value. Otherwise, the operation continues.

[0053] Figure 1 The algorithm architecture of the principal component analysis algorithm is demonstrated. The algorithm consists of four parts: Hestense calculation, CORDIC calculation, Jacobi rotation calculation, and multiplication calculation. The calculation results of the former serve as the calculation basis of the latter.

[0054] The key parameters of the algorithm include α and β output by the Hestense module, and cosθ and sinθ output by the CORDIC module. The value of β determines the number of operations of the algorithm.

[0055] Currently, there is no full hardware implementation architecture for the principal component analysis algorithm. This invention aims to provide a full hardware implementation solution for the principal component analysis algorithm, thereby improving the implementation, application and industrialization of the principal component analysis algorithm at the chip and FPGA levels.

[0056] Figure 2 The Hestense module calculates the corresponding α and β values for the input parameters while evaluating the value fed back by the Jacobi rotation module. If the β calculated from the feedback value is less than the preset convergence error, it is directly output. Otherwise, the calculation continues until the calculated β value is less than the preset value. This module operates by converting the calculation of the data set into multiplication operations between vectors.

[0057] Figure 3 The CORDIC module shown is composed of two cascaded CORDIC modules with different functions. The first CORDIC module is in vector mode and is used to calculate the rotation angle of each Jacobi rotation calculation; the second CORDIC module is in rotation mode and is used to simultaneously calculate the sine and cosine values of the corresponding rotation angle.

[0058] Figure 4 The Jacobi rotation module uses the unilateral Jacobi method to perform unilateral orthogonal rotations on a dataset, and can be used to rotate any matrix. Each iteration orthogonalizes any two columns of the dataset until the output is orthogonal.

[0059] A=USV' (0.8)

[0060] Then there is

[0061] AV=US (0.9)

[0062] We observe that the right side of the equation is an orthogonal matrix. Therefore, converting the matrix A into an orthogonal matrix through a series of linear transformations is the core of the Jacobi rotation.

[0063]

[0064] Available

[0065]

[0066]

[0067] Calculate sine and cosine values in CORDIC rotation mode

[0068]

[0069] Then we can get

[0070]

[0071] Input data is converted to matrix calculations using dual-port RAM. Single-sided Jacobian rotations are used for matrix calculations. Eigenvalues are rounded off in each iteration to ensure correctness, and a non-restoring square root algorithm is used to ensure accuracy. After the input data is converted to a matrix, the Hestense module performs a matrix transposition, expanding the algorithm to support matrix calculations of any size and shape. When the orthogonalization error set in the Hestense module reaches a predefined convergence value, all rows in the matrix generated from the dataset are orthogonal. Because any two row vectors in a matrix must be orthogonal for the algorithm to converge, a total of N(N-1) / 2 rotations of A are required. The CORDIC module calculates the sine and cosine values of two row vectors in the input matrix and transmits these values to the Jacobian rotation module. A new vector is generated through plane rotations, and the row vectors in the original matrix are updated until all row vectors are orthogonal. The updated row vectors are then filtered and multiplied by the original matrix to obtain the reduced-dimensional dataset.

[0072] This algorithm is primarily used for: 1. Resolving collinearity issues between variables in multiple linear regression; 2. Reducing the number of variables (i.e., dimensionality reduction) in factor analysis, cluster analysis, and discriminant analysis; and 3. As a basis for determining variable weights in comprehensive evaluation problems. Specifically, it can be used to eliminate irrelevant and redundant data and correlations between variables when processing large datasets. Singular value decomposition is implemented using the one-sided Jacobi algorithm, which is optimized using the CORDIC module.

[0073] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0074] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0075] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0076] The above embodiments should be understood as merely illustrating the present invention and not as limiting the scope of protection of the present invention. After reading the contents of the present invention, technicians may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A hardware circuit for principal component analysis algorithm based on singular value decomposition, characterized in that: include: Control module, Henstense module, CORDIC module, Jacobi rotation module and a multiplier module, where: The control module is used to control the timing, data flow and working status of each module; The Hestense module is responsible for calculating the square of the norm and the covariance between column vectors, and provides the parameters ERR and YX required for calculating the CORDIC module; The CORDIC module is used to calculate inverse trigonometric functions. It is composed of a multiplexer, registers, and shift registers. The CORDIC module receives the values of YX and ERR calculated by the Hestense module. The values enter the first CORDIC module in vector mode and calculate the corresponding A i and A j The rotation angle is input to the second CORDIC module in rotation mode, and the sine value sinθ and cosine value cosθ are output; The Jacobi rotation module is used to calculate the orthogonalization result of the input data. It consists of four multipliers, an adder and a subtractor. The four multipliers perform parallel operations and have four input ports to calculate A i 、A j The product of their corresponding sine and cosine values is output to the adder and subtractor to obtain the corresponding orthogonalized result C i and C j ; The Hestense module is composed of a first multiplier, an accumulator, a register, a shifter, a first dual-port RAM, and a second dual-port RAM. The input end of the first dual-port RAM is connected to the output end of the multiplexer, the output end of the first dual-port RAM is connected to the input end of the first multiplier, the output end of the first multiplier is connected to the accumulator, and the accumulator is connected to the adder through the second dual-port RAM. When data is taken out of the memory in vector format and sequentially input to the input end of the Hestense module, it is recorded as A. i , at this time A j The input terminal inputs each row vector in the array in sequence, and i First, the two vectors are multiplied, that is, the elements in the two vectors are multiplied and added; second, when A i When performing multiplication and accumulation operations with itself, the output is X, A j The output of the multiplication and accumulation operation with itself is Y, A i With A j The output of the multiplication and accumulation operation is ERR. When ERR is less than the preset value, it indicates that the orthogonalization operation is completed.

2. The hardware circuit of the principal component analysis algorithm based on singular value decomposition according to claim 1, characterized in that: The Jacobi rotation module realizes the calculation of eigenvalues in the principal component analysis algorithm through the calculation method of unilateral Jacobi rotation.

3. The hardware circuit of the principal component analysis algorithm based on singular value decomposition according to claim 2, characterized in that: The Jacobi rotation module implements the calculation of eigenvalues in the principal component analysis algorithm by a unilateral Jacobi rotation calculation method, specifically comprising the following steps: Perform rotation calculation on any matrix. Each round of iteration makes any two columns of the data set orthogonal once until the output value is orthogonal. A=USV' (1) Then there is AV=US (2) Where A is the matrix of the input data set; S is the singular value corresponding to the input matrix; U is the left singular vector; V is the right singular vector. Observe that the right side of the equation is an orthogonal matrix. Therefore, the core of the Jacobi rotation is to transform the matrix A into an orthogonal matrix through a series of linear transformations. Represents the orthogonalized vector obtained after calculation; represents the vector of the i or jth row of the input, θ represents the rotation angle corresponding to the orthogonalized vector, and we can get Calculate sine and cosine values in CORDIC rotation mode x0, y0 represent the two initial vectors of input, x n 、y n Represents cosine and sine values, z n Indicates the angle after superposition, set to 0 in rotation mode, K n Represents the scaling factor, z0 represents the rotation angle, n represents the number of rows or columns of the matrix, and i represents the number of iterations, thus we can get The input data is converted into matrix calculations using dual-port RAM, and the matrix calculations use a one-sided Jacobian rotation. Rounding the eigenvalues in each iteration ensures the correctness of the calculation, and a non-restoring square root algorithm is used to ensure accuracy. After the input data is converted into matrix operations, the matrix is transposed through the Hestense module and expanded to allow matrix calculations of any size and shape. When the orthogonalization error set in Hestense reaches the predefined convergence value, it indicates that all rows in the matrix generated by the data set are orthogonal.

4. A method for implementing the hardware circuit according to any one of claims 1 to 3, characterized in that: The following steps are involved: (1) The control module controls the input data parameters to be sent to the Hestense module for calculation; (2) The Hestense module performs multiplication and accumulation operations on the input parameters; (3) The CORDIC module calculates the corresponding sine and cosine values based on the output of the Hestense module; (4) The Jacobi rotation module calculates the orthogonalization results corresponding to the system input parameters based on the sine and cosine values calculated by the CORDIC module, and feeds them back to the Hestense module for calculation.

Images