Modeling method for optimizing gradient descent process based on SVD algorithm
By introducing gradient descent and bias terms into the SVD algorithm, the learning rate function is optimized, and the time complexity and sparse matrix adaptability of the SVD algorithm in matrix decomposition is solved, and the degree of matching between users and products and the efficiency of the production process is improved. .
Patent Information
- Application Number
- CN202111074435.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-14
- Publication Date
- 2024-12-17
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing SVD algorithm has high time complexity and is not adapted to sparse matrices during matrix decomposition, which makes it difficult to match products with users in the clothing industrial Internet, and it is difficult to predict user needs, affecting production and market investment.
The optimized gradient descent process based on SVD algorithm is adopted, and the dimension reduction is decomposed into a low-rank matrix through the FunkSVD algorithm, and bias terms are added to the gradient descent process to form the BiasSVD algorithm, and the iteration process is optimized by the learning rate function to improve the efficiency of model construction.
It improves the speed and accuracy of the matrix decomposition model, improves the degree of matching between users and products, and thus optimizes product investment and market adaptability in the production process.
Smart Images
Figure CN113987742B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of clothing industrial Internet, and in particular to a modeling method for optimizing a gradient descent process based on an SVD algorithm. Background Art
[0002] In the existing technology, it is difficult to achieve the match between products and users, and it is even more difficult to predict the needs of users, which is not conducive to the investment in equipment in the production line, the processing and production of products, and the stable investment in the market. In linear algebra, the singular value decomposition method is a typical matrix factorization method, which is essentially a promotion and application of feature decomposition. It is also widely used in image processing. The core idea of using matrix decomposition for recommendation is to decompose the known rating matrix and project it into a low-dimensional space, avoiding problems such as data being too sparse. The traditional SVD algorithm decomposes the original large matrix into three small matrices for calculation. The process is relatively cumbersome and the time complexity is very high. In addition, the traditional algorithm requires the matrix to be as dense as possible, but in actual application scenarios, the matrix is generally sparse, so the shortcomings of the SVD algorithm lead to limitations in its use. Summary of the invention
[0003] The present invention overcomes the deficiencies of the prior art and provides a modeling method for optimizing the gradient descent process based on the SVD algorithm; the model building efficiency of the matrix decomposition SVD algorithm can be improved.
[0004] To achieve the above object, the technical solution adopted by the present invention is: a modeling method for optimizing the gradient descent process based on the SVD algorithm, comprising the following steps;
[0005] Step 1: Reduce the dimension of the problem to be solved into two low-rank matrices using the FunkSVD algorithm. Formula 1 is as follows: Where R is an m×n matrix, P is a k×m matrix, and Q is a k×n matrix. m and n are the length and width of the matrix in the k-dimensional space, respectively.
[0006] Step 2: Map both users and items into k-dimensional space; in k-dimensional space, there are k corresponding hidden factors; map items and users into k-dimensional space, and use the gradient descent process to optimize the distribution results, where.
[0007] Step three, use the gradient descent algorithm to solve; and add a bias term to the FunkSVD algorithm in the gradient descent algorithm to form a BiasSVD algorithm; use the gradient descent method to iteratively process, calculate the parameter values in the formula, find the optimal solution by comparing the parameter values, and obtain the model.
[0008] In a preferred embodiment of the present invention, in step 2, in the established model, the closer the distance between the user and the item, the more likely the user is to like the item, which is manifested in that the positive and negative values of each hidden factor are more consistent, and the degree of liking can be quantified by the score; the hidden factor is the vector projection of the user in the K-dimensional space. Specifically, the hidden factor is the k-dimensional vector corresponding to each user / item in the decomposed matrix.
[0009] Specifically, in the case of the FunkSVD algorithm, the user's rating of the item depends on these hidden factors. The process of decomposing into two matrices is essentially a dimensionality reduction process, in which the idea of linear regression is adopted. After the items and users are mapped to the k-dimensional space, the distribution results are optimized by the gradient descent process. In the final trained model, the closer the distance between the user and the item, the more likely the user is to like the item, which is manifested in the more consistent positive and negative values of each hidden factor, and the degree of liking can be quantified by the score.
[0010] Specifically, the bias term includes the global average μ of the score in the parameter set, the user bias b i 、Item bias b j ; User bias b i is the user's preference, item bias b j Add a bias term for the quality difference of items; user bias b i Independent of item feature factors, it indicates the user’s rating habits. j Independent of user interest factors, it indicates the score of the item.
[0011] In a preferred embodiment of the present invention, in step 3, the objective function of the FunkSVD algorithm is Formula 2: Where K is the set of existing rating records, m ij is the rating of user i on item j, λ is the regularization coefficient; p i is the user vector, q j is the item vector, and T is the number of iterations.
[0012] In a preferred embodiment of the present invention, in step 4, a bias term is added to the FunkSVD algorithm to form a BiasSVD algorithm; Formula 3 of the BiasSVD algorithm is,
[0013]
[0014] The obtained gradient descent formula of BiasSVD is as follows;
[0015] Formula 4: Calculate the prediction error value
[0016] Formula 5,
[0017] Formula 6:
[0018] Formula 7,
[0019] Formula 8
[0020] Among them, Formula 5 to Formula 8 represent the updates of p matrix and q matrix, and b i , b j The parameter update process, formula 4 is used to calculate the prediction error value; μ is the average value of the existing scores, b i and b j denote user and item bias terms respectively, is the prediction score of the algorithm, λ is the regularization coefficient, and p i is the user vector, q j is the item vector, α is the learning rate; p ik and q jk are each user vector and item vector respectively; m ij is the rating of item j by user i.
[0021] In a preferred embodiment of the present invention, an optimization function is introduced in the modeling process to improve the change of the learning rate, which is used in the iteration process of the BiasSVD algorithm in the modeling process to optimize the algorithm operation efficiency and quickly build the model.
[0022] In a preferred embodiment of the present invention, the learning rate function formula is Formula 9 Where t is the number of iterations, a0 is the initial value of the learning rate, β is the slope control constant; a1 is the iterative process control constant; according to the gradient descent formula, the algorithm model is constructed to optimize the algorithm's operating efficiency.
[0023] Specifically, the learning rate function formula nine a(t) corresponds to α in the above formulas five to eight. The learning rate function formula controls the speed of the gradient descent process and controls the accuracy according to the speed. The initial value of the learning rate is pre-set in the modeling; the change of the learning rate is controlled by combining the power function and the exponential function in the change function. The value calculated by the learning rate function formula nine a(t) is the learning rate function α in formulas five to eight. Among them, the learning rate function α is 0.027. The initial value of the learning rate a0 pre-set by the system is 0.003. a1 is 1.3 to 1.9. β is 0.04 to 0.08. Specifically, a1 is 1.5; β is 0.05.
[0024] A method for optimizing a gradient descent process based on an SVD algorithm, a model established by a modeling method based on an SVD algorithm to optimize a gradient descent process, and the method for optimizing a gradient descent process comprises the following steps:
[0025] Step S1, initializing the user-item rating matrix;
[0026] Step S2, initializing a feature matrix, wherein the feature matrix includes a user feature matrix and an item feature matrix;
[0027] Step S3, initializing a bias vector, the bias vector includes a user bias term and an item bias term;
[0028] Step S4, calculating the prediction score;
[0029] Step S5, calculating the error value, the error is calculated as the difference between the error of this iteration and the error of the previous iteration, and updating the parameters;
[0030] Step S6, calculating the error change value according to the parameters, the error change value being the difference between the error of this iteration and the error of the previous iteration; updating the parameters to the parameters in formulas 5 to 8;
[0031] Step 7: Determine whether to stop the algorithm or execute the loop according to the size of the square mean error value of the error change value.
[0032] In a preferred embodiment of the present invention, step S5 includes step S5-1, updating the user feature matrix and the item feature matrix according to the scores calculated in step S4; and step S5-2, updating the user bias matrix and the item bias matrix.
[0033] In a preferred embodiment of the present invention, step S6 includes step S6-1, determining the current square mean error value, that is, The square mean error exceeds the error change threshold of 0.001; step S6-2, the current square mean error value does not exceed the error change threshold of 0.001, indicating that the square mean error value does not change substantially, and the optimization operation can be terminated; or / and, in step S7, the calculated error value is compared with the error value in the previous cycle, and if the error value is stable within 0.0001, the calculation is terminated.
[0034] A system for realizing an optimized gradient descent process based on an SVD algorithm comprises a computer having a model established by a modeling method for realizing an optimized gradient descent process based on an SVD algorithm.
[0035] The present invention solves the defects existing in the background technology, and the beneficial effects of the present invention are as follows:
[0036] The gradient descent learning rate function provided by the present invention is used to construct the model, which optimizes the speed and accuracy of the model construction. The construction and running speed of the model are optimized to ensure the overall accuracy of the model.
[0037] Model building through the gradient descent learning rate function can improve the matching degree between users and products, and then adjust the output of products in the production process according to the matching degree, and help improve the adaptability of products in the market in the subsequent production process. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0039] Figure 1 It is a modeling flow chart in a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0040] The present invention will now be further described in detail with reference to the accompanying drawings and embodiments. These drawings are simplified schematic diagrams that only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.
[0041] Embodiment 1
[0042] To achieve the above object, the technical solution adopted by the present invention is: a modeling method for optimizing the gradient descent process based on the SVD algorithm, comprising the following steps;
[0043] Step 1: Reduce the dimension of the problem to be solved into two low-rank matrices using the FunkSVD algorithm. Formula 1 is as follows: Where R is an m×n matrix, P is a k×m matrix, and Q is a k×n matrix. m and n are the length and width of the matrix in the k-dimensional space, respectively.
[0044] Step 2: Map users and items to k-dimensional space; in k-dimensional space, there are k hidden factors; map items and users to k-dimensional space, and use the gradient descent process to optimize the distribution results, where. In the established model, the closer the distance between the user and the item, the more likely the user is to like the item, which is reflected in the more consistent positive and negative values of each hidden factor, and the degree of liking can be quantified by the score; the hidden factor is the vector projection of the user in the k-dimensional space. Specifically, the hidden factor is the k-dimensional vector corresponding to each user / item in the decomposed matrix.
[0045] Step three, use the gradient descent algorithm to solve; and add a bias term to the FunkSVD algorithm in the gradient descent algorithm to form a BiasSVD algorithm; use the gradient descent method to iteratively process, calculate the parameter values in the formula, find the optimal solution by comparing the parameter values, and obtain the model.
[0046] The objective function of the FunkSVD algorithm is Formula 2: Where K is the set of existing rating records, m ij is the rating of user i on item j, λ is the regularization coefficient; p i is the user vector, q j is the item vector, and T is the number of iterations.
[0047] Specifically, in the case of the FunkSVD algorithm, the user's rating of the item depends on these hidden factors. The process of decomposing into two matrices is essentially a dimensionality reduction process, in which the idea of linear regression is adopted. After the items and users are mapped to the k-dimensional space, the distribution results are optimized by the gradient descent process. In the final trained model, the closer the distance between the user and the item, the more likely the user is to like the item, which is manifested in the more consistent positive and negative values of each hidden factor, and the degree of liking can be quantified by the score.
[0048] Specifically, the bias term includes the global average μ of the score in the parameter set, the user bias b i 、Item offset b j ; User bias b i is the user's preference, item bias b j Add a bias term for the quality difference of items; user bias b i Independent of item feature factors, it indicates the user’s rating habits. j Independent of user interest factors, it indicates the score of the item.
[0049] In step 4, a bias term is added to the FunkSVD algorithm to form the BiasSVD algorithm; the formula 3 of the BiasSVD algorithm is,
[0050]
[0051] The obtained gradient descent formula of BiasSVD is as follows;
[0052] Formula 4: Calculate the prediction error value
[0053] Formula 5,
[0054] Formula 6:
[0055] Formula 7,
[0056] Formula 8,
[0057] Among them, Formula 5 to Formula 8 represent the updates of p matrix and q matrix, and bi , b j The parameter update process, formula 4 is used to calculate the prediction error value; μ is the average value of the existing scores, b i and b j denote user and item bias terms respectively, is the prediction score of the algorithm, λ is the regularization coefficient, and p i is the user vector, q j is the item vector, α is the learning rate; p ik and q jk are each user vector and item vector respectively; m ij is the rating of item j by user i.
[0058] In a preferred embodiment of the present invention, an optimization function is introduced in the modeling process to improve the change of the learning rate, which is used in the iteration process of the BiasSVD algorithm in the modeling process to optimize the algorithm operation efficiency and quickly build the model.
[0059] In a preferred embodiment of the present invention, the learning rate function formula is Formula 9 Where t is the number of iterations, a0 is the initial value of the learning rate, β is the slope control constant, a1 is the iterative process control constant, and according to the gradient descent formula, the algorithm model is constructed to optimize the algorithm's operating efficiency.
[0060] Specifically, the learning rate function formula nine a(t) corresponds to α in the above formulas five to eight. The learning rate function formula controls the speed of the gradient descent process and controls the accuracy according to the speed. The initial value of the learning rate a0 is pre-set in the modeling; the change of the learning rate is controlled by combining the power function and the exponential function in the change function. The value calculated by the learning rate function formula nine a(t) is the learning rate function α in formulas five to eight. Among them, the learning rate function α is 0.027. The initial value of the learning rate a0 pre-set by the system is 0.003. a1 is 1.3 to 1.9. β is 0.04 to 0.08. Specifically, a1 is 1.5; β is 0.05.
[0061] Furthermore, the user bias term refers to the use of a model to calculate and correct the user's scoring tendency for all items. For example, user A will give 3 points to an item that he is not very satisfied with, and 5 points to an item that he likes. User B may give 1 point and 4 points to the same item. The same item is measured by different subjective standards during the user's scoring process, resulting in a large gap. The user bias term is used to neutralize and reduce this subjective difference, thereby obtaining a more accurate score. Similarly, the item bias term can be used to adjust the score difference caused by the characteristics of the item itself.
[0062] Embodiment 2
[0063] A method for optimizing a gradient descent process based on an SVD algorithm, wherein the method for optimizing a gradient descent process in a model established by the modeling method for optimizing a gradient descent process based on an SVD algorithm in Embodiment 1 comprises the following steps:
[0064] Step S1, initializing the user-item rating matrix;
[0065] Step S2, initializing a feature matrix, wherein the feature matrix includes a user feature matrix and an item feature matrix;
[0066] Step S3, initializing a bias vector, the bias vector includes a user bias term and an item bias term;
[0067] Step S4, calculating the prediction score;
[0068] Step S5, calculate the error value, the error is calculated as the difference between the error of this iteration and the error of the previous iteration, and update the parameters.
[0069] The method comprises step S5-1, updating the user feature matrix and the item feature matrix according to the scores calculated in step S4; and step S5-2, updating the user bias matrix and the item bias matrix.
[0070] Step S6, calculating the error change value according to the parameters, the error change value being the difference between the error of this iteration and the error of the previous iteration; updating the parameters to the parameters in Formula 5 to Formula 8.
[0071] Including step S6-1, judging the current square mean error value, that is, Whether the square mean error exceeds the error change threshold of 0.001; Step S6-2, the current square mean error value does not exceed the error change threshold of 0.001, indicating that the square mean error value basically does not change, and the optimization operation can be ended.
[0072] Step 7: Determine whether to stop the algorithm or loop execution based on the size of the square mean error of the error change value. Compare the calculated error value with the error value in the previous loop. If the error value is stable within 0.0001, the calculation ends.
[0073] Embodiment 3
[0074] Based on the first or second embodiment, a system for optimizing a gradient descent process based on an SVD algorithm includes a computer having a model established by a modeling method for optimizing a gradient descent process based on an SVD algorithm.
[0075] Working principle:
[0076] Compared with the learning rate variation function in the conventional SVD algorithm, the gradient descent learning rate function constructed by the present invention optimizes the speed and accuracy of model construction. In the process of optimization, the application of the algorithm in the actual system is mainly considered, the running speed of the algorithm is mainly optimized, and the overall accuracy of the model built is guaranteed. The present invention also has good practicality in other algorithms of SVD and matrix decomposition. Model construction through the gradient descent learning rate function can improve the matching degree between users and products, and then adjust the output of the product put into production in the production process according to the matching degree. Assist in improving the adaptability of the product in the market in the subsequent production process, and at the same time assist in improving the use correspondence of production equipment, and assist in improving the rationalization investment of the equipment in the corresponding production line.
[0077] The above is based on the ideal embodiment of the present invention. Through the above description, relevant personnel can make various changes and modifications without departing from the technical concept of the present invention. The technical scope of the present invention is not limited to the content in the specification, and the technical scope must be determined according to the scope of the claims.
Claims
1. A modeling method for optimizing the gradient descent process based on the SVD algorithm, characterized in that: The steps include: Step 1: Reduce the dimension of the problem to be solved into two low-rank matrices using the FunkSVD algorithm. Formula 1 is as follows: Where R is an m×n matrix, P is a k×m matrix, and Q is a k×n matrix. m and n are the length and width of the matrix in the k-dimensional space respectively. Step 2: Map users and items into k-dimensional space; in k-dimensional space, there are k hidden factors; map items and users into k-dimensional space, and optimize the distribution results using the gradient descent process; In step 2, in the established model, the closer the distance between the user and the item, the more likely the user is to like the item, which is reflected in the more consistent positive and negative values of each hidden factor, and the degree of liking can be quantified by the score; the hidden factor is the vector projection of the user in the K-dimensional space; Step 3: Use gradient descent algorithm to solve; And add a bias term to the FunkSVD algorithm in the gradient descent algorithm to form the BiasSVD algorithm; Use the gradient descent method to iterate, calculate the parameter values in the formula, find the optimal solution by comparing the parameter values, and obtain the model; In step 3, the objective function of the FunkSVD algorithm is Formula 2: Where K is the set of existing rating records, m ij is the rating of user i on item j, λ is the regularization coefficient; p i is the user vector, q j is the item vector, T is the number of iterations; Step 4: Add a bias term to the FunkSVD algorithm to form the BiasSVD algorithm; Formula 3 of the BiasSVD algorithm is: The obtained gradient descent formula of BiasSVD is as follows: Formula 4, calculate the prediction error value: Formula 5, Formula 6: Formula 7, Formula 8, Among them, Formula 5 to Formula 8 represent the updates of p matrix and q matrix, and b i ,b j The parameter update process, formula 4 is used to calculate the prediction error value; μ is the average value of the existing scores, b i and b j denote user and item bias terms respectively, is the prediction score of the algorithm, λ is the regularization coefficient, and p i is the user vector, q j is the item vector, α is the learning rate; p ik and q jk are each user vector and item vector respectively; m ij is the rating of item j by user i; The learning rate function formula is Formula 9 Where t is the number of iterations, a0 is the initial value of the learning rate, β is the slope control constant, a1 is the iterative process control constant, and according to the gradient descent formula, the algorithm model is constructed to optimize the algorithm's operating efficiency.
2. The modeling method for optimizing the gradient descent process based on the SVD algorithm according to claim 1, characterized in that: An optimization function is introduced in the modeling process to improve the change of learning rate. It is used in the iteration process of the BiasSVD algorithm in the modeling process to optimize the algorithm operation efficiency and quickly build the model.
3. A method for optimizing the gradient descent process based on the SVD algorithm, characterized in that: The model established by the modeling method for optimizing the gradient descent process based on the SVD algorithm according to claim 1 or 2 is adopted, and the method for optimizing the gradient descent process comprises the following steps: Step S1, initializing the user-item rating matrix; Step S2, initializing a feature matrix, wherein the feature matrix includes a user feature matrix and an item feature matrix; Step S3, initializing a bias vector, the bias vector includes a user bias term and an item bias term; Step S4, calculating the prediction score; Step S5, calculating the error value, the error is calculated as the difference between the error of this iteration and the error of the previous iteration, and updating the parameters; Step S6, calculating the error change value according to the parameters, the error change value is the difference between the error of this iteration and the error of the previous iteration; Update the parameters to those in Formula 5 to Formula 8; Step S7, judging whether to stop the algorithm or execute it in a loop according to the size of the square mean error value of the error change value.
4. The method for optimizing the gradient descent process based on the SVD algorithm according to claim 3, characterized in that: The step S5 includes: step S5-1, updating the user feature matrix and the item feature matrix according to the scores calculated in step S4; step S5-2, updating the user bias matrix and the item bias matrix.
5. The method for optimizing the gradient descent process based on the SVD algorithm according to claim 4, characterized in that: The step S6 includes step S6-1, determining the current square mean error value, that is, The square mean error exceeds the error change threshold of 0.001; step S6-2, the current square mean error value does not exceed the error change threshold of 0.001, indicating that the square mean error value does not change substantially, and the optimization operation can be terminated; or / and, in step S7, the calculated error value is compared with the error value in the previous cycle, and if the error value is stable within 0.0001, the calculation is terminated.
6. A system for optimizing the gradient descent process based on the SVD algorithm, characterized in that: The invention comprises a computer having a model established by a modeling method for optimizing a gradient descent process based on an SVD algorithm as described in claim 1 or 2.