Application software recommendation method and system based on big data

By cleaning and feature extraction of user behavior data, combined with the adjustment of the recommendation model, the integration of different data sources is solved, the accuracy and stability of application software recommendations are improved, the problems of data loss and overfitting are solved, and the problems of data loss and overfitting are achieved. More efficient data integration and recommendation results.

CN118708820BActive Publication Date: 2024-12-06SANMEN ZHENSHUN ELECTRONIC PROD CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411188472.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

In the prior art, due to the different data formats in different data sources, the data cannot be correctly integrated during integration, and some data is lost, resulting in a decrease in the accuracy of application software recommendations.

Method used

By collecting user behavior data for cleaning, conversion, deduplication and compression, extracting features and building user portraits, using the recommended model for training and real-time updates, adjusting the data compression ratio and learning rate according to the number of recommended errors and interrupt time, and adjusting data packet transmission Cross distances to improve recommendation accuracy and stability.

Benefits of technology

By reducing the data compression ratio and learning rate, reducing the risk of data loss and overfitting, increasing the cross distance of data packet transmission to improve data integration accuracy and recommendation real-time, and improving the recommendation accuracy of application software.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118708820B_ABST
    Figure CN118708820B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of data processing technology, and in particular to a method and system for recommending application software based on big data, the method comprising: transmitting the collected user behavior data to a location to be processed, and performing cleaning, conversion, deduplication and compression operations in sequence to output standardized data, and extracting features from the standardized data; constructing a user portrait based on the user's personal information and the user behavior data, using the extracted features to train a recommendation model and update it in real time, and using the trained recommendation model in combination with the user portrait to generate a recommendation list; determining whether the recommendation accuracy of the application software meets the requirements based on the proportion of the number of errors of the recommended application software; if the recommendation accuracy does not meet the requirements, adjusting the compression ratio of the user behavior data; if the operating stability does not meet the requirements, adjusting the learning rate of the recommendation model. The present invention improves the recommendation accuracy of application software.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for recommending application software based on big data. Background Art

[0002] In the existing technology, with the rapid development of mobile Internet and the popularization of smart phones, the data traffic in the operator's network has continued to surge; at the same time, the combination of the Internet and mobile communications has also changed the original ecological environment of the communications industry, and the proportion of traffic generated by mobile application software in the communication network has also increased. The reason is that the operator's own business products have long been unable to meet the diverse needs of customers. Therefore, in addition to marketing their own business, operators recommending mobile application software to customers has become an important supplementary means to meet customer needs and improve customer satisfaction.

[0003] Chinese Patent Publication No.: CN104408640A discloses an application software recommendation method and device, including: obtaining application software to be recommended; taking each application software to be recommended as the application software to be calculated, respectively, calculating the recommendation value of the application software to be calculated according to at least two preset recommendation strategies, and calculating the sum of all the recommendation values ​​corresponding to each application software to be recommended; if the sum of all the recommendation values ​​corresponding to the application software to be recommended is greater than or equal to a threshold, then recommending the application software to be recommended to the user. It can be seen that the application software recommendation method and device have the problem that the formats of the data in different data sources may be different, resulting in the inability to correctly integrate the processed data, and part of the data is lost, thereby causing the recommendation accuracy of the application software to decrease. Summary of the invention

[0004] To this end, the present invention provides an application software recommendation method and system based on big data, so as to overcome the problem in the prior art that the formats of data in different data sources may be different, resulting in the inability to correctly integrate the processed data and part of the data being lost, thereby reducing the accuracy of application software recommendations.

[0005] To achieve the above-mentioned purpose, the present invention provides an application software recommendation method based on big data, comprising: transmitting the collected user behavior data to a location to be processed, and cleaning, converting, deduplicating and compressing the user behavior data in sequence to output standardized data, and extracting features from the standardized data; constructing a user portrait based on the user's personal information and the user behavior data, using the extracted features to train and update the recommendation model in real time, and using the trained recommendation model in combination with the user portrait to generate a recommendation list; respectively obtaining the number of errors of recommended application software and the total number of recommended application software within several recommendation cycles; determining whether the recommendation accuracy of the application software meets the requirements based on the proportion of the number of errors of the recommended application software; if the recommendation accuracy does not meet the requirements, adjusting the compression ratio of the user behavior data or determining the running stability of the recommendation model based on the average interruption time of generating the recommendation list; if the running stability does not meet the requirements, adjusting the learning rate of the recommendation model or adjusting the transmission cross distance of the data packet of the user behavior data based on the average delay time of the recommendation model update.

[0006] Furthermore, the accuracy of application software recommendations is determined, including:

[0007] Obtain the number of errors of the recommended application software and the total number of recommended application software in several recommendation cycles respectively, and calculate the proportion of the number of errors of the recommended application software;

[0008] Comparing the error frequency ratio of the recommended application software with a preset first ratio;

[0009] If the error ratio of the recommended application software is greater than the preset first ratio, it is determined that the recommendation accuracy of the application software does not meet the requirements.

[0010] Furthermore, the operational stability of the recommendation model is determined, including:

[0011] Compare the error times ratio of the recommended application software with the preset first ratio and the preset second ratio respectively;

[0012] If the error ratio of the recommended application software is greater than the preset first ratio and less than or equal to the preset second ratio, it is preliminarily determined that the operating stability of the recommendation model does not meet the requirements, and a secondary determination is made on whether the operating stability of the recommendation model meets the requirements based on the average interruption duration of generating the recommendation list.

[0013] Furthermore, the compression ratio of the user behavior data is adjusted, including:

[0014] Comparing the error frequency ratio of the recommended application software with a preset second ratio;

[0015] If the error times ratio of the recommended application software is greater than the preset second ratio, the compression ratio of the user behavior data is reduced.

[0016] Furthermore, the reduction degree of the compression ratio of the user behavior data is determined by the difference between the error number ratio of the recommended application software and a preset second ratio.

[0017] Furthermore, the learning rate of the recommendation model is adjusted, including:

[0018] Obtain the interruption duration of generating recommendation lists within several recommendation cycles, and calculate the average interruption duration of generating recommendation lists;

[0019] Comparing the average interruption duration of the generated recommendation list with the preset first interruption duration and the preset second interruption duration respectively;

[0020] If the average interruption duration of generating the recommendation list is greater than the preset first interruption duration, it is determined that the operation stability of the recommendation model does not meet the requirements;

[0021] If the average interruption duration of generating the recommendation list is greater than the preset first interruption duration and less than or equal to the preset second interruption duration, reducing the learning rate of the recommendation model;

[0022] If the average interruption duration of generating the recommendation list is greater than the preset second interruption duration, it is preliminarily determined that the real-time performance of the recommendation model update does not meet the requirements, and a secondary determination is made as to whether the real-time performance of the recommendation model update meets the requirements based on the average delay duration of the recommendation model update.

[0023] Furthermore, the reduction range of the learning rate of the recommendation model is determined by the difference between the average interruption duration of generating the recommendation list and the preset first interruption duration.

[0024] Furthermore, the amount of user behavior data transmitted in a single batch is adjusted, including:

[0025] Obtain the delay time of the recommendation model update in several update cycles, and calculate the average delay time of the recommendation model update;

[0026] Comparing the average delay time of updating the recommendation model with the preset delay time;

[0027] If the average delay time of the recommendation model update is greater than the preset delay time, it is determined that the real-time performance of the recommendation model update does not meet the requirements, and the transmission cross distance of the data packet of the user behavior data is increased.

[0028] Furthermore, the increase in the transmission cross distance of the data packets of the user behavior data is determined by the difference between the average delay time of the recommendation model update and the preset delay time.

[0029] The present invention also provides an application software recommendation system based on big data, comprising:

[0030] Data collection module, used to collect user behavior data;

[0031] A data processing module, which is connected to the data acquisition module and includes a data transmission unit for transmitting the user behavior data to a location to be processed and a preprocessing unit connected to the data transmission unit for performing a preprocessing operation on the user behavior data to output standardized data;

[0032] a software recommendation module connected to the data processing module, comprising a model training unit connected to the feature extraction unit for training a recommendation model using the extracted features, a model updating unit connected to the model training unit for updating the recommendation model, and a list generating unit connected to the model updating unit for generating a recommendation list;

[0033] A storage module, which is connected to the data collection module, the data processing module and the software recommendation module respectively, and is used to store the user behavior data, the standardized data, the features, the recommendation model and the recommendation list respectively;

[0034] A control module, which is respectively connected to the data acquisition module, the data processing module, the software recommendation module and the storage module, and is used to adjust the compression ratio of the user behavior data or the learning rate of the recommendation model according to the average interruption time of generating the recommendation list when determining that the recommendation accuracy of the application software does not meet the requirements according to the proportion of the number of errors of the recommended application software, and to adjust the transmission cross-distance of the data packet of the user behavior data according to the average interruption time of generating the recommendation list and the average delay time of updating the recommendation model.

[0035] Compared with the prior art, the beneficial effect of the present invention lies in that the method of the present invention adjusts the compression ratio of user behavior data according to the proportion of the number of errors in the recommended application software. Since the formats of the data in different data sources may be different, the processed data cannot be correctly integrated when being integrated, resulting in the loss of some data, which makes it impossible to accurately understand the user's preferences for recommendation. By reducing the compression ratio of user behavior data, it is helpful to retain more data details and features, thereby improving the accuracy of data integration. The learning rate of the recommendation model is adjusted according to the average interruption time of generating the recommendation list. Since the model is trained too many times and the training data is almost not increased, the error of the model on the training set is very small, which makes the model overfit, resulting in errors when generating the recommendation list. Errors or interruptions occur. By reducing the learning rate of the recommendation model, the model can be adjusted with a smaller step size when it is close to the optimal solution, reducing the risk of skipping the optimal solution or excessive oscillation near the optimal solution, which helps to alleviate overfitting. The transmission cross-distance of the user behavior data packet is adjusted according to the average delay time of the recommendation model update. Due to the excessive amount of data that needs to be processed, some data packets may need to be retransmitted due to network fluctuations during the transmission process, resulting in increased transmission time, thereby reducing the real-time nature of the update. By increasing the transmission cross-distance of the user behavior data packet, the time delay caused by retransmission and the excessive occupation of network bandwidth are reduced, the transmission frequency is increased, the real-time nature of the update is improved, and the recommendation accuracy of the application software is improved.

[0036] Furthermore, the method of the present invention adjusts the compression ratio of user behavior data by setting a preset first ratio and a preset second ratio. Since the formats of data in different data sources may be different, the processed data cannot be correctly integrated, resulting in partial data loss, which makes it impossible to accurately understand user preferences for recommendation. By reducing the compression ratio of user behavior data, it helps to retain more data details and features, thereby improving the accuracy of data integration and further improving the accuracy of application software recommendations.

[0037] Furthermore, the method of the present invention adjusts the learning rate of the recommendation model by setting a preset first interruption duration and a preset second interruption duration. Since the model is trained too many times and the training data is almost not increased, the error of the model on the training set is very small, which makes the model overfit, resulting in errors or interruptions when generating the recommendation list. By reducing the learning rate of the recommendation model, the model can be adjusted with a smaller step size when approaching the optimal solution, reducing the risk of skipping the optimal solution or excessively oscillating near the optimal solution, which helps to alleviate the overfitting situation and further improves the recommendation accuracy of the application software.

[0038] Furthermore, the method of the present invention adjusts the transmission cross distance of the data packets of the user behavior data by setting a preset delay time. Since the amount of data that needs to be processed is too large, some data packets may need to be retransmitted due to network fluctuations during the transmission process, resulting in an increase in transmission time, thereby causing the real-time performance of the update to decrease. By increasing the transmission cross distance of the data packets of the user behavior data, the time delay caused by retransmission and the excessive occupation of the network bandwidth are reduced, the frequency of transmission is increased, the real-time performance of the update is improved, and the accuracy of the recommendation of the application software is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is an overall flow chart of the application software recommendation method based on big data according to an embodiment of the present invention;

[0040] Figure 2 This is an overall structural diagram of the application software recommendation system based on big data according to an embodiment of the present invention;

[0041] Figure 3 A logical flow chart of a method for recommending application software based on big data according to an embodiment of the present invention;

[0042] Figure 4 This is a specific flow chart of the process of adjusting the compression ratio of user behavior data in the application software recommendation method based on big data in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0045] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0046] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0047] See also Figure 1 , Figure 2 , Figure 3 as well as Figure 4 As shown, they are respectively an overall flow chart, an overall structure block diagram, a logic flow chart, and a specific flow chart of the process of adjusting the compression ratio of user behavior data of the application software recommendation method and system based on big data in the embodiment of the present invention. The present invention is an application software recommendation method based on big data, comprising:

[0048] Step S1, transmitting the collected user behavior data to a location to be processed, and sequentially cleaning, converting, deduplicating and compressing the user behavior data to output standardized data, and performing feature extraction on the standardized data;

[0049] Step S2, constructing a user profile based on the user's personal information and the user behavior data, using the extracted features to train a recommendation model and update it in real time, and using the trained recommendation model in combination with the user profile to generate a recommendation list;

[0050] Step S3, respectively obtaining the number of errors of the recommended application software and the total number of recommended application software in several recommendation cycles;

[0051] Step S4, determining whether the accuracy of application software recommendation meets the requirements based on the error ratio of the recommended application software;

[0052] Step S5: if the recommendation accuracy does not meet the requirement, the compression ratio of the user behavior data is adjusted or the operation stability of the recommendation model is determined based on the average interruption time of generating the recommendation list;

[0053] Step S6: If the operation stability does not meet the requirements, the learning rate of the recommendation model is adjusted or the transmission cross distance of the data packet of the user behavior data is adjusted based on the average delay time of the recommendation model update.

[0054] Specifically, user behavior data includes records of installed software, types of uninstalled software, and frequency of using the same software.

[0055] Specifically, standardized data includes the click rate of users on the software, the length of time users browse the software, and the keyword information used by users in search.

[0056] Specifically, the features include the number of times a user browses the software, the number of times a user clicks on the software, and the frequency of a user's search keywords.

[0057] Specifically, the user's personal information includes the user's age, gender, and interests and hobbies.

[0058] Specifically, user portraits include user purchase records, user browsing records, and user stay time.

[0059] Specifically, the recommendation model may be a collaborative filtering model, a matrix decomposition model, or a hybrid recommendation model.

[0060] Specifically, the recommendation list includes the name of the application software, the rating of the application software, and the type of the application software.

[0061] Specifically, the learning rate of the recommendation model is a hyperparameter that controls the step size of model parameter updates during training.

[0062] Specifically, the transmission cross distance of the data packets of the user behavior data is the interval between adjacent data packets of the user behavior data during the transmission process.

[0063] Specifically, before transmitting user behavior data, the data packets are divided into two categories, one is real data packets and the other is virtual data packets. The real data packets and the virtual data packets are transmitted crosswise, which can reduce the damage rate of the real data packets. In order to avoid the increase in transmission pressure due to the increase in data packets, the data stored in the virtual data packets are converted into a small part of the real data to be transmitted that is unimportant but related.

[0064] In implementation, the method of the present invention adjusts the compression ratio of user behavior data according to the proportion of the number of errors in the recommended application software. Since the formats of data in different data sources may be different, the processed data cannot be correctly integrated when being integrated, resulting in partial data loss, which makes it impossible to accurately understand user preferences for recommendation. By reducing the compression ratio of user behavior data, it is helpful to retain more data details and features, thereby improving the accuracy of data integration. The learning rate of the recommendation model is adjusted according to the average interruption time of generating the recommendation list. Since the model is trained too many times and the training data is almost not increased, the error of the model on the training set is very small, which makes the model overfit, resulting in errors or interruptions when generating the recommendation list. By reducing the learning rate of the recommendation model, the model can be adjusted with a smaller step size when approaching the optimal solution, reducing the risk of skipping the optimal solution or excessive oscillation near the optimal solution, which helps to alleviate overfitting. The transmission cross-distance of the user behavior data packets is adjusted according to the average delay time of the recommendation model update. Since the amount of data that needs to be processed is too much, some data packets may need to be retransmitted due to network fluctuations during the transmission process, resulting in increased transmission time, which in turn reduces the real-time nature of the update. By increasing the transmission cross-distance of the user behavior data packets, the time delay caused by retransmission and excessive occupation of network bandwidth are reduced, the transmission frequency is increased, the real-time nature of the update is improved, and the accuracy of the application software recommendation is improved.

[0065] Specifically, determining the accuracy of application recommendations includes:

[0066] Obtain the number of errors of the recommended application software and the total number of recommended application software in several recommendation cycles respectively, and calculate the proportion of the number of errors of the recommended application software;

[0067] Comparing the error frequency ratio of the recommended application software with a preset first ratio;

[0068] If the error ratio of the recommended application software is greater than the preset first ratio, it is determined that the recommendation accuracy of the application software does not meet the requirements.

[0069] Specifically, determine the operational stability of the recommendation model, including:

[0070] Compare the error times ratio of the recommended application software with the preset first ratio and the preset second ratio respectively;

[0071] If the error ratio of the recommended application software is greater than the preset first ratio and less than or equal to the preset second ratio, it is preliminarily determined that the operating stability of the recommendation model does not meet the requirements, and a secondary determination is made on whether the operating stability of the recommendation model meets the requirements based on the average interruption duration of generating the recommendation list.

[0072] It can be understood that the three intervals corresponding to the preset first proportion Q1 and the preset second proportion Q2 correspond to three situations respectively. The first interval is Q≤Q1, corresponding to the situation where the recommendation accuracy of the application software meets the requirements; the second interval is Q1<Q≤Q2, corresponding to too many model training times and almost no increase in training data, resulting in very small errors in the model on the training set, making the model overfitting, resulting in errors or interruptions when generating the recommendation list; the third interval is Q>Q2, corresponding to the data formats in different data sources may be different, resulting in the inability to correctly integrate the processed data, resulting in the loss of some data, and thus the inability to accurately understand the user's preferences for recommendation. In practice, Q1 is generally selected in the range of [0.04, 0.06], and Q2 is generally selected in the range of [0.07, 0.09].

[0073] Preferably, the preferred embodiment of the preset first proportion Q1 is 0.05, and the preferred embodiment of the preset second proportion Q2 is 0.08.

[0074] Specifically, the percentage of errors in the recommended application software is denoted as Q.

[0075] Specifically, the calculation formula for the percentage of errors in the recommended application software is:

[0076]

[0077] Among them, Q is the ratio of the number of errors in the recommended application software, T1 is the number of errors in the recommended application software within a number of recommendation cycles, and T2 is the total number of recommended application software.

[0078] In implementation, the method of the present invention determines the accuracy of application software recommendations by setting a preset first ratio and a preset second ratio, thereby reducing the impact of decreased stability in recommendation of application software and further improving the accuracy of application software recommendations.

[0079] Specifically, the compression ratio of user behavior data is adjusted, including:

[0080] Comparing the error frequency ratio of the recommended application software with a preset second ratio;

[0081] If the error times ratio of the recommended application software is greater than the preset second ratio, the compression ratio of the user behavior data is reduced.

[0082] Specifically, the reduction degree of the compression ratio of the user behavior data is determined by the difference between the error number ratio of the recommended application software and a preset second ratio.

[0083] Specifically, the calculation formula for the compression ratio of the reduced user behavior data is:

[0084]

[0085] Among them, V' is the compression ratio of the reduced user behavior data, V is the compression ratio of the current user behavior data, A is the minimum unit ratio for adjusting the compression ratio of the user behavior data, and v is the compression ratio of the user behavior data when the error ratio of the recommended application software is the minimum unit ratio.

[0086] In practice, the value of A is generally 0.8 to 1, and preferably, the preferred embodiment of A is 0.9.

[0087] In implementation, the optional range of the value of v is [0.3, 0.5]. Preferably, when the compression ratio of the user behavior data is less than 0.4, data loss may occur. Therefore, the preferred embodiment of v is 0.4.

[0088] Specifically, when the value of V' is a decimal, the value of V' is automatically rounded up to one decimal place.

[0089] For example, using the big data-based application software recommendation method to recommend application software to users, where V=0.3, Q=1.2, the calculation is .

[0090] Specifically, the closer the compression ratio of the user behavior data is to 0, the greater the compression ratio is. Conversely, the closer the compression ratio of the user behavior data is to 1, the smaller the compression ratio is.

[0091] In implementation, the method of the present invention adjusts the compression ratio of user behavior data by setting a preset first ratio and a preset second ratio. Since the formats of data in different data sources may be different, the processed data cannot be correctly integrated, resulting in partial data loss, which makes it impossible to accurately understand user preferences for recommendation. By reducing the compression ratio of user behavior data, it helps to retain more data details and features, thereby improving the accuracy of data integration and further improving the accuracy of application software recommendations.

[0092] Specifically, the learning rate of the recommendation model is adjusted, including:

[0093] Obtain the interruption duration of generating recommendation lists within several recommendation cycles, and calculate the average interruption duration of generating recommendation lists;

[0094] Comparing the average interruption duration of the generated recommendation list with the preset first interruption duration and the preset second interruption duration respectively;

[0095] If the average interruption duration of generating the recommendation list is greater than the preset first interruption duration, it is determined that the operation stability of the recommendation model does not meet the requirements;

[0096] If the average interruption duration of generating the recommendation list is greater than the preset first interruption duration and less than or equal to the preset second interruption duration, reducing the learning rate of the recommendation model;

[0097] If the average interruption duration of generating the recommendation list is greater than the preset second interruption duration, it is preliminarily determined that the real-time performance of the recommendation model update does not meet the requirements, and a secondary determination is made as to whether the real-time performance of the recommendation model update meets the requirements based on the average delay duration of the recommendation model update.

[0098] It can be understood that the three intervals corresponding to the preset first interruption duration P1 and the preset second interruption duration P2 correspond to three situations respectively. The first interval is P≤P1, corresponding to the situation where the operational stability of the recommendation model meets the requirements; the second interval is P1<P≤P2, corresponding to too many model training times and almost no increase in training data, resulting in a very small error of the model on the training set, making the model overfitting, resulting in errors or interruptions when generating the recommendation list; the third interval is P>P2, corresponding to too much data to be processed, resulting in the situation where some data packets may need to be retransmitted due to network fluctuations during transmission, resulting in increased transmission time. In practice, the range of P1 is generally selected from [0.2s, 0.4s], and the range of P2 is generally selected from [0.4s, 0.6s].

[0099] Preferably, the preferred embodiment of the preset first interruption duration P1 is 0.3 s, and the preferred embodiment of the preset second interruption duration P2 is 0.5 s.

[0100] Specifically, the average interruption duration of generating the recommendation list is denoted as P.

[0101] Specifically, the calculation formula for the average interruption duration of generating the recommendation list is:

[0102]

[0103] Among them, P is the average interruption time of generating recommendation lists, X a is the interruption duration of generating the recommendation list in the ath recommendation period, n is the number of recommendation periods, and n is a natural number greater than or equal to 1.

[0104] In implementation, the method of the present invention performs a secondary judgment on the operating stability of the recommendation model by setting a preset first interruption duration and a preset second interruption duration, thereby reducing the impact of the decreased recommendation accuracy of the application software due to inaccurate secondary judgment on the operating stability of the recommendation model, and further improving the recommendation accuracy of the application software.

[0105] Specifically, the reduction range of the learning rate of the recommendation model is determined by the difference between the average interruption duration of generating the recommendation list and the preset first interruption duration.

[0106] Specifically, the calculation formula for the reduced learning rate of the recommendation model is:

[0107]

[0108] Among them, H' is the learning rate of the recommendation model after reduction, H is the learning rate of the current recommendation model, B is the minimum unit interruption duration for adjusting the learning rate of the recommendation model, and h is the learning rate of the recommendation model when the average interruption duration for generating the recommendation list is the minimum unit interruption duration.

[0109] In practice, the value of B is generally 0.5s to 0.7s. Preferably, the preferred embodiment of B is 0.6s.

[0110] In implementation, the optional range of h is [0.02, 0.04]. Preferably, when the learning rate of the recommendation model is greater than 0.03, it may cause interruption in generating the recommendation list. Therefore, the preferred embodiment of h is 0.03.

[0111] For example, using the big data-based application software recommendation method to recommend application software to users, where H=0.04, P=0.6, we can calculate .

[0112] In implementation, the method of the present invention adjusts the learning rate of the recommendation model by setting a preset first interruption duration and a preset second interruption duration. Since the model is trained too many times and the training data is almost not increased, the error of the model on the training set is very small, which makes the model overfit, resulting in errors or interruptions when generating the recommendation list. By reducing the learning rate of the recommendation model, the model can be adjusted with a smaller step size when approaching the optimal solution, reducing the risk of skipping the optimal solution or excessively oscillating near the optimal solution, which helps to alleviate overfitting and further improves the recommendation accuracy of the application software.

[0113] Specifically, the amount of user behavior data transmitted in a single batch is adjusted, including:

[0114] Obtain the delay time of the recommendation model update in several update cycles, and calculate the average delay time of the recommendation model update;

[0115] Comparing the average delay time of updating the recommendation model with the preset delay time;

[0116] If the average delay time of the recommendation model update is greater than the preset delay time, it is determined that the real-time performance of the recommendation model update does not meet the requirements, and the transmission cross distance of the data packet of the user behavior data is increased.

[0117] It can be understood that the two intervals corresponding to the preset delay time Y0 correspond to two situations. The first interval is Y≤Y0, corresponding to the situation where the real-time update of the recommendation model meets the requirements; the second interval is Y>Y0, corresponding to the situation where the amount of data to be processed is too much, resulting in the possibility that some data packets need to be retransmitted due to network fluctuations during the transmission process, resulting in increased transmission time. In practice, Y0 is generally selected in the range of [0.3s, 0.5s].

[0118] Preferably, the preferred embodiment of the preset delay time length Y0 is 0.4s.

[0119] Specifically, the average delay time of the recommendation model update is denoted as Y.

[0120] Specifically, the calculation formula for the average delay time of the recommendation model update is:

[0121]

[0122] Among them, Y is the average delay time of recommendation model update, E f is the delay time of the recommendation model update in the fth update cycle, i is the number of update cycles, and i is a natural number greater than or equal to 1.

[0123] In implementation, the method of the present invention performs a secondary judgment on the real-time nature of the recommendation model update by setting a preset delay time, thereby reducing the impact of the decreased recommendation accuracy of the application software due to inaccurate secondary judgment on the real-time nature of the recommendation model update, and further improving the recommendation accuracy of the application software.

[0124] Specifically, the increase in the transmission cross distance of the data packets of the user behavior data is determined by the difference between the average delay time of the recommendation model update and the preset delay time.

[0125] Specifically, the calculation formula for the increased transmission cross distance of the data packet of the user behavior data is:

[0126]

[0127] Among them, L' is the increased transmission cross distance of the user behavior data packet, L is the current transmission cross distance of the user behavior data packet, C is the minimum unit delay time for adjusting the transmission cross distance of the user behavior data packet, and l is the transmission cross distance of the user behavior data packet when the average delay time of the recommendation model update is the minimum unit delay time.

[0128] In practice, the value of C is generally 0.4s to 0.6s, and preferably, the preferred embodiment of C is 0.5s.

[0129] In implementation, the optional value range of l is [4, 6]. Preferably, when the transmission cross-distance of the data packet of the user behavior data is greater than 5, the transmission time may increase. Therefore, the preferred embodiment of l is 5.

[0130] For example, using the big data-based application software recommendation method to recommend application software to users, where L=5.2, Y=0.6s, the calculation is .

[0131] In implementation, the method of the present invention adjusts the transmission cross-distance of the data packets of the user behavior data by setting a preset delay time. Since the amount of data that needs to be processed is too much, some data packets may need to be retransmitted due to network fluctuations during the transmission process, resulting in an increase in transmission time, thereby causing the real-time performance of the update to decrease. By increasing the transmission cross-distance of the data packets of the user behavior data, the time delay caused by retransmission and the excessive occupation of the network bandwidth are reduced, the transmission frequency is increased, the real-time performance of the update is improved, and the accuracy of the recommendation of the application software is further improved.

[0132] An application software recommendation system based on big data, comprising:

[0133] Data collection module, used to collect user behavior data;

[0134] A data processing module, which is connected to the data acquisition module and includes a data transmission unit for transmitting the user behavior data to a location to be processed and a preprocessing unit connected to the data transmission unit for performing a preprocessing operation on the user behavior data to output standardized data;

[0135] a software recommendation module connected to the data processing module, comprising a model training unit connected to the feature extraction unit for training a recommendation model using the extracted features, a model updating unit connected to the model training unit for updating the recommendation model, and a list generating unit connected to the model updating unit for generating a recommendation list;

[0136] A storage module, which is connected to the data collection module, the data processing module and the software recommendation module respectively, and is used to store the user behavior data, the standardized data, the features, the recommendation model and the recommendation list respectively;

[0137] A control module, which is respectively connected to the data acquisition module, the data processing module, the software recommendation module and the storage module, and is used to adjust the compression ratio of the user behavior data or the learning rate of the recommendation model according to the average interruption time of generating the recommendation list when determining that the recommendation accuracy of the application software does not meet the requirements according to the proportion of the number of errors of the recommended application software, and to adjust the transmission cross-distance of the data packet of the user behavior data according to the average interruption time of generating the recommendation list and the average delay time of updating the recommendation model.

[0138] Specifically, the data processing module also includes a feature extraction unit connected to the preprocessing unit for extracting features of the standardized data.

[0139] Specifically, the preprocessing operations include cleaning, converting, deduplicating, and compressing the user behavior data in sequence.

[0140] Specifically, the software recommendation module also includes a portrait construction unit connected to the model training unit for constructing a user portrait based on the user's personal information and the user behavior data.

[0141] Specifically, the user portrait is stored in the storage module.

[0142] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for recommending application software based on big data, characterized in that: include: The collected user behavior data is transmitted to a location to be processed, and the user behavior data is cleaned, converted, deduplicated, and compressed in sequence to output standardized data, and feature extraction is performed on the standardized data; Build a user profile based on the user's personal information and the user behavior data, use the extracted features to train and update the recommendation model in real time, and use the trained recommendation model combined with the user profile to generate a recommendation list; Obtain the number of errors of the recommended application software and the total number of recommended application software in several recommendation cycles respectively; Determining whether the accuracy of application software recommendation meets the requirements based on the percentage of errors of the recommended application software; If the recommendation accuracy does not meet the requirements, the compression ratio of the user behavior data is adjusted or the operation stability of the recommendation model is determined based on the average interruption time of generating the recommendation list; If the operation stability does not meet the requirements, the learning rate of the recommendation model is adjusted or the transmission cross distance of the data packet of the user behavior data is adjusted based on the average delay time of the recommendation model update; Determine the accuracy of application recommendations, including: Obtain the number of errors of the recommended application software and the total number of recommended application software in several recommendation cycles respectively, and calculate the proportion of the number of errors of the recommended application software; Comparing the error frequency ratio of the recommended application software with a preset first ratio; If the error ratio of the recommended application software is greater than the preset first ratio, it is determined that the recommendation accuracy of the application software does not meet the requirement; Determine the operational stability of the recommended model, including: Compare the error times ratio of the recommended application software with the preset first ratio and the preset second ratio respectively; If the error times ratio of the recommended application software is greater than the preset first ratio and less than or equal to the preset second ratio, it is preliminarily determined that the operation stability of the recommendation model does not meet the requirements, and a secondary determination is made on whether the operation stability of the recommendation model meets the requirements based on the average interruption time of generating the recommendation list; Adjust the compression ratio of user behavior data, including: Comparing the error frequency ratio of the recommended application software with a preset second ratio; If the error times ratio of the recommended application software is greater than the preset second ratio, reducing the compression ratio of the user behavior data; The reduction degree of the compression ratio of the user behavior data is determined by the difference between the error number ratio of the recommended application software and the preset second ratio.

2. The method for recommending application software based on big data according to claim 1, characterized in that: Adjust the learning rate of the recommendation model, including: Obtain the interruption duration of generating recommendation lists within several recommendation cycles, and calculate the average interruption duration of generating recommendation lists; Comparing the average interruption duration of the generated recommendation list with the preset first interruption duration and the preset second interruption duration respectively; If the average interruption duration of generating the recommendation list is greater than the preset first interruption duration, it is determined that the operation stability of the recommendation model does not meet the requirements; If the average interruption duration of generating the recommendation list is greater than the preset first interruption duration and less than or equal to the preset second interruption duration, reducing the learning rate of the recommendation model; If the average interruption duration of generating the recommendation list is greater than the preset second interruption duration, it is preliminarily determined that the real-time performance of the recommendation model update does not meet the requirements, and a secondary determination is made as to whether the real-time performance of the recommendation model update meets the requirements based on the average delay duration of the recommendation model update.

3. The method for recommending application software based on big data according to claim 2, characterized in that: The reduction range of the learning rate of the recommendation model is determined by the difference between the average interruption duration of generating the recommendation list and the preset first interruption duration.

4. The method for recommending application software based on big data according to claim 3, characterized in that: Adjust the transmission cross distance of the data packets of user behavior data, including: Obtain the delay time of the recommendation model update in several update cycles, and calculate the average delay time of the recommendation model update; Comparing the average delay time of updating the recommendation model with the preset delay time; If the average delay time of the recommendation model update is greater than the preset delay time, it is determined that the real-time performance of the recommendation model update does not meet the requirements, and the transmission cross distance of the data packet of the user behavior data is increased.

5. The method for recommending application software based on big data according to claim 4, characterized in that: The increase in the transmission cross distance of the data packets of the user behavior data is determined by the difference between the average delay time of the recommendation model update and the preset delay time.

6. A big data-based application software recommendation system using the big data-based application software recommendation method according to any one of claims 1 to 5, characterized in that: include: Data collection module, used to collect user behavior data; A data processing module, which is connected to the data acquisition module and includes a data transmission unit for transmitting the user behavior data to a location to be processed and a preprocessing unit connected to the data transmission unit for performing a preprocessing operation on the user behavior data to output standardized data; a software recommendation module connected to the data processing module, comprising a model training unit connected to the feature extraction unit for training a recommendation model using the extracted features, a model updating unit connected to the model training unit for updating the recommendation model, and a list generating unit connected to the model updating unit for generating a recommendation list; A storage module, which is connected to the data collection module, the data processing module and the software recommendation module respectively, and is used to store the user behavior data, the standardized data, the features, the recommendation model and the recommendation list respectively; A control module, which is respectively connected to the data acquisition module, the data processing module, the software recommendation module and the storage module, and is used to adjust the compression ratio of the user behavior data or the learning rate of the recommendation model according to the average interruption time of generating the recommendation list when determining that the recommendation accuracy of the application software does not meet the requirements according to the proportion of the number of errors of the recommended application software, and to adjust the transmission cross-distance of the data packet of the user behavior data according to the average interruption time of generating the recommendation list and the average delay time of updating the recommendation model.

Citation Information

Patent Citations

  • Application software recommending method and apparatus

    CN104408640A

  • Government affair user portrait construction method and system based on information resource library

    CN111723256A

  • Real-time product recommendation method and system based on time sequence deep reinforcement learning

    CN114240549A

  • Personalized search information recommendation system and method based on user portrait

    CN117493677A

  • Federal generalized matrix decomposition recommendation method based on incentive mechanism

    CN118132845A