Method, device, computer equipment and storage medium for processing content recommendation business

Abstract

The present application relates to a method, apparatus, computer equipment and storage medium for processing content recommendation services, the method comprising: obtaining a content recommendation service request, the content recommendation service request including an object identifier; searching for object features in a disk cache based on the object identifier; obtaining a disk cache hit rate when the object features found in the disk cache based on the object identifier are historical object features, wherein after the object features in the database are updated, the corresponding object features in the disk cache become historical object features; if the disk cache hit rate is lower than a threshold, obtaining the object features corresponding to the object identifier from the disk cache; and determining recommended content based on the object features obtained from the disk cache. This method can increase the disk hit rate and reduce CPU usage. The embodiments of the present application can be applied to various scenarios such as cloud technology, AI, smart transportation, and vehicle-mounted.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, apparatus, computer equipment, and storage medium for processing content recommendation services. Background Art

[0002] With the development of terminal technology, terminals have become indispensable devices for people. Recommending content that users may need or like to their terminals based on their needs or preferences can improve user experience and exposure.

[0003] To achieve accurate recommendations, users are typically analyzed based on their data to extract object features. These features reflect their needs and preferences, and then content matching these features is determined. For example, by analyzing user data and extracting user characteristics such as female, 22 years old, and Jay Chou, the user's characteristics indicate that they like Jay Chou. Therefore, songs by Jay Chou can be recommended to the user in a music app.

[0004] Clearly, object features are key to achieving accurate content recommendations. The traditional approach is to store object features in a database and query them when content recommendations are triggered. However, object features are typically multi-dimensional, requiring multiple tables to be queried for a single user. If multiple users trigger content recommendations simultaneously, this results in high CPU usage and slow response times. Summary of the Invention

[0005] Based on this, it is necessary to provide a method, apparatus, computer equipment and storage medium for processing content recommendation services that can improve response speed in response to the above technical problems.

[0006] A method for processing a content recommendation service, characterized in that the method comprises:

[0007] Obtaining a content recommendation service request, wherein the content recommendation service request includes an object identifier;

[0008] searching for object features in a disk cache based on the object identifier;

[0009] When the object feature found in the disk cache based on the object identifier is a historical object feature, obtaining a disk cache hit rate, wherein after the object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature;

[0010] If the disk cache hit rate is lower than a threshold, obtaining the historical object feature corresponding to the object identifier from the disk cache;

[0011] Recommended content is determined based on the object features obtained from the disk cache.

[0012] A business data processing device, characterized in that the device comprises:

[0013] A service request acquisition module, configured to acquire a content recommendation service request, wherein the content recommendation service request includes an object identifier;

[0014] A search module, configured to search for object features in a disk cache based on the object identifier;

[0015] a hit rate acquisition module, configured to acquire a disk cache hit rate when an object feature found in the disk cache based on the object identifier is a historical object feature, wherein after the object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature;

[0016] a cache processing module, configured to obtain, from the disk cache, historical object features corresponding to the object identifier when the disk cache hit rate is lower than a threshold;

[0017] A recommendation module is used to determine recommended content based on object features obtained from the disk cache.

[0018] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0019] Obtaining a content recommendation service request, wherein the content recommendation service request includes an object identifier;

[0020] searching for object features in a disk cache based on the object identifier;

[0021] When the object feature found in the disk cache based on the object identifier is a historical object feature, obtaining a disk cache hit rate, wherein after the object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature;

[0022] If the disk cache hit rate is lower than a threshold, obtaining the historical object feature corresponding to the object identifier from the disk cache;

[0023] Recommended content is determined based on the object features obtained from the disk cache.

[0024] A computer-readable storage medium, characterized in that the storage medium stores a computer program, characterized in that when the processor executes the computer program, the following steps are implemented:

[0025] Obtaining a content recommendation service request, wherein the content recommendation service request includes an object identifier;

[0026] searching for object features in a disk cache based on the object identifier;

[0027] When the object feature found in the disk cache based on the object identifier is a historical object feature, obtaining a disk cache hit rate, wherein after the object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature;

[0028] If the disk cache hit rate is lower than a threshold, obtaining the historical object feature corresponding to the object identifier from the disk cache;

[0029] Recommended content is determined based on the object features obtained from the disk cache.

[0030] The above-mentioned method, apparatus, computer equipment and storage medium for processing content recommendation service data, when receiving a content recommendation service request, first searches for object features in the disk cache. If the object features found are historical object features, when the disk cache rate is lower than a threshold, the historical object features are forced to be used to determine recommended content for the user. On the one hand, the object features corresponding to the object identifier are directly obtained from the disk cache, eliminating the need for complex database read operations, thereby reducing CPU usage and load. Moreover, by utilizing the disk cache instead of the memory cache, the large capacity of the disk is effectively utilized, thereby improving the cache hit rate and response speed. On the other hand, when the disk cache hit rate is lower than a threshold, the data in the cache is forced to be used, thereby increasing the disk hit rate, reducing the CPU usage, and periodically updating the cache hit rate to ensure that new object features are slowly flushed into the cache. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A diagram illustrating an application environment of a method for processing content recommendation services in one embodiment;

[0032] Figure 2 A flowchart of a method for processing content recommendation services in one embodiment;

[0033] Figure 3 A schematic diagram of the system architecture of a method for processing content recommendation services according to an embodiment;

[0034] Figure 4 A schematic diagram of a cached data structure in another embodiment;

[0035] Figure 5 A flowchart of a method for processing content recommendation services in another embodiment;

[0036] Figure 6 A schematic diagram of multiple cache instances in parallel in one embodiment;

[0037] Figures 7 to 9 A schematic diagram of the effect of a method for processing a content recommendation service in one embodiment;

[0038] Figure 10 A schematic diagram of the structure of an apparatus for processing content recommendation services in one embodiment;

[0039] Figure 11 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0041] The method for processing content recommendation business provided by this application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the service server 104 via the network. The terminal 102 sends a service request to the service server 104 to trigger the recommendation service. The service server 104 obtains the content recommendation service request, which includes the object identifier. The response interface of the service server 104 responds to the content recommendation service request, extracts the object features corresponding to the object identifier in the cache or database, and sends the object features to the content recommendation system. The content recommendation system determines the recommended content based on the object features, and the response interface returns the recommended content to the terminal 102. As disclosed in the present application, the method for processing the content recommendation service, wherein the object feature data can be stored on the blockchain.

[0042] Specifically, the business server obtains a content recommendation business request, which includes an object identifier; searches for object features in the disk cache based on the object identifier; when the object features found in the disk cache based on the object identifier are historical object features, obtains a disk cache hit rate, wherein after the object features in the database are updated, the corresponding object features in the disk cache become historical object features; if the disk cache hit rate is lower than a threshold, obtains the historical object features corresponding to the object identifier from the disk cache; and determines recommended content based on the object features obtained from the disk cache.

[0043] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, vehicle-mounted terminals, tablet computers, and portable wearable devices, and the service server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0044] In one embodiment, Figure 2 As shown, the following steps are included:

[0045] Step S202: Obtain a content recommendation service request, where the content recommendation service request includes an object identifier.

[0046] In this embodiment, the content recommendation service covers business scenarios for recommending content to users, including but not limited to advertising recommendations, video recommendations, music recommendations, product recommendations, game recommendations, application recommendations, etc. The relevant business scenario is the scenario in which the user uses the relevant application. In the relevant application scenario, when the content recommendation is triggered, a content recommendation service request is obtained. Among them, the content recommendation service request is triggered when the user opens the application or refreshes the application page. The content recommendation service request includes an object identifier. The object identifier is the user's identification identifier, which can be the user ID of the user.

[0047] Step S204: searching for object features in the disk cache based on the object identifier.

[0048] For users, they usually use applications in a concentrated period of time to trigger content recommendation services, which makes the user's service requests to the content recommendation system have temporal locality. For example, users usually use applications to trigger content recommendations before eating, taking the subway, or going to bed, and stop using applications during work or school time. In other words, users usually use applications for a period of time, then stop using them, and then use them again after a period of time. Therefore, during the time when applications are used intensively, requests to the content recommendation system are repeated in a short period of time. For example, a user uses a news application while on the subway. During this usage period, the object features are not updated frequently. In other words, the temporal locality of service requests provides the basis for the use of cache.

[0049] Traditional caching mechanisms typically use memory, trading memory for CPU. However, for application servers with limited memory and large disk capacity, cache capacity is limited. For example, a server uses the TS80a model, which features a large SSD hard drive and supports random reads, but has a relatively limited memory capacity of only 64GB. This server's memory is primarily used for full-memory indexing, which can occupy over 20GB. Optimistically, assuming a 30GB cache and an average single object feature size of 10KB, data for a maximum of 3 million keys can be cached, which is relatively limited.

[0050] In this embodiment, the server is a disk-type storage, and many optimizations are made for disk IO. Therefore, the disk is creatively used as a cache, and disk IO is exchanged for memory and CPU to obtain several times the cache space, thereby increasing the cache hit rate.

[0051] When a content recommendation service request is made, object features are extracted from the database and the object features corresponding to the object identifier are written to the disk cache. If the object features in the disk cache are valid and the user triggers a content recommendation service request again, the corresponding object features are retrieved from the disk cache based on the object identifier.

[0052] S206 , when the object feature found in the disk cache based on the object identifier is a historical object feature, obtaining a disk cache hit rate, wherein after the object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature.

[0053] After an object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature. That is, the object feature in the database is updated, but the object feature in the disk cache is not. Specifically, the object features in the disk cache are derived from the database index file, which records the object features based on user data. When user data is updated, the object features in the database index file are also updated. However, the object features in the disk cache remain unchanged and become historical object features.

[0054] Specifically, a disk cache hit means that the object feature corresponding to the object identifier can be found in the disk cache, and the object feature is a valid object feature (not a historical object feature). A valid object feature means that the object feature in the disk cache is consistent with the object feature in the database index file, that is, the object feature in the database index file has not changed.

[0055] In real-world applications, object features are determined based on user data, which includes both user behavior data and attribute data. Attribute data typically refers to personal information, such as gender, education level, region, and age. Behavioral data typically refers to user application usage data, such as purchase history, browsing history, and usage time. By mining user behavior data, we can derive user usage habits and preferences. Therefore, object features are updated as user data changes.

[0056] In one embodiment, the update time of the user data is obtained, and the update time of the user data is compared with the time of the object feature recorded in the cache to determine whether the object feature in the cache is a historical object feature. When the cache time of the object feature recorded in the cache is later than the update time of the user data, it means that the object feature in the cache and the database index file comes from the user data, the object feature has not changed, the object feature in the cache is valid, and the object identifier can hit the cache. When the storage time of the object feature recorded in the cache is earlier than the update time of the user data, it means that the user data has been recently updated and the object feature in the database index file has changed. The object feature in the cache is invalid and is a historical object feature, which causes the object identifier to miss the cache.

[0057] Step S208: If the disk cache hit rate is lower than the threshold, the historical object features corresponding to the object identifier are obtained from the disk cache.

[0058] The cache hit rate is the ratio of the number of cache hit requests to the number of responses over a period of time. The threshold is a pre-set reference value. If the cache hit rate is below the threshold, it indicates a low cache hit rate. The content recommendation system relies on object features obtained from the database to make content recommendations, resulting in low cache utilization.

[0059] Generally speaking, when new user data is loaded, the object features in the database index file will be updated, and at this time, the corresponding object features in the cache file will become invalid. In a cache miss, the conventional processing is to use the data in the database for business processing. In this embodiment, in order to avoid large-scale cache failure caused by the loading of new batches of user data, causing the system to return to a high-load state without cache, an additional cache smoothing expiration mechanism is designed, that is, when batches of user data are updated, some of the requested caches will become invalid, and the cache hit rate will be reduced. If the cache hit rate is lower than the threshold, the object features in the cache are forced to be used, and the cache hit rate is updated periodically to ensure that new data is slowly flushed into the cache.

[0060] That is to say, when the object feature found in the disk cache based on the object identifier is a historical object feature, it indicates that the user data corresponding to the object identifier is updated, but the corresponding object feature found in the disk cache is not updated. When the disk cache hit rate is lower than the threshold, the data in the cache is forced to be used, thereby increasing the disk hit rate and reducing the CPU usage. The cache hit rate is updated periodically to ensure that the new object features are slowly flushed into the cache. For the content recommendation business, the real-time requirements are not high, and even if the historical object features are used to determine the recommended content, it will not affect the user experience. Therefore, forcing the use of historical object features is sufficient and effective for the content recommendation business.

[0061] In this embodiment, by setting a cache smoothing mechanism, the disk cache hit rate can be increased, the CPU usage can be reduced, and the minimum query volume per machine can be maintained at 100,000+ / second.

[0062] Step S210 : determining recommended content based on object features obtained from the disk cache.

[0063] Specifically, the content recommendation system determines recommended content for users based on historical object features. The content recommendation system implements content recommendation logic and determines matching recommended content for users based on object features. Specifically, the recommended content in the recommended content library is set with content tags. Based on the similarity between the object features and the content tags of the recommended content, target recommended content is determined for the user. The target recommended content is the recommended content whose content tags are similar to the object features. The similarity can be the cosine similarity between the object features and the content tags. If the object feature is a game, the target recommended content is determined to be the product to be recommended with the tag game. Specifically, a pre-trained content recommendation model can also be used, with object features as input, to predict the category of the target recommended content and determine the target recommended content from the content of that category. The recommended content is related to the business scenario and can be applied to different applications to implement advertising recommendations, video recommendations, music recommendations, product recommendations, game recommendations, application recommendations, etc.

[0064] The above method for processing content recommendation service data, when receiving a content recommendation service request, first searches for object features in the disk cache. If the object features found are historical object features, when the disk cache rate is lower than a threshold, the historical object features are forced to be used to determine recommended content for the user. On the one hand, the object features corresponding to the object identifier are directly obtained from the disk cache, without the need for complex database read operations, which reduces CPU usage and load. Moreover, by using the disk cache instead of the memory cache, the large capacity of the disk is effectively utilized, which improves the cache hit rate and response speed. On the other hand, when the disk cache hit rate is lower than the threshold, the data in the cache is forced to be used, thereby increasing the disk hit rate, reducing the CPU usage, and periodically updating the cache hit rate to ensure that new object features are slowly flushed into the cache.

[0065] In another embodiment, when the object features found in the disk cache based on the object identifier are not historical object features, the step of obtaining the object features corresponding to the object identifier from the disk cache is performed, and recommended content is determined for the user based on the object features obtained from the disk cache.

[0066] That is to say, when the object feature found in the disk cache based on the object identifier is not a historical user description feature, it means that the user data corresponding to the object identifier has not been updated. The object feature in the disk cache and the object feature in the database index file are both the latest object features, then the found object feature is obtained from the disk cache.

[0067] In another embodiment, the method for processing content recommendation services further includes: if the disk hit rate is greater than a threshold, querying the database index file corresponding to each feature dimension based on the object identifier to obtain the corresponding multi-dimensional object features, wherein the object features of the object identifier in the database index file are updated after the user data is updated; writing the mapping relationship between the object identifier and the object features into the disk cache, and determining the recommended content based on the object features obtained from the database.

[0068] Specifically, when the disk hit rate is greater than a threshold, it indicates high cache utilization. If the object features found in the disk cache based on the object identifier are historical object features, and the cache hit rate is greater than the threshold, it indicates high cache utilization. In this case, the object features in the database are used for content recommendation to update the content features in the cache.

[0069] In another embodiment, when the object features of the object are not found in the disk cache based on the object identifier, it indicates that content has not been recommended to the user and the object features of the user do not exist in the cache. Then, the object features in the database are used to make content recommendations to update the content features in the cache.

[0070] Among them, the architecture of the method for processing content recommendation business is as follows Figure 3 As shown, each database index file is queried according to the object identifier to obtain the object features of each dimension corresponding to the object identifier, wherein each database index file records the object features of one dimension, and the object features of the object identifier in the database index file are updated after the user data is updated; the object features of each dimension are preset processed to obtain multi-dimensional object features; the preset processing includes at least one of merging processing, filtering and compression processing.

[0071] Among them, object features come from user data, which includes user behavior data and user attribute data. Each object feature maintains a database index file. The database index file uses the object identifier as the key, and the corresponding value is the object feature of the dimension.

[0072] For example, object characteristics include dimensions such as gender, education level, age, and interests.

[0073] The database index file corresponding to each feature dimension is queried according to the object identifier, and after the query, decompression, deserialization, merging, filtering and serialization operations are performed to obtain the multi-dimensional object features of the user. Among them, the merging process refers to merging repeated object features. The filtering process is to filter one or more preset dimensions in the multi-dimensional object features. Multi-dimensional object features have multiple dimensions. Filtering the object features of dimensions that are not required for the content recommendation business can avoid useless object features from increasing the complexity of the calculation. Among them, the object features in the database are usually compressed, and the decompression, deserialization and serialization operations are to restore the compressed object features.

[0074] After the object features corresponding to the object identifier are read from the database, the object features are also written to the disk cache to update the cache.

[0075] In this embodiment, the object features found in the disk cache are historical object features. When the disk cache hit rate is low, or when the object features are not found in the disk cache, the object features are read from the database to implement the content recommendation service and the object features are updated to the cache. Specifically, updating to the cache involves writing the object features corresponding to the object identifier to the disk cache and establishing a cache index in the disk cache. The cache index records the storage parameters of the object features corresponding to the object identifier in the disk area. Therefore, when searching the disk cache, the object features of the user with the object identifier can be extracted from the corresponding disk area based on the storage parameters recorded in the cache index.

[0076] Specifically, based on a preset read-write control policy, the object features are written to the disk, and storage parameters of the object features corresponding to the object identifier in the disk area are recorded in the cache index, where the storage parameters include the storage location.

[0077] Specifically, the cached data structure is as follows Figure 4 As shown in the figure, when the cache is updated, data is first written to a cache area of ​​tens of MB in size. Data is written continuously in the cache area. Another cache index (key hash) is similar to a memory index, recording the cache block ID corresponding to the object identifier, as well as the location and size of the data in the block. When the cache area is full, it is flushed to disk, becoming an immutable file block. The disk can accommodate hundreds or thousands of file blocks. The process maintains a maximum number of file blocks. If the number of file blocks exceeds the maximum number, the oldest file block is deleted.

[0078] When reading from the cache, the cache index is searched based on the object ID. The cache index is a mapping table maintained by the cache module, using the object ID as the key and the single-dimensional object feature as the value. It records the location of the value corresponding to the key in the cache. The location is a triplet (cache area ID (block_id), cache position (offset), cache size (size)). When reading, the cache area ID, position, and size are located based on the cache index, and the data is read from the corresponding cache block or file block. Reading data from a file block is similar to reading a database index file on disk.

[0079] In this embodiment, when the object identifier misses the disk cache, the object features are read from the database and written into the disk cache to update the disk cache.

[0080] Correspondingly, based on the preset cache read-write control policy, object characteristics corresponding to the object identifier are obtained from the disk cache, including: obtaining storage parameters of the object characteristics corresponding to the object identifier in the disk area from the cache index; based on the preset cache read-write control policy and storage parameters, obtaining the object characteristics corresponding to the object identifier, wherein the object characteristics include valid object characteristics or historical object characteristics; the preset cache read-write control policy includes: the maximum amount of data written at one time is less than the maximum amount of data read at one time.

[0081] Specifically, when reading the cache, the cache index is checked according to the Key (the object identifier in this embodiment), and the storage parameters corresponding to the object characteristics are located, including the cache area id (block_id), the position in the cache (offset), and the cache size (size), and data can be read from the corresponding cache area.

[0082] Specifically, to improve the disk cache's read speed, this embodiment uses an SSD disk. An SSD, short for Solid State Disk, is a solid-state electronic storage array hard disk consisting of a control unit and a solid-state storage unit (Flash chip), offering fast read speeds. A cache read / write control policy is also implemented to improve the disk cache's read speed. Specifically, the preset cache read / write control policy includes ensuring that the maximum amount of data written at one time is less than the maximum amount of data read at one time.

[0083] Specifically, disk caches both read and write data, ultimately aiming to read data from the disk cache. If writing to the disk causes data jitter, this can delay data reads. To improve data read and write performance, the amount of data written to the disk during cache updates can be controlled. In this embodiment, the maximum amount of data written at a time is set to be smaller than the maximum amount of data read at a time, effectively providing more resources for reads. In this embodiment, by utilizing an SSD and a preset cache read and write control strategy, disk cache performance and cache processing speed can be improved.

[0084] In another embodiment, Figure 5 As shown, the method for processing content recommendation business includes:

[0085] S502: Obtain a content recommendation service request, where the content recommendation service request includes an object identifier.

[0086] S504: Check whether the object feature is found in the disk cache based on the object identifier. If so, execute step S506; if not, execute step S509.

[0087] If the result of step S504 is yes, it means that the object feature corresponding to the object identifier is found in the disk cache, that is, there is a history of content recommendations for the user. If the result of step S504 is no, it means that the object feature corresponding to the object identifier is not found in the disk cache, that is, there is no history of content recommendations for the user.

[0088] If the determination result of step S504 is yes, step S506 is further executed to determine whether the found object feature is a historical object feature.

[0089] If the judgment result of step S504 is no, step S509 is executed to obtain the object features corresponding to the object identifier from the database to perform content recommendation.

[0090] S506: Determine whether the object feature corresponding to the found object identifier is a historical object feature. If so, execute step S508; if not, execute step S510.

[0091] If the result of step S506 is yes, it means that the object features corresponding to the object identifier in the database index file are updated, and the object features in the disk cache are historical object features, and step S508 is executed. If the result of step S506 is no, it means that the object features in the disk cache and the object features in the database index file are both the latest object features, and step S510 is executed.

[0092] S508: Determine whether the disk cache hit rate is greater than a threshold. If not, execute step S510; if so, execute step S509.

[0093] The cache hit rate is the ratio of the number of cache hit requests to the number of responses over a period of time. The threshold is a pre-set reference value used to evaluate cache hit performance. If the cache hit rate is below the threshold, it indicates a low cache hit rate. The content recommendation system relies on user profile features obtained from the database to make content recommendations, resulting in low cache utilization.

[0094] S510: Obtain the found object features from the disk cache.

[0095] That is, when the object features found in the disk cache based on the object identifier are not historical object features, indicating that the user data corresponding to the object identifier has not been updated, the object features in the disk cache and the object features in the database index file are both the latest object features. Step S510 is executed to retrieve the found object features from the disk cache, thereby using the cached data. Alternatively, when the found object features are historical object features, meaning the object features in the cache have not been updated, while the object data in the database has been updated due to user data, if the disk cache hit rate is less than a threshold, step S510 is executed to retrieve the found object features from the disk cache, thereby forcing the use of the cached data.

[0096] After step S510 , step S512 is executed to determine recommended content based on the object features obtained from the disk cache.

[0097] If the judgment result of step S504 is no, that is, the object feature corresponding to the object identifier is not found in the disk cache, and if the judgment result of step S508 is yes, that is, the feature found in the disk cache based on the object identifier is a historical object feature, if the cache hit rate is greater than the threshold, execute S509 to use the object feature in the database.

[0098] S509: Search the database index file corresponding to each feature dimension according to the object identifier to obtain the corresponding multi-dimensional object features.

[0099] S511, writing the mapping relationship between the object identifier and the object feature into the disk cache.

[0100] And after S511, execute step S513.

[0101] S513: Determine recommended content based on the object features obtained from the database.

[0102] In this embodiment, by setting up a cache smoothing mechanism, the disk cache hit rate can be increased, the CPU usage can be reduced, and the minimum query volume per second on a single machine can be maintained at 100,000+. In another embodiment, searching the disk cache based on the object identifier includes: searching the disk cache for object features corresponding to the object identifier based on multiple cache instances; the number of cache instances is the same as the number of threads responding to content recommendation service requests.

[0103] Specifically, since the process responds to query requests in multiple threads, each query request has the possibility of updating the cache. In order to reduce the lock conflict problem when multiple threads update the cache, such as Figure 6As shown, multiple cache instances (cache_manager, CM) are designed, and the number of cache instances is consistent with the number of query threads. Since the number of threads updating the cache is increased, multiple threads update the cache in parallel, which can reduce the probability of lock conflicts when updating the cache and locking operations. Taking the object feature table as an example, if the number of query threads is 48, the number of cache instances is set to 48. Each cache instance maintains 128 blocks, including a mem_block (cache area) and a file_block (file block). The size of each block is set to 30MB, so the size of a single instance cache is close to 4GB. The total cache size of 48 instances is 192GB, far exceeding the system memory of 64GB.

[0104] In actual business, the method of processing content recommendation business of this application is used to achieve the following results: Figures 7 to 9 As shown, the CPU usage rate dropped from 96% to 50%, the cache hit rate was 90% (miss / hit=90%), and the single-machine QPS (query rate per second) exceeded 100,000 / second.

[0105] It should be understood that although Figure 2 、 Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 、 Figure 5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0106] In one embodiment, Figure 10 As shown, a device for processing content recommendation services is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes:

[0107] A service request acquisition module 1002 is configured to acquire a content recommendation service request, wherein the content recommendation service request includes an object identifier;

[0108] A search module 1004 is configured to search for object features in a disk cache based on the object identifier;

[0109] A hit rate acquisition module 1006 is configured to acquire a disk cache hit rate when the object feature found in the disk cache based on the object identifier is a historical object feature, wherein after the object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature;

[0110] A cache processing module 1008 is configured to obtain the historical object feature corresponding to the object identifier from the disk cache when the disk cache hit rate is lower than a threshold;

[0111] The recommendation module 1010 is configured to determine recommended content based on the object features obtained from the disk cache.

[0112] The above-mentioned device for processing content recommendation service data, when receiving a content recommendation service request, first searches for object features in the disk cache. If the object features found are historical object features, when the disk cache rate is lower than a threshold, the historical object features are forced to be used to determine recommended content for the user. On the one hand, the object features corresponding to the object identifier are directly obtained from the disk cache, without the need for complex database read operations, which reduces CPU usage and load. Moreover, by using the disk cache instead of the memory cache, the large capacity of the disk is effectively utilized, which improves the cache hit rate and improves the response speed. On the other hand, when the disk cache hit rate is lower than the threshold, the data in the cache is forced to be used, thereby increasing the disk hit rate, reducing the CPU usage, and periodically updating the cache hit rate to ensure that new object features are slowly flushed into the cache.

[0113] In another embodiment, further comprising:

[0114] The database processing module is used to query the database index file corresponding to each feature dimension according to the object identifier if the disk hit rate is greater than a threshold, and obtain the corresponding multi-dimensional object features, wherein the object features of the object identifier in the database index file are updated after the user data is updated.

[0115] The cache update module is used to write the mapping relationship between the object identifier and the object feature into the disk cache.

[0116] The recommendation module is configured to determine recommended content based on the object features obtained from the database.

[0117] In another embodiment, the cache processing module is configured to obtain the historical object feature corresponding to the object identifier from the disk cache when the object feature found in the disk cache based on the object identifier is not a historical object feature.

[0118] In another embodiment, the database processing module is further used to query the database index file corresponding to each feature dimension based on the object identifier to obtain the corresponding multi-dimensional object feature when the corresponding object feature is not found in the disk cache based on the object identifier.

[0119] In another embodiment, a cache processing module is used to obtain storage parameters of the object characteristics corresponding to the object identifier in the disk area from the cache index; based on a preset cache read-write control strategy and the storage parameters, obtain the object characteristics corresponding to the object identifier; wherein, the object characteristics include historical object characteristics; the preset cache read-write control strategy includes: the maximum amount of data written at one time is less than the maximum amount of data read at one time.

[0120] In another embodiment, a database processing module is used to query each database index file according to the object identifier to obtain object features of each dimension corresponding to the object identifier, wherein each database index file records the object features of one dimension; the object features of each dimension are preset processed to obtain multi-dimensional object features; the preset processing includes at least one of merging processing, filtering and compression processing.

[0121] In another embodiment, the cache processing module is used to search for the object feature corresponding to the object identifier in the disk cache based on multiple cache instances; the number of the cache instances is the same as the number of threads responding to the content recommendation service request.

[0122] For the specific definition of the device for processing content recommendation services, please refer to the definition of the method for processing content recommendation services above, which will not be repeated here. The various modules in the above-mentioned device for processing content recommendation services can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0123] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 11As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for object features. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for processing content recommendation services is implemented.

[0124] Those skilled in the art will understand that Figure 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0125] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0126] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0127] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0128] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0129] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for processing content recommendation service, characterized in that: The method comprises: Obtaining a content recommendation service request, wherein the content recommendation service request includes an object identifier; searching for object features in a disk cache based on the object identifier; When no object features are found in the disk cache based on the object identifier, content recommendation is performed using the object features in the database, and the object features in the disk cache are updated; When the object feature found in the disk cache based on the object identifier is a historical object feature, obtaining a disk cache hit rate, wherein after the object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature; If the disk cache hit rate is lower than a threshold, obtaining the object feature corresponding to the object identifier from the disk cache; determining recommended content based on the object features obtained from the disk cache; When the object feature found in the disk cache based on the object identifier is not a historical object feature, performing the step of obtaining the object feature corresponding to the object identifier from the disk cache; If the disk cache hit rate is greater than a threshold, query the database index file corresponding to each feature dimension according to the object identifier to obtain the corresponding multi-dimensional object features; Writing the mapping relationship between the object identifier and the object feature into a disk cache; Recommended content is determined based on the object features obtained from a database.

2. The method according to claim 1, characterized in that The object characteristics of the object identifier in the database index file are updated after the user data is updated.

3. The method according to claim 2, characterized in that The method further comprises: When the corresponding object feature is not found in the disk cache based on the object identifier, the step of querying the database index file corresponding to each feature dimension according to the object identifier to obtain the corresponding multi-dimensional object feature is performed.

4. The method according to claim 1, wherein The acquiring the object feature corresponding to the object identifier from the disk cache includes: Obtaining storage parameters of the object feature corresponding to the object identifier in the disk area from the cache index; Based on a preset cache read / write control policy and the storage parameters, object characteristics corresponding to the object identifier are obtained; wherein the object characteristics include historical object characteristics; the preset cache read / write control policy includes: a maximum amount of data written at one time is less than a maximum amount of data read at one time.

5. The method according to claim 2, characterized in that The step of querying the database index file corresponding to each feature dimension according to the object identifier to obtain the corresponding multi-dimensional object features includes: Querying each database index file according to the object identifier to obtain object features of each dimension corresponding to the object identifier, wherein each database index file records the object features of one dimension; Performing preset processing on the object features of each dimension to obtain multi-dimensional object features; the preset processing includes at least one of merging processing, filtering and compression processing.

6. The method according to claim 1, characterized in that The searching for the object feature in the disk cache based on the object identifier includes: The object feature corresponding to the object identifier is searched in the disk cache based on multiple cache instances; the number of the cache instances is the same as the number of threads responding to the content recommendation service request.

7. A device for processing content recommendation services, characterized in that: The device comprises: A service request acquisition module, configured to acquire a content recommendation service request, wherein the content recommendation service request includes an object identifier; A search module, configured to search for object features in a disk cache based on the object identifier; a database processing module configured to, when no object features are found in the disk cache based on the object identifier, recommend content using the object features in the database and update the object features in the disk cache; a hit rate acquisition module, configured to acquire a disk cache hit rate when an object feature found in the disk cache based on the object identifier is a historical object feature, wherein after the object feature in the database is updated, the corresponding object feature in the disk cache becomes a historical object feature; a cache processing module, configured to obtain the object feature corresponding to the object identifier from the disk cache when the disk cache hit rate is lower than a threshold; A recommendation module, configured to determine recommended content based on object features obtained from the disk cache; The cache processing module is further configured to, when the object feature found in the disk cache based on the object identifier is not a historical object feature, execute the step of acquiring the object feature corresponding to the object identifier from the disk cache; The database processing module is further configured to query the database index files corresponding to each feature dimension according to the object identifier to obtain the corresponding multi-dimensional object features if the disk cache hit rate is greater than a threshold; write the mapping relationship between the object identifier and the object features into the disk cache; and determine recommended content based on the object features obtained from the database.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, wherein the computer program implements the method according to any one of claims 1 to 6 when executed by a processor.

Images