Data pinning method, device, equipment and program product

Abstract

The application provides a data sticking method, device, equipment and program product; the method comprises the following steps: in response to a data sticking request of a target object routed by a data sticking node, obtaining an object identifier of the target object, an object label of the target object, a first sticking data list corresponding to the object label, and a bitmap of each first to-be-stuck data in the first sticking data list; determining a second sticking data list from the first sticking data list according to the bitmap and the object identifier; the second sticking data list comprises a plurality of second to-be-stuck data; determining a sticking score of each second to-be-stuck data; determining target sticking data from the plurality of second to-be-stuck data according to the sticking score; and sending the target sticking data to the data sticking node. Through the application, the accuracy of the target sticking data can be improved while reducing the cost of data sticking.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a data pinning method, apparatus, device, and program product. Background Technology

[0002] In streaming data systems with massive user bases, such as short video applications with hundreds of millions of users, Bloom filters relying on third-party distributed caching are typically used to avoid duplicate recommendations of top-ranked data. However, for services with hundreds of millions of daily active users and tens of billions of requests, this data ranking method based on third-party distributed caching has extremely high performance requirements and incurs high storage costs. Therefore, it is necessary to explore a more cost-effective data ranking method to effectively solve the problem of ranking frequently used data at a lower cost. Summary of the Invention

[0003] This application provides a data pinning method, apparatus, device, and program product that can improve the accuracy of the target pinned data while reducing the cost of data pinning.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a data pinning method, the method comprising: responding to a data pinning request of a target object routed by a data pinning node; obtaining an object identifier of the target object, an object tag of the target object, a first pinned data list corresponding to the object tag, and a bitmap of each first unpinned data in the first pinned data list; determining a second pinned data list from the first pinned data list based on the bitmap and the object identifier; the second pinned data list including a plurality of second unpinned data; determining a pinning score for each second unpinned data; determining a target pinned data from the plurality of second unpinned data based on the pinning score; and sending the target pinned data to the data pinning node.

[0006] This application provides a data pinning device, comprising: a first acquisition module, configured to, in response to a data pinning request of a target object routed by a data pinning node, acquire an object identifier of the target object, an object tag of the target object, a first pinned data list corresponding to the object tag, and a bitmap of each first pending pinned data in the first pinned data list; a first determination module, configured to, based on the bitmap and the object identifier, determine a second pinned data list from the first pinned data list; the second pinned data list includes a plurality of second pending pinned data; a second determination module, configured to, determine a pinning score for each of the second pending pinned data; a third determination module, configured to, based on the pinning score, determine a target pinned data from the plurality of second pending pinned data; and a sending module, configured to, send the target pinned data to the data pinning node.

[0007] In the above scheme, each point in the bitmap of each first data to be pinned corresponds one-to-one with the object identifier of each object in the bitmap; the first determining module is further configured to: determine the point status of the point corresponding to the object identifier in the bitmap; when the point status is not pinned, determine the first data to be pinned corresponding to the bitmap as the second data to be pinned, and obtain the second pinned data list.

[0008] In the above scheme, the second determining module is further configured to: for each second piece of data to be pinned, obtain the current timestamp, the number of times the second piece of data to be pinned has been pinned, the timestamp of the first pinning, the pinning weight, and the click-to-display ratio; the click-to-display ratio is the ratio between the number of clicks and the number of displays for the second piece of data to be pinned; and determine the pinning score of the second piece of data to be pinned based on the click-to-display ratio, the number of times the data has been pinned, the timestamp of the first pinning, the current timestamp, and the pinning weight.

[0009] In the above scheme, the third determining module is further used to: obtain a preset top score threshold; and determine the second data to be topped with a top score greater than the top score threshold as the target topped data.

[0010] In the above scheme, the device further includes an update module, which is used to: obtain the top-pin notification for the target top-pin data from the top-pin node route; in response to the top-pin notification, determine the target point corresponding to the target object in the bitmap of the target top-pin data; and update the point status of the target point from unpinned to pinned.

[0011] In the above scheme, the device further includes a first detection module, which is configured to: in response to a data tag detection request for multiple first top-ranked data in the first top-ranked data list, obtain the data tag of each first data to be top-ranked; for each first data to be top-ranked, detect the data tag to obtain a data tag detection result corresponding to the first data to be top-ranked; when the data tag detection result indicates that the data tag of the first data to be top-ranked has changed relative to the data tag corresponding to the first top-ranked data list, delete the bitmap of the first data to be top-ranked and delete the first data to be top-ranked from the first top-ranked data list.

[0012] In the above scheme, the device further includes a second detection module, which is used to: in response to an object tag detection request for all objects in the bitmap of the first data to be pinned, obtain the object tags of all objects; for each object, detect the object tag to obtain the object tag detection result of the object; when the object tag detection result indicates that the object tag of the object has changed relative to the object tags of other objects in the first data to be pinned, delete the point data of the object in each bitmap of the first data to be pinned.

[0013] In the above scheme, the device further includes a third detection module, which is used to: in response to a data status detection request for multiple first top-listed data in the first top-listed data list, obtain the data status of each first top-listed data; for each first top-listed data, detect the data status to obtain a data status detection result corresponding to the first top-listed data; when the data status detection result indicates that the data status of the first top-listed data has changed from an up-to-the-shelf status to an off-to-the-shelf status, delete the bitmap of the first top-listed data and delete the first top-listed data from the first top-listed data list.

[0014] This application provides a data pinning method, the method comprising: in response to a data pinning request of a target object, obtaining an object tag and an object identifier of the target object; determining a target service cluster based on a first hash value obtained by performing a first hash operation on the object tag; determining a target node from the target service cluster based on a second hash value obtained by performing a second hash operation on the object identifier; routing the data pinning request to the target node and receiving the target pinned data determined by the target node using the data pinning method; and performing pinning processing on the target pinned data.

[0015] This application provides a data pinning device, comprising: a second acquisition module, configured to acquire an object tag and an object identifier of the target object in response to a data pinning request of the target object; a fourth determination module, configured to determine a target service cluster based on a first hash value obtained by performing a first hash process on the object tag; a fifth determination module, configured to determine a target node from the target service cluster based on a second hash value obtained by performing a second hash process on the object identifier; a routing module, configured to route the data pinning request to the target node and receive the target pinned data determined by the target node using the above-described data pinning method; and a pinning processing module, configured to perform pinning processing on the target pinned data.

[0016] In the above scheme, the number of nodes in the target service cluster is multiple, and the number of objects corresponding to the data placement request is multiple; the fifth determining module is further configured to: when the number of objects is greater than the threshold of the number of objects processed by the target service cluster, horizontally expand the target service cluster to obtain an expanded service cluster; the number of nodes in the expanded service cluster is greater than the number of nodes in the target service cluster; and determine the target node from the expanded service cluster.

[0017] In the above scheme, the number of nodes in the target service cluster is multiple, and the number of objects corresponding to the data pinning request is multiple; the fifth determining module is further used to: when the number of data to be pinned is greater than the data quantity threshold processed in the target service cluster, split the target service cluster to obtain multiple sub-service clusters; and determine the target node from the multiple sub-service clusters.

[0018] This application provides an electronic device, including: a memory for storing computer-executable instructions; and a processor for implementing the data placement method provided in this application when executing the computer-executable instructions stored in the memory.

[0019] This application provides a computer-readable storage medium storing computer-executable instructions for implementing the data placement method provided in this application when executed by a processor.

[0020] This application provides a computer program product, which includes computer-executable instructions stored in a computer-readable storage medium. When the processor of an electronic device reads and executes the computer-executable instructions from the computer-readable storage medium, it implements the data placement method provided in this application.

[0021] The embodiments of this application have the following beneficial effects:

[0022] First, based on the object identifier of the target object and the bitmap of each first unpinned data in the first pinned data list corresponding to the object tag of the target object, a second pinned data list that has not been pinned for the target object can be quickly filtered out from the first pinned data list. Furthermore, by calculating the pinning score of each second pinned data in the second pinned data list, target pinned data with higher pinning priority can be quickly filtered out from the second pinned data list, improving the accuracy of target pinned data and thus enhancing the user experience. Moreover, the data pinning node routes the data pinning request of the target object to the target node in the target service cluster corresponding to the object tag by performing a hash calculation on the object tag and object identifier of the target object. This ensures that the subsequent determination of target pinned data relies only on the logical judgment and storage space of the target node, without relying on third-party distributed caching, greatly reducing the pinning cost of data pinning. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the data placement system architecture provided in the embodiments of this application;

[0024] Figure 2 This is an optional structural diagram of the data placement device provided in the embodiments of this application;

[0025] Figure 3 This is another optional structural diagram of the data placement device provided in the embodiments of this application.

[0026] Figure 4 This is an optional flowchart illustrating the data prioritization method provided in this application embodiment;

[0027] Figure 5 This is another optional flowchart illustrating the data placement method provided in the embodiments of this application;

[0028] Figure 6 This is another optional flowchart of the data placement method provided in the embodiments of this application;

[0029] Figure 7 This is a schematic diagram of an optional implementation process for determining a target node provided in an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of another optional implementation process for determining the target node provided in an embodiment of this application;

[0031] Figure 9 This is a schematic diagram of the implementation process for determining the second top-ranked data list provided in an embodiment of this application;

[0032] Figure 10This is a schematic diagram of the implementation process for determining the top score provided in the embodiments of this application;

[0033] Figure 11 This is a schematic diagram illustrating the implementation process of determining target top data provided in an embodiment of this application;

[0034] Figure 12 This is a schematic diagram of the short video recommendation page provided in an embodiment of this application;

[0035] Figure 13 This is a timing diagram of the short video stream service system provided in the embodiments of this application;

[0036] Figure 14 This is a schematic diagram illustrating the implementation of the routing data priority request provided in an embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments, but it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0039] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0040] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0041] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0042] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0043] In streaming data systems with massive user bases, such as short video applications with hundreds of millions of users, Bloom filters relying on third-party distributed caching are typically used to avoid duplicate recommendations of top-ranked data. However, for services with hundreds of millions of daily active users and tens of billions of requests, this data ranking method based on third-party distributed caching has extremely high performance requirements and incurs high storage costs. Therefore, it is necessary to explore a more cost-effective data ranking method to effectively solve the problem of ranking frequently used data at a lower cost.

[0044] To address at least one of the problems in the aforementioned related technologies, this application proposes a low-cost, feasible method for solving the problem of pinning streaming data. This method uses consistent hashing to distribute pinning requests from users belonging to the same category to a single machine node, and employs an inverted bitmap and a threshold for pinning scores to handle the pinning of streaming data. The distributed concurrent system is transformed into single-machine logic. A local bitmap is allocated to each video to be pinned, which identifies whether the video has been recommended to the current user. The bitmap data is stored in the machine node's storage space, resulting in very fast processing of pinning requests and significantly reducing the cost of pinning data. Furthermore, before each pinning request returns pinned data, the pinning score of the pinned data is locally checked to see if it exceeds the set pinning score threshold, thereby improving the accuracy of the target pinned data.

[0045] The following describes exemplary applications of the data prioritization device (i.e., electronic device) provided in this application embodiment. The data prioritization device provided in this application embodiment can be implemented as a node (i.e., a data prioritization node and a target node) in a data prioritization system. This node can be implemented as various types of user terminals capable of data prioritization, such as laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals. It can also be implemented as a server. The following will describe exemplary applications when the data prioritization device is implemented as a node in a data prioritization system, and when this node is implemented as a server.

[0046] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the data placement system 100 provided in this application embodiment. In order to support a data placement application, the terminal 400 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0047] Terminal 400 sends a data placement request to server 200. Server 200 constitutes the data placement device in this embodiment of the application. In response to a data placement request from a target object routed by a data placement node, server 200 obtains the object identifier of the target object, the object tag of the target object, a first list of placed data corresponding to the object tag, and a bitmap of each first data to be placed in the first list of placed data. Based on the bitmap and the object identifier, server 200 determines a second list of placed data from the first list of placed data. The second list of placed data includes multiple second data to be placed. Server 200 determines a placement score for each second data to be placed. Based on the placement score, server 200 determines a target data to be placed from the multiple second data to be placed. The target data to be placed is then returned to terminal 400, enabling terminal 400 to perform placement processing on the target data.

[0048] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 may be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0049] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 40 provided in the embodiment of this application. Figure 2 The illustrated electronic device 40 may be a data hosting device, which includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the data hosting device are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0050] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0051] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0052] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0053] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0054] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0055] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0056] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB); the presentation module 453 is used to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (e.g., a display screen, a speaker, etc.); the input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0057] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A data loading device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a first acquisition module 4551, a first determination module 4552, a second determination module 4553, a third determination module 4554, and a transmission module 4555. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0058] In other embodiments, the apparatus provided in this application can be implemented in software. Figure 3 A data placement device 456 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a second acquisition module 4561, a fourth determination module 4562, a fifth acquisition module 4563, a routing module 4564, and a placement processing module 4565. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0059] In some other embodiments, the apparatus provided in this application can be implemented in hardware. For example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data placement method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0060] In some embodiments, the terminal or server can implement the data placement method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0061] The data prioritization methods provided in the embodiments of this application can be executed by an electronic device, which can be a server or a terminal. That is, the data prioritization methods in the embodiments of this application can be executed by a server, by a terminal, or through interaction between a server and a terminal.

[0062] See Figure 4 , Figure 4 This is an optional flowchart illustrating the data prioritization method provided in this application embodiment, which will be combined with... Figure 4 The steps shown are explained below, taking the data placement method with the target node as the execution subject as an example. The target node is implemented as a server, and the method includes the following steps S101 to S105:

[0063] Step S101: In response to the data pinning request of the target object in the data pinning node routing, obtain the object identifier of the target object, the object tag of the target object, the first pinning data list corresponding to the object tag, and the bitmap of each first data to be pinned in the first pinning data list.

[0064] In this embodiment, the target object refers to the entity that initiates the data pinning request. For example, the target object can be a user or an entity operated by the user. A data pinning request is a request initiated by the target object to pin certain data (e.g., posts, products, messages, video data, or images) to the top of the screen. For example, in a short video platform, the target object can be user A. When user A enters the short video pinning page, a data pinning request for the short videos is automatically generated, requesting the short video platform to pin a series of short videos. An object tag refers to the category or attribute tag of the target object, which describes the characteristics or category of the target object. For example, the object tag of the target object can be "technology enthusiast" or "food expert," etc. An object identifier is a unique identifier for the target object, which can be used to distinguish different target objects. For example, the object identifier for user A is 98765, which uniquely identifies user A.

[0065] The first top-pinned data list is a list of data to be pinned corresponding to an object tag. This list includes multiple first-pinned data items, meaning all data under that object tag that needs to be pinned. This first-pinned data list can be pre-stored in the target node's storage space. For example, assuming user A's object tag is "technology enthusiast," the first-pinned data list could contain all short video data related to "technology" that needs to be pinned, such as short video A, short video B, and short video C. A bitmap is a data structure used to record whether each first-pinned data item in the first-pinned data list has been recommended to a user. This bitmap includes multiple points; for each point, the object identifier is used as the key, and the pinning status of the first-pinned data is used as the value. A value of 1 indicates that the point is pinned, and a value of 0 indicates that the point is not pinned. Suppose there are N users and M short videos, where N and M are both integers greater than 0. The bitmap structure is as follows: each data point to be pinned in the first pinned data list corresponds to a bitmap, and the length of the bitmap is N (i.e., the number of users). If short video j in the M short videos has been pinned for the i-th user among the N users, then the i-th bit in the bitmap of short video j is 1; otherwise, it is 0.

[0066] Step S102: Determine the second top data list from the first top data list based on the bitmap and object identifier.

[0067] In this embodiment of the application, the second top-pinned data list is a list of data to be pinned that is filtered from the first top-pinned data list based on the bitmap and object identifier of each first data to be pinned in the first top-pinned data list. The second top-pinned data list includes multiple second data to be pinned, and each second data to be pinned is data to be pinned that has not been pinned for the target object.

[0068] Suppose there are 3 users (user A, user B, and user C) and 3 short videos (short video A, short video B, and short video C). The bitmap of short video A is [1, 0, 1], which means that short video A has been pinned to the top for users A and C. For user B, short video A can be the second pinned data in the second pinned data list. The bitmap of short video B is [0, 1, 1], which means that short video B has been pinned to the top for users B and C. For user A, short video B can be the second pinned data in the second pinned data list. The bitmap of short video C is [1, 1, 0], which means that short video C has been pinned to the top for users A and B. For user C, short video C can be the second pinned data in the second pinned data list.

[0069] Step S103: Determine the top score for each second data to be topped.

[0070] In this embodiment of the application, the top placement score is a score used to evaluate the top placement priority of each second piece of data to be placed on top. The higher the top placement score, the higher the top placement priority of the corresponding second piece of data to be placed on top. The process for determining the top placement score of each second piece of data to be placed on top is described below.

[0071] Step S104: Determine the target top data from multiple second data to be topped based on the top score.

[0072] In this embodiment, the target top-ranking data is the data that ultimately needs to be ranked based on its ranking score from the second top-ranking data list. For example, continuing from the example in step S103, when short video A and short video B are the second data to be ranked, short video A has a ranking score of 90, and short video B has a ranking score of 40. Therefore, short video A has a higher ranking priority than short video B, and short video A can be determined as the target top-ranking data. The number of target top-ranking data can be at least one.

[0073] Step S105: Send the target top data to the data top node.

[0074] In this embodiment of the application, the data pinning node refers to the server node used to receive the target pinned data corresponding to the data pinning request and to perform pinning processing on the target pinned data.

[0075] The data pinning method provided in this application embodiment, based on the object identifier of the target object and the bitmap of each first unpinned data in the first pinned data list corresponding to the object tag of the target object, can quickly filter out a second pinned data list for the target object that has not been pinned. Furthermore, by calculating the pinning score of each second pinned data in the second pinned data list, it can quickly filter out target pinned data with higher pinning priority from the second pinned data list, improving the accuracy of target pinned data and thus enhancing the user experience.

[0076] See Figure 5 , Figure 5 This is another optional flowchart illustrating the data prioritization method provided in the embodiments of this application, which will be combined with... Figure 5 The steps shown are explained below, taking the execution subject of the data placement method as an example, where the data placement node is implemented as a server. The method includes the following steps S201 to S205:

[0077] Step S201: In response to the data pinning request of the target object, obtain the object tag and object identifier of the target object.

[0078] In this embodiment of the application, the data pinning request of the target object encapsulates the object tag and object identifier of the target object, and the data pinning request can be directly parsed to obtain the object tag and object identifier of the target object.

[0079] Step S202: Determine the target service cluster based on the first hash value obtained by performing a first hash process on the object label.

[0080] In this embodiment, the first hash processing is the process of hashing the object tag of the target object. The purpose of the first hash processing is to map the object tag of the target object to a hash value (i.e., the first hash value). The first hash value is the result of the first hash processing, and it is used to determine the target service cluster from multiple service clusters. For example, assuming there are 3 service clusters (numbered 0, 1, and 2), and user A's object tag is "technology enthusiast", the first hash value obtained by calculating using a hash function (e.g., MD5 or SHA-1) is 987654. The service cluster range mapped to this first hash value is 987654%3 = 1, so the target service cluster is service cluster 1, and user A's data priority request is routed to service cluster 1.

[0081] Step S203: Based on the second hash value obtained by performing a second hash process on the object identifier, determine the target node from the target service cluster.

[0082] In this embodiment, the second hash processing is the process of hashing the object identifier of the target object. The purpose of the second hash processing is to map the object identifier of the target object to a hash value (i.e., the second hash value). The second hash value is the result of the second hash processing, and this second hash value is used to determine the specific server node from the target service cluster. The target node is the specific server node in the target service cluster, which is used to process data pinning requests and determine the target pinned data for pinning processing. For example, assuming that service cluster 1 has 3 server nodes (numbered 0, 1, and 2), and user A's object identifier is user123, the second hash value obtained by calculating using a hash function (e.g., MD5 or SHA-1) is 123456. The range of server nodes mapped to this second hash value is 123456%3 = 0. Therefore, the target node is server node 0, and user A's data pinning request is routed to node 0.

[0083] Step S204: Route the data pinning request to the target node and receive the target pinning data sent by the target node.

[0084] Step S205: Perform top-pinning processing on the target top-pinned data.

[0085] In this embodiment, the pinning process is the operation of displaying target pinned data at the top of the user interface. A data pinning node can pin the target pinned data to the target object. Specifically, when the data pinning node is a terminal device, the terminal device can directly display the target pinned data to pin it to the target object. When the data pinning node is a server, the server can pin the target pinned data to the terminal devices associated with the target object, so that the target pinned data is displayed on the terminal devices associated with the target object.

[0086] In this embodiment, the data pinning node performs hash calculation on the object tag and object identifier of the target object, and routes the data pinning request of the target object to the target node of the target service cluster corresponding to the object tag. This makes the subsequent determination process of the target pinned data only rely on the logical judgment and storage space of the target node, without relying on third-party distributed cache, which greatly reduces the pinning cost of data pinning.

[0087] The data placement method in this application embodiment will be described below in conjunction with the interaction between the terminal and the server in the data placement system. It should be noted that the data placement method here is implemented through interaction between the terminal and the server, and is essentially the same as the data placement method executed by the server in the above embodiments. The only difference is that this application embodiment also describes the actions performed by the terminal during the execution of the data placement method. Furthermore, some steps can be executed by either the terminal or the server. Therefore, for steps in this embodiment that are the same as those in the above embodiments but have different execution subjects, this embodiment is merely illustrative. In the implementation process, any execution subject can perform the steps, and this application embodiment does not limit this.

[0088] Figure 6 This is another optional flowchart illustrating the data prioritization method provided in the embodiments of this application, such as... Figure 6 As shown, the method includes the following steps S301 to S312:

[0089] Step S301: The data pinning node receives the data pinning request from the target object.

[0090] In this embodiment of the application, a data pinning application can run on the data pinning node. The target object can perform a data pinning operation on the client of the data pinning application. The data pinning application can provide a data pinning function, and the target object can perform a data pinning operation on the data pinning function page to trigger a data pinning request.

[0091] In some embodiments, when the target object performs the data pinning operation, it can also simultaneously input its object tag and object identifier. When the data pinning node receives the object tag and object identifier, a confirmation window for data pinning will pop up on the data pinning function page. After the data pinning node detects that the user has clicked the confirm data pinning button, it further processes the object tag and object identifier to achieve data pinning. Alternatively, in other embodiments, the target object can directly input its object tag and object identifier on the data pinning function page. Upon receiving the object tag and object identifier, the data pinning node can directly trigger the data pinning function to further process the object tag and object identifier to achieve data pinning.

[0092] In some embodiments, a data pinning node generates a data pinning request in response to a data pinning operation. The data input for the target object can be encapsulated within the data pinning request. For example, in the display interface of the data pinning application, the object label and object identifier of the target object are displayed. The user can confirm the object label and object identifier of the target object on this display interface according to actual needs, and then the object label and object identifier of the target object input by the user can be encapsulated within the data pinning request.

[0093] In step S302, the data pinning node responds to the data pinning request of the target object and obtains the object tag and object identifier of the target object.

[0094] In this embodiment of the application, the specific meaning of obtaining the object tag and object identifier of the target object in response to the data pinning request of the target object can be found in the description of step S201 above, and will not be repeated here.

[0095] In some embodiments, in response to a data pinning request for a target object, if the data pinning request encapsulates the object tag and object identifier of the target object, the object tag and object identifier of the target object can be directly parsed to obtain them.

[0096] In step S303, the top data node determines the target service cluster based on the first hash value obtained by performing a first hash process on the object label.

[0097] In this embodiment of the application, the specific meaning and implementation of the step of determining the target service cluster based on the first hash value obtained by performing a first hash process on the object label can be found in the description of step S202 above, and will not be repeated here.

[0098] Step S304: The top data node determines the target node from the target service cluster based on the second hash value obtained by performing a second hash process on the object identifier.

[0099] In this embodiment of the application, the specific meaning and implementation of the step of determining the target node from the target service cluster based on the second hash value obtained by performing a second hash processing on the object identifier can be found in the description of step S203 above, and will not be repeated here.

[0100] In some embodiments, the number of nodes in the target service cluster is multiple, and the number of objects corresponding to the data placement request is multiple. See also Figure 7 , Figure 7 The step S304, in which the data-prioritizing node determines the target node from the target service cluster, can also be achieved through the following steps S3041 to S3042:

[0101] Step S3041: When the number of objects exceeds the threshold for the number of objects processed by the target service cluster, the target service cluster is horizontally scaled up to obtain the scaled-up service cluster.

[0102] In this embodiment, horizontal scaling is a strategy to increase the processing capacity of a target service cluster. Horizontal scaling is achieved by adding more nodes to the existing target service cluster. The number of nodes in the expanded service cluster is greater than the number of nodes in the target service cluster. The expanded service cluster refers to the service cluster whose size has been increased by adding new nodes to the existing service cluster. The expanded service cluster has more computing and storage resources, enabling it to handle more user requests and larger data volumes. The object number threshold refers to the upper limit of the number of objects that the target service cluster can efficiently process. When the number of objects exceeds this threshold, the performance of the target service cluster may degrade, leading to longer response times, excessive resource utilization, or even service interruption. Therefore, the object number threshold is a key indicator in system design used to trigger horizontal scaling operations. The object number threshold can be dynamically adjusted according to business needs and is not specifically limited here.

[0103] When the number of objects in the target object and the threshold number of objects processed by the target service cluster are known, and when the number of objects in the target object exceeds the threshold number of objects processed by the target service cluster, horizontal scaling of the target service cluster can be achieved by adding new nodes to the existing target service cluster.

[0104] Step S3042: Determine the target node from the expanded service cluster.

[0105] In this embodiment of the application, after obtaining the horizontally scaled service cluster, the target node can be determined from the scaled service cluster based on the second hash value obtained by performing a second hash processing on the object identifier of the target object.

[0106] Here, as the number of objects increases, when the number of objects exceeds the threshold of the number of objects that the target service cluster can process, the system capacity can be seamlessly expanded by horizontal scaling to extend low-cost server nodes, supporting the continuous growth of the business without relying on third-party storage.

[0107] In some embodiments, see Figure 8 , Figure 8 The step S304, in which the data-prioritizing node determines the target node from the target service cluster, can also be achieved through the following steps S3043 to S3044:

[0108] Step S3043: When the number of data to be prioritized is greater than the threshold of the number of data processed in the target service cluster, the target service cluster is split into multiple sub-service clusters.

[0109] In this embodiment, the data quantity threshold refers to the upper limit of the amount of pending data that the target service cluster can efficiently process. When the amount of pending data exceeds this threshold, the performance of the target service cluster may degrade, leading to increased processing latency, excessive resource utilization, or even service interruption. The data quantity threshold is a key indicator in system design, used to trigger service cluster splitting operations. Splitting refers to distributing data or tasks in the target service cluster to multiple sub-service clusters according to certain rules (e.g., data range or business logic). For example, this splitting operation can be horizontal splitting, distributing data in the target service cluster by row or by range to different sub-service clusters. A sub-service cluster refers to multiple independent service clusters obtained after splitting from the target service cluster. Each sub-service cluster is responsible for processing a portion of the data or tasks and can operate and be managed independently, thereby reducing the load pressure on a single service cluster.

[0110] When the number of data to be prioritized and the threshold number of data processed in the target service cluster are known, and when the number of data to be prioritized exceeds the threshold number of data processed in the target service cluster, the target service cluster is split into multiple independent sub-service clusters, so that the data in the target service cluster can be distributed to multiple sub-service clusters, thereby splitting the target service cluster.

[0111] Step S3044: Determine the target node from multiple sub-service clusters.

[0112] In this embodiment of the application, after obtaining multiple sub-service clusters, the target node can be determined from the multiple sub-service clusters based on the second hash value obtained by performing a second hash processing on the object identifier of the target object.

[0113] Here, after splitting, each sub-service cluster processes only a portion of the data, reducing the load on a single cluster. The splitting operation allows the system to scale to support larger data volumes. As the amount of data to be prioritized grows, sub-service clusters can be further split, allowing for seamless and low-cost scaling of system capacity without relying on third-party storage.

[0114] Step S305: The data pinning node routes the data pinning request to the target node.

[0115] In step S306, the target node responds to the data placement request of the target object routed by the data placement node, and obtains the object identifier of the target object, the object tag of the target object, the first placement data list corresponding to the object tag, and the bitmap of each first data to be placed in the first placement data list.

[0116] In this embodiment of the application, for the data pinning request of the target object in response to the data pinning node routing, the specific meaning and implementation of obtaining the object identifier of the target object, the object tag of the target object, the first pinning data list corresponding to the object tag, and the bitmap of each first data to be pinned in the first pinning data list can be found in the description of step S101 above, and will not be repeated here.

[0117] In step S307, the target node determines the second top-ranked data list from the first top-ranked data list based on the bitmap and object identifier.

[0118] In this embodiment, the second top-pinned data list includes multiple second top-pinned data items. For an explanation of the specific meaning of the step of determining the second top-pinned data list from the first top-pinned data list based on the bitmap and object identifier, please refer to the description of step S102 above; it will not be repeated here.

[0119] In some embodiments, each bit in the bitmap of each first piece of data to be prioritized corresponds one-to-one with the object identifier of each object in the bitmap. See also Figure 9 , Figure 9 The step S307, in which the target node determines the second top-ranked data list from the first top-ranked data list based on the bitmap and object identifier, can be achieved through the following steps S3071 to S3072:

[0120] Step S3071: Determine the point status of the point corresponding to the object identifier in the bitmap.

[0121] In this embodiment, a bitmap is a data structure that represents information using a series of binary bits. Each bit corresponds to an object identifier, and the value of the bit (usually 0 or 1) indicates the bit's state. For example, 0 indicates an unpinned state, and 1 indicates a pinned state. For each first piece of data to be pinned, the bitmap corresponding to the first piece of data to be pinned is retrieved from the preset storage space of the target node, and the bit state of the corresponding bit is checked based on the known object identifier of the target object.

[0122] Step S3072: When the point status is not pinned, the first data to be pinned corresponding to the bitmap is determined as the second data to be pinned, and the second pinned data list is obtained.

[0123] In this embodiment, the "not pinned" state indicates that the first data to be pinned has not been pinned for the target object, while the "pinned" state indicates that the first data to be pinned has been pinned for the target object. When the position of the point corresponding to the object identifier of the target object in the bitmap of the first data to be pinned is in the "not pinned" state, the first data to be pinned can be determined as the second data to be pinned. For each data to be pinned in the first data to be pinned list, based on the position of the point corresponding to the object identifier of the target object in the bitmap of each data to be pinned, it is determined whether the first data to be pinned has been pinned for the target object, thereby filtering out the second data to be pinned list whose position is in the "not pinned" state from the first data to be pinned list.

[0124] Here, by analyzing the bitmap, the first unpinned data to be pinned corresponding to the target object can be efficiently filtered out, effectively avoiding the problem of duplicate pinning. This way, only the unpinned data to be pinned needs to be processed later, avoiding unnecessary processing of the pinned data to be pinned, thus saving resources.

[0125] Step S308: The target node determines the top score for each second data to be placed on top.

[0126] In this embodiment of the application, the specific meaning of the step of determining the pinning score of each second data to be pinned can be explained in the description of step S103 above, and will not be repeated here.

[0127] In some embodiments, see Figure 10 , Figure 10 The step S308 shows that the target node determines the placement score for each second piece of data to be placed, which can be achieved through the following steps S3081 to S3082:

[0128] Step S3081: For each second piece of data to be pinned, obtain the current timestamp, the number of times the second piece of data to be pinned has been pinned, the timestamp of the first pinning, the pinning weight, and the point display ratio.

[0129] In this embodiment, the current timestamp refers to the timestamp of the current moment recorded. The number of times the second data to be pinned has been pinned refers to the number of times the second data to be pinned has been pinned by other objects besides the target object. The first pinning timestamp refers to the timestamp when the second data to be pinned was first pinned. Pinning weight refers to the weight value assigned to the second data to be pinned. The click-to-display ratio is the ratio between the number of clicks and the number of displays for the second data to be pinned. The number of clicks refers to the number of times other objects click on the second data to be pinned, and the number of displays refers to the number of times the second data to be pinned is displayed to other objects.

[0130] In some embodiments, the current timestamp can be obtained through system functions or API calls. The number of times the second piece of data to be pinned has been pinned can be pre-stored in storage and updated each time the second piece of data to be pinned is pinned. The first pinning timestamp can also be pre-stored in storage, recording the timestamp when the second piece of data to be pinned was first pinned. This pinning weight can be obtained based on data analysis, that is, dynamically adjusting the pinning weight according to historical data (e.g., click-through rate, play count, or user feedback). For example, the pinning weight of the second piece of data to be pinned with a high click-through rate is increased. The pinning weight of the second piece of data to be pinned with a low play count is decreased. This pinning weight can also be obtained based on a machine learning model, that is, using a machine learning model to predict the potential value of the second piece of data to be pinned and automatically assigning a pinning weight. For example, if the model predicts that a certain piece of data to be pinned has a high click-through rate, then a higher pinning weight is assigned.

[0131] Step S3082: Determine the placement score of the second data to be placed based on the point display ratio, the number of times it has been placed on top, the timestamp of the first placement, the current timestamp, and the placement weight.

[0132] In this embodiment of the application, the pinning score is used to characterize the likelihood of pinning the second data to be pinned. The larger the pinning score, the greater the likelihood of pinning the second data to be pinned; the smaller the pinning score, the less likely the second data to be pinned.

[0133] In some embodiments, it is necessary to calculate the reciprocal of the number of times the data has been pinned, the ratio between the first pinned timestamp and the current timestamp, and multiply the point display ratio, the reciprocal of the number of times the data has been pinned, the ratio between the first pinned timestamp and the current timestamp, and the pinning weight to obtain the pinning score of the second data to be pinned.

[0134] Here, by calculating the top score, the data to be topped can be dynamically evaluated and sorted to ensure that the most relevant data is displayed first. This allows the top score of the second data to be topped to be quickly determined from multiple second data to be topped, thus improving the accuracy of the target top data.

[0135] Step S309: The target node determines the target top data from multiple second data to be topped based on the top score.

[0136] In this embodiment of the application, the specific meaning of the step of determining the target top data from multiple second data to be topped based on the top score can be found in the description of step S104 above, and will not be repeated here.

[0137] In some embodiments, see Figure 11 , Figure 11The step S309, in which the target node determines the target top data from multiple second data to be topped based on the top score, can be achieved through the following steps S3091 to S3092:

[0138] Step S3091: Obtain the preset top score threshold.

[0139] In this embodiment, the top-ranking score threshold refers to a preset score limit, which can be used to determine which second pieces of data are eligible to become target top-ranking data. The preset top-ranking score threshold can be obtained in the following ways: it can be configured through the target node, meaning the top-ranking score threshold can be stored as a configuration parameter of the target node in a configuration file, database, or environment variable. It can be dynamically set, meaning the top-ranking score threshold can be manually set based on experience and business needs, or it can be dynamically adjusted based on real-time performance or object behavior. It can also be obtained through a machine learning model, meaning the top-ranking score threshold can be automatically recommended or calculated using a machine learning model based on historical data. The specific method for obtaining the preset top-ranking score threshold is not limited here.

[0140] Step S3092: The second data to be pinned with a pinning score greater than the pinning score threshold is determined as the target pinning data.

[0141] In this embodiment, the target top-ranking data is the final determined data to be ranked, and the ranking score of the target top-ranking data is greater than the ranking score threshold. After determining the ranking scores corresponding to multiple second data to be ranked, the second data to be ranked with ranking scores greater than the ranking score threshold are selected, and these second data to be ranked with ranking scores greater than the ranking score threshold are determined as the target top-ranking data. The number of target top-ranking data is at least one.

[0142] Here, by setting a top-ranking score threshold, it can be ensured that only the second piece of data to be ranked can be determined as the target top-ranking data for ranking processing, thus guaranteeing the data quality of the target top-ranking data and helping to improve the user experience.

[0143] Step S310: The target node sends the target top data to the data top node.

[0144] Step S311: The data top node receives the target top data sent by the target node.

[0145] Step S312: The data pinning node pins the target pinned data.

[0146] In this embodiment of the application, the specific meaning and implementation of the step of pinning the target pinned data can be found in the description of step S205 above, and will not be repeated here.

[0147] In some embodiments, after pinning the target pinned data, the data pinning node can generate a pinned notification for the target pinned data and route the pinned notification to the target node. The target node obtains the pinned notification routed by the data pinning node for the target pinned data; in response to the pinned notification, it determines the target point corresponding to the target object in the bitmap of the target pinned data; and updates the point status of the target point from unpinned to pinned.

[0148] In this embodiment, a "pinned" notification refers to a message notifying a target node that target pinned data has been pinned. This notification may contain information about the target pinned data, such as its data identifier or pinning time. In the bitmap, a target point refers to a specific location corresponding to a target object, and its value indicates its position. After the target pinned data is pinned, the position status of the target point corresponding to the target object in the bitmap is updated from unpinned to pinned, i.e., the target point's value is changed from 0 to 1.

[0149] Here, by sending a "pinned" notification, the pinned data node and the target node can synchronize their pinning status, ensuring that all parts of the data pinning system have a unified understanding of the pinning status of the target pinned data. The target node can respond to the pinning operation immediately and update the status of the target point to "pinned," thus reflecting the pinning status of the target pinned data in real time.

[0150] In some embodiments, the data tags of multiple first-top data items in the first-top data list can be detected at regular intervals to confirm whether the data tags have changed. In response to a data tag detection request for multiple first-top data items in the first-top data list, the data tag of each first-to-be-topped data item is obtained; for each first-to-be-topped data item, the data tag is detected to obtain the data tag detection result corresponding to the first-to-be-topped data item; when the data tag detection result indicates that the data tag of the first-to-be-topped data item has changed relative to the data tag corresponding to the first-top data list, the bitmap of the first-to-be-topped data item is deleted, and the first-to-be-topped data item is removed from the first-top data list.

[0151] In this embodiment, the data tag of the first data to be pinned refers to the tag attached to the first data to be pinned. This data tag is used to characterize the data category of the first data to be pinned. The data tag corresponding to the first pinned data list refers to the tag associated with the first pinned data in the first pinned data list. These tags are used to identify or classify the data in the list. Detection refers to the process of checking the data tags of the first data to be pinned to confirm whether they are consistent with the corresponding data tags in the first pinned data list. The data tag detection result refers to the output of the detection process, indicating whether the data tags of the first data to be pinned have been changed.

[0152] A data tag detection request is a signal or instruction that triggers the detection process, instructing the system to begin detecting whether data tags have changed. If a data tag has changed, it means that the first data to be pinned no longer meets the pinning criteria, therefore the bitmap of the first data to be pinned needs to be deleted. This may involve physically deleting the bitmap from storage or logically marking it. Similarly, if the first data to be pinned no longer meets the pinning criteria, it will be removed from the pinned data list to maintain the accuracy and up-to-date state of the list.

[0153] Here, by detecting changes in data tags, the system ensures that only valid and relevant data is processed, while data that no longer meets the requirements is discarded. Deleting irrelevant or changed data tags helps the system free up resources and avoid unnecessary processing and maintenance. Ensuring the accuracy of data tags improves the accuracy of data processing, thereby improving the overall system performance. Regular detection enables the system to respond promptly to changes in data tags, maintaining the timeliness and relevance of data. Regular detection and cleanup prevent the accumulation of outdated or inconsistent data, keeping the system clean and efficient. Automated detection and deletion processes reduce the need for manual intervention and improve the efficiency of system maintenance.

[0154] In some embodiments, the object labels of all objects in the bitmap of the first data to be pinned can be detected at regular intervals to confirm whether the object labels have changed. In response to an object label detection request for all objects in the bitmap of the first data to be pinned, the object labels of all objects are obtained; for each object, the object label is detected to obtain the object label detection result; when the object label detection result indicates that the object label of the object has changed relative to the object labels of other objects in the entire dataset, the point data of the object in each bitmap of the first data to be pinned is deleted.

[0155] In this embodiment, the object tag detection request is an instruction or signal used to trigger the process of detecting the object tags of all objects in the first bitmap to be pinned. The object tags of all objects refer to the set of tags possessed by all objects in the bitmap. Object tags are used to describe the attributes of objects, such as category, priority, or keywords. Detection refers to the process of verifying the object tags to confirm whether they have changed over time. The object tag detection result is the output of the detection process, indicating whether the object tags have changed and providing detailed information about the changes. Point data refers to the data in the bitmap representing a specific object at a corresponding point, typically 0 or 1, representing an unpinned state or a pinned state, respectively.

[0156] For each object, its object label is checked one by one to determine if it has changed. If the object label has changed and no longer meets the condition for being on top, the object's bit data can be deleted from all related bitmaps.

[0157] Here, regularly checking object labels helps ensure that the labels of all objects in the system remain consistent, thereby avoiding data processing errors caused by label confusion. Timely updating or deleting objects that no longer meet the criteria can improve data accuracy and reliability. Deleting unnecessary object location data can free up storage space and optimize resource utilization.

[0158] In some embodiments, the data status of multiple first-top data items in the first-top data list can be detected at regular intervals to confirm whether the data status has changed. In response to a data status detection request for multiple first-top data items in the first-top data list, the data status of each first-to-be-topped data item is obtained; for each first-to-be-topped data item, the data status is detected to obtain the data status detection result corresponding to the first-to-be-topped data item; when the data status detection result indicates that the data status of the first-to-be-topped data item has changed from an "on-demand" status to an "off-demand" status, the bitmap of the first-to-be-topped data item is deleted, and the first-to-be-topped data item is removed from the first-top data list.

[0159] In this embodiment, the data status detection request is a triggering action that instructs the system to begin checking the data status of the first list of data to be pinned. The data status of the first list of data to be pinned refers to its current state, which includes both "listed" and "unlisted" states. Detection is a process involving checking and evaluating the data status of the first list of data to be pinned to determine whether the data status has changed. The data status detection result is the output of the detection process, providing information about whether the status of the first list of data to be pinned has changed.

[0160] In some embodiments, the data status of each first piece of data to be pinned can be checked to determine whether it has changed from an "on" status to an "off" status. If the detection result shows that the data status has changed to an "off" status, the bitmap of that first piece of data to be pinned can be deleted from storage. Similarly, that first piece of data to be pinned will also be removed from the list of first pieces of data to be pinned.

[0161] Here, regularly checking data status ensures the system can reflect the latest data status in a timely manner, avoiding the risk of processing invalid or outdated data. By deleting data that is no longer needed, the system can better manage resources, including storage and computing resources. Removing invalid data reduces the system's burden, thereby improving overall performance.

[0162] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0163] This application provides a low-cost, feasible method for solving the problem of pinning streaming data. This method uses consistent hashing to distribute user data pinning requests to single machine nodes and employs an inverted bitmap and threshold methods to handle the streaming data pinning problem. This method can be applied to streaming-like data scenarios, such as short video streams or news streams. This application uses a short video stream scenario as an example for illustration. When a user enters a short video stream... Figure 12 When viewing the short video recommendation page, if a list of trending videos exists, consistent hashing routing is used to distribute user requests to the corresponding backend service nodes. These backend service nodes then select trending videos matching the user's tags from the list and pin them to the top of the page, saving this information. The first few trending videos played when a user first enters the short video recommendation page are these pinned videos.

[0164] This method can be applied to a short video streaming service system, which includes a terminal 130, a gateway layer service cluster 131, a top-of-view module 132, an operation configuration module 133, a tag module 134, and a video module 135, such as... Figure 13 As shown in the image. This method tags the pinned video content, and users with the same tag pin corresponding to a list of pinned videos with the same tag.

[0165] Data placement requests generated by terminal 130 are first uniformly sent to the gateway layer service cluster 131. The gateway layer service cluster 131 performs a hash operation on the user tag to obtain the hash value of the user tag, ensuring that data placement requests corresponding to the same user tag are routed to the same tag placement service cluster. Then, a hash operation is performed on the user identifier to obtain the hash value of the user identifier, ensuring that data placement requests corresponding to the same user identifier are uniformly routed to the same machine node within the same tag placement service cluster, such as... Figure 14 As shown. The goal is to minimize the number of users and videos processed by each machine node. When the number of users is too large, the machine nodes in the tag-based service cluster can be horizontally scaled up; when the amount of video data is too large, the tag-based service cluster can be horizontally split and scaled up.

[0166] Considering that the number of trending video content is far less than the number of users, two data structures were designed for local caching: a bitmap and a hashmap. Each video to be featured corresponds to a bitmap, where the user identifier is used as the key and 1 or 0 as the value. In the hashmap, the video identifier is used as the key and the video content as the value. All data is stored locally on the machine node, and logical judgments and data dependencies are handled entirely locally without relying on third-party services, thus significantly reducing performance overhead and latency. The machine node can also continuously monitor whether user tags or video tags in the tag module 134 have changed, and whether video content (i.e., upload / download status) in the video module 135 has changed. If changes are found, local data is cleaned up in real time, i.e., the inverted bitmap is cleared, and local storage capacity is released in real time.

[0167] Furthermore, before each data pinning request returns the pinned video, it is determined locally whether the pinning score of the pinned video exceeds a set threshold. The pinning score of the pinned video can be calculated using the following formula (1):

[0168] S = (a * 1 / number of times pinned * pinned timestamp / current timestamp * pinning weight) (1)

[0169] Where 'a' represents the click-to-display ratio of the pinned video, i.e., the ratio between the number of user clicks and the number of impressions, and its value is between [0.125, 0.75]. The pinning timestamp is the timestamp corresponding to when the pinned video was first pinned. S is the pinning score. When the pinning score is greater than the threshold set by the operation configuration module 133, the pinned video is pinned; otherwise, it is not pinned. The more times the pinned video has been pinned, the smaller the pinning score; the more the pinning timestamp / current timestamp moves forward over time, the smaller the pinning score; the greater the pinning weight of the pinned video, the greater the pinning score.

[0170] This application provides a low-cost, high-performance solution for prioritizing the recommendation of hot data in streaming data. It has no system bottlenecks, does not rely on third-party storage, and only requires the expansion of low-cost servers to realize a short video recommendation system with no capacity limit, and improves the read and write access performance of local memory cache.

[0171] It is understood that in the embodiments of this application, if the content involving user information, such as object tags, object identifiers, and the first top data list, involves data related to user information or enterprise information, when the embodiments of this application are applied to specific products or technologies, it is necessary to obtain user permission or consent, or to obfuscate this information to eliminate the correspondence between this information and the user; and the collection and processing of related data should strictly comply with the requirements of relevant national laws and regulations when applied in practice, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0172] The following description continues to illustrate the exemplary structure of the data placement device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the data placement device 455 of the memory 450 may include: a first acquisition module 4551, configured to acquire, in response to a data placement request of a target object routed by the data placement node, an object identifier of the target object, an object tag of the target object, a first placement data list corresponding to the object tag, and a bitmap of each first data to be placed in the first placement data list; a first determination module 4552, configured to determine a second placement data list from the first placement data list based on the bitmap and the object identifier; the second placement data list includes a plurality of second data to be placed; a second determination module 4553, configured to determine a placement score for each of the second data to be placed; a third determination module 4554, configured to determine a target placement data from the plurality of second data to be placed based on the placement score; and a sending module 4555, configured to send the target placement data to the data placement node.

[0173] In some embodiments, each point in the bitmap of each first data to be pinned corresponds one-to-one with the object identifier of each object in the bitmap; the first determining module 4552 is further configured to: determine the point status of the point corresponding to the object identifier in the bitmap; when the point status is not pinned, determine the first data to be pinned corresponding to the bitmap as the second data to be pinned, and obtain the second pinned data list.

[0174] In some embodiments, the second determining module 4553 is further configured to: for each of the second data to be pinned, obtain the current timestamp, the number of times the second data to be pinned has been pinned, the timestamp of the first pinning, the pinning weight, and the click-to-display ratio; the click-to-display ratio is the ratio between the number of clicks and the number of displays for the second data to be pinned; and determine the pinning score of the second data to be pinned based on the click-to-display ratio, the number of times the data has been pinned, the timestamp of the first pinning, the current timestamp, and the pinning weight.

[0175] In some embodiments, the third determining module 4554 is further configured to: obtain a preset top-ranking score threshold; and determine the second data to be ranked that has a top-ranking score greater than the top-ranking score threshold as the target top-ranking data.

[0176] In some embodiments, the apparatus 455 further includes an update module, which is configured to: obtain a notification of being pinned to the target pinned data from the data pinned node route; in response to the notification of being pinned to the target data, determine the target point corresponding to the target object in the bitmap of the target pinned data; and update the point status of the target point from unpinned to pinned.

[0177] In some embodiments, the apparatus 455 further includes a first detection module, the first detection module being configured to: in response to a data tag detection request for a plurality of first top-ranked data in the first top-ranked data list, acquire a data tag for each first data to be top-ranked; for each first data to be top-ranked, detect the data tag to obtain a data tag detection result corresponding to the first data to be top-ranked; when the data tag detection result indicates that the data tag of the first data to be top-ranked has changed relative to the data tag corresponding to the first top-ranked data list, delete the bitmap of the first data to be top-ranked and delete the first data to be top-ranked from the first top-ranked data list.

[0178] In some embodiments, the device 455 further includes a second detection module, the second detection module being configured to: in response to an object tag detection request for all objects in the bitmap of the first data to be pinned, obtain object tags for all objects; for each object, detect the object tag to obtain an object tag detection result for the object; when the object tag detection result indicates that the object tag of the object has changed relative to the object tags of other objects in the first data to be pinned, delete the point data of the object in each bitmap of the first data to be pinned.

[0179] In some embodiments, the device 455 further includes a third detection module, the third detection module being configured to: in response to a data status detection request for a plurality of first top-ranked data in the first top-ranked data list, acquire the data status of each first data to be top-ranked; for each first data to be top-ranked, detect the data status to obtain a data status detection result corresponding to the first data to be top-ranked; when the data status detection result indicates that the data status of the first data to be top-ranked has changed from an up-to-the-shelf status to an off-to-the-shelf status, delete the bitmap of the first data to be top-ranked and delete the first data to be top-ranked from the first top-ranked data list.

[0180] In other embodiments, such as Figure 3 As shown, the software modules stored in the data placement device 456 of the memory 450 may include: a second acquisition module 4561, used to acquire the object tag and object identifier of the target object in response to the data placement request of the target object; a fourth determination module 4562, used to determine the target service cluster based on the first hash value obtained by performing a first hash processing on the object tag; a fifth determination module 4563, used to determine the target node from the target service cluster based on the second hash value obtained by performing a second hash processing on the object identifier; a routing module 4564, used to route the data placement request to the target node and receive the target placement data determined by the target node using the data placement method; and a placement processing module 4565, used to perform placement processing on the target placement data.

[0181] In some embodiments, the number of nodes in the target service cluster is multiple, and the number of objects corresponding to the data placement request is multiple; the fifth determining module 4563 is further configured to: when the number of objects is greater than the threshold of the number of objects processed by the target service cluster, horizontally expand the target service cluster to obtain an expanded service cluster; the number of nodes in the expanded service cluster is greater than the number of nodes in the target service cluster; and determine the target node from the expanded service cluster.

[0182] In some embodiments, the number of nodes in the target service cluster is multiple, and the number of objects corresponding to the data pinning request is multiple; the fifth determining module 4563 is further configured to: when the number of data to be pinned is greater than the data quantity threshold processed in the target service cluster, split the target service cluster to obtain multiple sub-service clusters; and determine the target node from the multiple sub-service clusters.

[0183] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For technical details not disclosed in this apparatus embodiment, please refer to the description of the method embodiment of this application for understanding.

[0184] This application provides a computer-readable storage medium storing computer-executable instructions. When these instructions are executed by a processor, they cause the processor to execute the data loading method provided in this application, for example... Figure 4 The method for pinning data is shown.

[0185] This application provides a computer program product including computer-executable instructions stored in a computer-readable storage medium. An electronic device's processor reads the computer-executable instructions from the computer-readable storage medium and executes the instructions, causing the electronic device to perform the data loading method described in this application.

[0186] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0187] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0188] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0189] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0190] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for prioritizing data, characterized in that, The method includes: In response to the data pinning request of the target object in the data pinning node routing, the object identifier of the target object, the object tag of the target object, the first pinned data list corresponding to the object tag, and the bitmap of each first data to be pinned in the first pinned data list are obtained. Based on the bitmap and the object identifier, a second top-pinned data list is determined from the first top-pinned data list; the second top-pinned data list includes multiple second data items to be topped. Determine the placement score for each of the second data items to be placed; Based on the pinning score, determine the target pinning data from the plurality of second data to be pinned; The target top data is sent to the data top node.

2. The method according to claim 1, characterized in that, Each point in the bitmap of the first data to be pinned corresponds one-to-one with the object identifier of each object in the bitmap; determining the second pinned data list from the first pinned data list based on the bitmap and the object identifier includes: Determine the position status of the point corresponding to the object identifier in the bitmap; When the point is in an unpinned state, the first data to be pinned corresponding to the bitmap is determined as the second data to be pinned, thus obtaining the second pinned data list.

3. The method according to claim 2, characterized in that, Determining the placement score for each of the second data to be placed includes: For each of the second data to be pinned, obtain the current timestamp, the number of times the second data to be pinned has been pinned, the timestamp of the first pinning, the pinning weight, and the click-to-display ratio; the click-to-display ratio is the ratio between the number of clicks and the number of displays for the second data to be pinned. The placement score of the second data to be placed is determined based on the point display ratio, the number of times it has been placed to the top, the timestamp of the first placement, the current timestamp, and the placement weight.

4. The method according to claim 3, characterized in that, The step of determining the target top-ranked data from the plurality of second data to be top-ranked based on the top-ranking score includes: Get the preset top score threshold; The second piece of data to be pinned, whose pinning score is greater than the pinning score threshold, is determined as the target pinning data.

5. The method according to any one of claims 1 to 4, characterized in that, After sending the target top data to the data top node, the method further includes: Obtain the pinned notification for the target pinned data from the pinned node route; In response to the pinned notification, determine the target point position corresponding to the target object in the bitmap of the target pinned data; Update the target point's position status from unpinned to pinned.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In response to a data tag detection request for multiple first top data in the first top data list, obtain the data tag for each first data to be topped; For each of the first data to be pinned, the data tag is detected to obtain the data tag detection result corresponding to the first data to be pinned. When the data tag detection result indicates that the data tag of the first data to be pinned has changed relative to the data tag corresponding to the first pinned data list, the bitmap of the first data to be pinned is deleted, and the first data to be pinned is removed from the first pinned data list.

7. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In response to an object tag detection request for all objects in the bitmap of the first data to be pinned, obtain the object tags of all objects; For each object, the object label is detected to obtain the object label detection result of the object; When the object label detection result indicates that the object label of the object has changed relative to the object labels of other objects in all objects, delete the point data of the object in the bitmap of each of the first data to be placed on top.

8. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In response to a data status detection request for multiple first top data in the first top data list, the data status of each first data to be topped is obtained; For each of the first data to be pinned, the data status is detected to obtain the data status detection result corresponding to the first data to be pinned. When the data status detection result indicates that the data status of the first data to be pinned has changed from the "on-shelf" status to the "off-shelf" status, the bitmap of the first data to be pinned is deleted, and the first data to be pinned is removed from the list of the first pinned data.

9. A method for prioritizing data, characterized in that, The method includes: In response to a data pinning request for the target object, obtain the object tag and object identifier of the target object; The target service cluster is determined based on the first hash value obtained by performing a first hash process on the object label. The target node is determined from the target service cluster based on the second hash value obtained by performing a second hash process on the object identifier; The data pinning request is routed to the target node, and the target pinned data determined by the target node using the data pinning method according to any one of claims 1 to 8 is received; The target data is pinned to the top.

10. The method according to claim 9, characterized in that, The target service cluster contains multiple nodes, and the target object corresponding to the data pinning request contains multiple objects; Determining the target node from the target service cluster includes: When the number of objects exceeds the threshold for the number of objects processed by the target service cluster, the target service cluster is horizontally scaled up to obtain an expanded service cluster; the number of nodes in the expanded service cluster is greater than the number of nodes in the target service cluster. The target node is determined from the expanded service cluster.

11. The method according to claim 9, characterized in that, Determining the target node from the target service cluster includes: When the number of data to be prioritized exceeds the threshold of the number of data processed in the target service cluster, the target service cluster is split into multiple sub-service clusters. The target node is determined from the plurality of sub-service clusters.

12. A data prioritization device, characterized in that, include: The first acquisition module is used to respond to the data pinning request of the target object in the data pinning node routing, and acquire the object identifier of the target object, the object tag of the target object, the first pinning data list corresponding to the object tag, and the bitmap of each first data to be pinned in the first pinning data list; The first determining module is used to determine the second top data list from the first top data list based on the bitmap and the object identifier; The second list of top-pinned data includes multiple second data items to be pinned. The second determining module is used to determine the top score for each of the second data to be topped; The third determining module is used to determine the target top data from the plurality of second data to be topped based on the top score; The sending module is used to send the target top data to the data top node.

13. A data prioritization device, characterized in that, include: The second acquisition module is used to acquire the object tag and object identifier of the target object in response to the data pinning request of the target object; The fourth determining module is used to determine the target service cluster based on the first hash value obtained by performing a first hash process on the object label; The fifth determining module is used to determine the target node from the target service cluster based on the second hash value obtained by performing a second hash process on the object identifier; A routing module is used to route the data pinning request to the target node and receive the target pinned data determined by the target node using the data pinning method according to any one of claims 1 to 8; The top-pinning processing module is used to pin the target top-pinning data.

14. An electronic device, characterized in that, include: Memory is used to store executable instructions for a computer; The processor, when executing computer-executable instructions stored in the memory, implements the data occupancy method according to any one of claims 1 to 8, or implements the data occupancy method according to any one of claims 9 to 11.

15. A computer program product, characterized in that, The computer program product includes computer-executable instructions, which are stored in a computer-readable storage medium. When the processor of the electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, it implements the data placement method according to any one of claims 1 to 8, or implements the data placement method according to any one of claims 9 to 11.

Images