Data processing method and device, storage medium, equipment and program product

Abstract

The invention discloses a data processing method and device, a storage medium, equipment and a program product. The method comprises the following steps: receiving playing behavior data from a plurality of terminals; for each piece of playing behavior data, generating a main index key for uniquely identifying the playing behavior data; corresponding auxiliary index keys are generated for at least one screenable attribute dimension contained in the playing behavior data, and an associated main index key is stored in the value of each auxiliary index key; in response to a playing record query request initiated by a client, if the playing record query request contains a screening condition for a target screenable attribute dimension, performing retrieval based on a target auxiliary index key corresponding to the target screenable attribute dimension to obtain at least one associated main index key, and obtaining corresponding target playing behavior data based on the at least one main index key. According to the method, efficient retrieval and flexible query of the playing behavior data are realized through a cooperation mechanism of the main index key and the auxiliary index key.

Description

Technical Field

[0001] This application relates to the field of game technology, specifically to a data processing method, apparatus, storage medium, device, and program product. Background Technology

[0002] With the rapid development of multimedia content platforms, the scale and complexity of user playback behavior data are growing exponentially. Traditional technologies exhibit significant performance bottlenecks when dealing with massive amounts of playback behavior data. Traditional systems typically employ a single index structure, which cannot effectively support multi-dimensional filtering queries, resulting in excessively long response times in query scenarios with filtering conditions. Traditional solutions often create a complete data copy for each query dimension, leading to significant waste of storage space and increased system maintenance costs. When new query dimensions need to be added, traditional architectures require large-scale data reconstruction, severely impacting system availability and maintenance efficiency. Summary of the Invention

[0003] This application provides a data processing method, apparatus, storage medium, device, and program product, which achieves efficient retrieval and flexible querying of playback behavior data through a collaborative mechanism of primary index key and secondary index key.

[0004] On one hand, embodiments of this application provide a data processing method, the method comprising: Receive playback behavior data from multiple terminals; For each of the playback behavior data, a primary index key is generated to uniquely identify the playback behavior data; For each of the at least one filterable attribute dimension contained in the playback behavior data, a corresponding auxiliary index key is generated, and the value of each auxiliary index key is stored in relation to the associated primary index key. In response to a playback history query request initiated by the client, if the playback history query request includes filtering conditions for the target filterable attribute dimension, then a search is performed based on the target auxiliary index key corresponding to the target filterable attribute dimension to obtain at least one associated primary index key, and the corresponding target playback behavior data is obtained based on the at least one primary index key.

[0005] On the other hand, embodiments of this application provide a data processing apparatus, the apparatus comprising: The receiving unit is used to receive playback behavior data from multiple terminals; The first generation unit is used to generate a primary index key for each playback behavior data to uniquely identify the playback behavior data. The second generation unit is used to generate corresponding auxiliary index keys for at least one filterable attribute dimension contained in the playback behavior data, and the value of each auxiliary index key stores the associated primary index key. The processing unit is configured to respond to a playback history query request initiated by the client. If the playback history query request includes filtering conditions for a target filterable attribute dimension, it performs a retrieval based on the target auxiliary index key corresponding to the target filterable attribute dimension to obtain at least one associated primary index key, and obtains the corresponding target playback behavior data based on the at least one primary index key.

[0006] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the data processing method as described in any of the above embodiments.

[0007] On the other hand, embodiments of this application provide a computer device, the computer device including a processor and a memory, the memory storing a computer program, the processor executing the data processing method as described in any of the above embodiments by calling the computer program stored in the memory.

[0008] On the other hand, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the data processing method as described in any of the above embodiments.

[0009] The data processing method provided in this application embodiment receives playback behavior data from multiple terminals; for each playback behavior data, a primary index key is generated to uniquely identify the playback behavior data; for at least one filterable attribute dimension included in the playback behavior data, corresponding secondary index keys are generated, and the value of each secondary index key stores the associated primary index key; in response to a playback record query request initiated by a client, if the playback record query request includes filtering conditions for a target filterable attribute dimension, a retrieval is performed based on the target secondary index key corresponding to the filtering conditions to obtain at least one associated primary index key, and the corresponding target playback behavior data is obtained based on at least one primary index key. This application embodiment, by generating a primary index key for playback behavior data and generating secondary index keys for filterable attribute dimensions and establishing a relationship between the two, enables efficient retrieval and flexible querying of playback behavior data when a client initiates a playback record query request with filtering conditions, by retrieving the associated primary index key based on the target secondary index key. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating an application scenario of the data processing system provided in an embodiment of this application.

[0012] Figure 2 This is a flowchart illustrating the data processing method provided in an embodiment of this application.

[0013] Figure 3 This is a schematic diagram of a first application scenario of the data processing method provided in the embodiments of this application.

[0014] Figure 4 This is a schematic diagram of a second application scenario for the data processing method provided in the embodiments of this application.

[0015] Figure 5 This is a schematic diagram of a third application scenario of the data processing method provided in the embodiments of this application.

[0016] Figure 6 This is a schematic diagram of a fourth application scenario of the data processing method provided in the embodiments of this application.

[0017] Figure 7 This is a schematic diagram of the fifth application scenario of the data processing method provided in the embodiments of this application.

[0018] Figure 8 This is another schematic diagram of the data processing method provided in the embodiments of this application.

[0019] Figure 9 This is another schematic flowchart of the data processing method provided in the embodiments of this application.

[0020] Figure 10 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application.

[0021] Figure 11 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] This application provides a data processing method, apparatus, storage medium, device, and program product. Specifically, the data processing method of this application can be executed by a computer device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, smart TV, wearable smart device, smart vehicle terminal, etc. The terminal can also include a client, which can be an audio / video client, browser client, instant messaging client, or mini-program, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0024] It should be noted that, in this embodiment, the entity executing the data processing method can be a terminal device or a server. The terminal device can be a local terminal device or a client device in the aforementioned cloud gaming. This embodiment does not limit the type of the entity executing the method.

[0025] It is understood that in the specific implementation of this application, data related to user playback behavior and playback record query requests are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0026] For example, in conjunction with the above description, Figure 1 This application illustrates a data processing system 1000 for implementing a data processing method, as provided in an embodiment of this application. The data processing system 1000 may include at least one terminal 1001, at least one server 1002, at least one database 1003, and a network. The user-held terminal 1001 can connect to different servers via the network. The terminal is any device with computing hardware capable of supporting and executing software applications corresponding to the game.

[0027] In the aforementioned data processing system 1000, terminal 1001 is used to install and run audio and video applications. In some cases, terminal 1001 may not need to have the audio and video applications pre-installed; users can directly access the audio and video content platform through clients such as browsers. When a user logs into the audio and video application, terminal 1001 sends a login request to server 1002. Server 1002 verifies the user's account and determines the corresponding interaction mechanism based on the login request. If verification is successful, server 1002 returns a login success notification to terminal 1001. During the user's content interaction through the audio and video application, terminal 1001 and server 1002 exchange data. Terminal 1001 sends various information to server 1002. Server 1002 determines the display data for terminal 1001 based on the stored interaction mechanism and the received information, and sends the display data back to terminal 1001 so that terminal 1001 can display the data sent by server 1002 to the user.

[0028] Furthermore, when the data processing system 1000 includes multiple terminals, multiple servers, and multiple networks, different terminals can connect to each other through different networks and servers. The network can be a wireless network or a wired network, such as a wireless local area network (WLAN), local area network (LAN), cellular network, 2G network, 3G network, 4G network, 5G network, etc. Additionally, different terminals can also connect to other terminals or servers using their own Bluetooth networks or hotspot networks. Furthermore, the system 100 can include multiple databases coupled to different servers, and can continuously store information related to the audio and video content platform in the databases while different users are playing multi-user games online.

[0029] It should be noted that, Figure 1 The schematic diagram of the data processing system shown is merely an example. The data processing system 1000 described in this application embodiment is intended to more clearly illustrate the technical solutions of this application embodiment and does not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of data processing systems and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0030] Please see Figures 2 to 9 , Figure 2 , Figure 8 and Figure 9 These are all schematic flowcharts of the data processing methods provided in the embodiments of this application. Figures 3 to 7This is a schematic diagram illustrating an application scenario of the data processing method provided in this application embodiment. It should be noted that the steps shown may be executed in a logical order different from that shown in the flowchart. The method may include the following steps: Step 110: Receive playback behavior data from multiple terminals.

[0031] In some embodiments, the plurality of terminals includes at least two of the following types of devices: mobile terminal, personal computer, tablet device, smart TV, and in-vehicle device. For example, the plurality of terminals encompasses a wide variety of device types, including but not limited to: Mobile terminals, such as smartphones, are highly portable and flexible in usage scenarios, allowing users to play audio and video content anytime, anywhere through various audio and video applications; there are also audio and video playback terminals in wearable devices such as smartwatches. Personal computers, including desktops and laptops, typically have larger screens and more powerful processing capabilities, making them suitable for playing complex content for extended periods of time, and users may use them in office or entertainment scenarios. Tablet devices, which combine portability with a large screen display, are often used for audio and video playback in leisure and entertainment scenarios. Smart TVs are characterized by large screens and high-quality audio-visual experiences. They are mostly used in family living rooms and other settings to watch various movies, TV shows and other content. With the development of intelligent vehicles, in-vehicle entertainment systems have become an important terminal, allowing users to play music, audiobooks, and other content while driving.

[0032] These different types of terminals reflect users' audio and video playback needs in different scenarios, and receiving playback behavior data from them can comprehensively cover users' diverse usage habits and scenarios.

[0033] In some embodiments, the playback behavior data is a record of the user's playback behavior on an audio and video content platform.

[0034] In some embodiments, the playback behavior data includes the following information: user identifier, device identifier, resource identifier, playback start time, playback end time, playback duration, playback completion status, and resource type.

[0035] User ID (uid): A unique identifier for each user, used to distinguish the playback behavior of different users. It is a core dimension for data aggregation and index building.

[0036] Device ID: A unique identifier for the source device of the playback, used to track playback behavior on a specific device, and is essential for implementing "filtering by device" (e.g., ...). Figure 4 This is the direct basis for the application scenarios shown.

[0037] Resource identifiers (e.g., artistid, songid, etc.): Unique strings or numbers that identify the audio or video resource being played, i.e., identifiers of the content being played, such as artist ID, song ID, etc., facilitating the statistical analysis of the playback status of specific resources, and enabling "search by song or artist" (e.g., Figure 5 or Figure 6 This is the direct basis for the application scenarios shown.

[0038] Time information (playback start time, end time): The playback start time records the specific time when the user begins playing the audio / video resource; the playback end time records the specific time when the user ends playing the audio / video resource. This time information is used to calculate playback duration, determine valid playback, and serves as the source for constructing reverse timestamps to achieve time-based sorting (e.g., ...). Figure 3 (Application scenarios shown) and filtering by day (such as) Figure 7 (The application scenarios shown).

[0039] Playback duration: The actual duration of audio and video resources played by the user, calculated based on the start and end times. Playback duration is used to determine valid playback; for example, only records with a playback duration exceeding 30 seconds will be processed further to filter invalid clicks, improve data quality, and increase storage efficiency.

[0040] Playback completion status: Indicates whether the user has fully played the audio and video resources, such as "Playback completed" or "Playback incomplete".

[0041] Resource type: Identifies the type of audio and video resource being played, such as music, video, short video, audiobook, etc., which facilitates the classification, statistics and analysis of the playback of different types of resources.

[0042] Step 120: For each playback behavior data, generate a primary index key to uniquely identify the playback behavior data.

[0043] In some embodiments, the primary index key includes: a hash prefix obtained by hashing the user identifier, the user identifier, the data status identifier, the device identifier, the resource identifier, and a behavior timestamp in reverse order.

[0044] In some embodiments, generating a primary index key to uniquely identify the playback behavior data includes: performing a hash operation on the user identifier, taking the first M bits of the result as a hash prefix, where M is a positive integer; concatenating the hash prefix, user identifier, data status identifier, device identifier, resource identifier, and reverse-ordered behavior timestamps to generate the primary index key, wherein the data status identifier is used to identify the validity of the playback behavior data.

[0045] The hash operation uses the CRC32 algorithm, with M set to 8, making the hash prefix 8 bytes long.

[0046] The hash prefix is ​​used to determine the storage partition of the primary index key in the distributed database, so that the data of different users can be evenly distributed across multiple storage nodes.

[0047] For example, a primary index key mainly includes the following components: Hash prefix: Obtained by hashing the user identifier (uid). The hash operation can use the CRC32 algorithm, a widely used cyclic redundancy check algorithm that generates a fixed-length hash value. For example, the first M bits of the hash result (e.g., M=8) can be used as the hash prefix, making the hash prefix M bytes long. The purpose of the hash prefix is ​​to determine the storage partition of the primary index key in a distributed database (such as HBase), ensuring that data from different users is evenly distributed across multiple storage nodes, thereby improving the system's parallel processing capability and scalability.

[0048] User ID (uid): A unique identifier for a user, and an important component of the primary index key, used to distinguish the playback behavior of different users.

[0049] Data status identifiers: These are used to identify the validity of playback behavior data. For example, "1" can represent valid data, and "0" can represent invalid data. The introduction of data status identifiers helps to quickly filter out valid data (i.e., valid playback behavior data) during subsequent data processing, improving data processing efficiency.

[0050] Device ID: A unique identifier for the source device of playback, used to track playback behavior on a specific device. Combining the device ID with the user ID allows for more precise identification of the source of playback behavior.

[0051] Resource Identifier: This consists of a combination of a content resource identifier (such as artistid, songid, etc.) and a reporting time identifier. The content resource identifier identifies the audio or video resource being played, while the reporting time identifier (especially a millisecond-level timestamp) distinguishes multiple reports from the same user to the same content resource within a very short period. If the same user reports playback behavior data for the same content resource multiple times within the same millisecond-level time frame, these multiple reports of playback behavior data will be treated as the same playback behavior data when generating the primary index key to avoid data duplication and redundancy.

[0052] Reverse-order timestamps: The timestamps of playback actions are sorted in reverse chronological order and used as part of the primary index key. Reverse-order timestamps facilitate storing and retrieving data in reverse chronological order in the database, making it easier to retrieve the latest playback action data records.

[0053] The process of generating the primary index key is as follows: First, perform a hash operation on the user identifier (uid), for example, using the CRC32 algorithm. Take the first M bits (e.g., M=8) of the result as the hash prefix, i.e., the hash prefix is ​​M bytes long. Then, concatenate the hash prefix, user identifier (uid), data status identifier, device identifier (deviceid), resource identifier (including content resource identifier and reporting time identifier), and the reverse-ordered timestamp to generate the complete primary index key. During concatenation, a certain order and format can be followed to ensure the uniqueness and readability of the primary index key. For example, the format "hash prefix_user identifier_data status identifier_device identifier_resource identifier_reverse-order timestamp" can be used.

[0054] The primary index key serves as a unique identifier for playback behavior data, playing a crucial role in subsequent data aggregation, storage, and retrieval. It allows for quick location of specific playback behavior data records, enabling efficient data retrieval and statistical analysis. The primary index key design also considers the storage characteristics of distributed databases, achieving uniform data distribution and load balancing through hash prefixes, thus improving the overall system performance and scalability.

[0055] In some embodiments, the resource identifier includes a combination of a content resource identifier and a reporting time identifier, used to distinguish multiple reports of playback behavior data of the same content resource by the same user within a millisecond time period. The method further includes: if the same user reports playback behavior data of the same content resource multiple times within the same millisecond time period, then when generating the primary index key, the multiple reported playback behavior data will be treated as the same playback behavior data.

[0056] To address potential rowkey conflicts arising from multiple submissions of the same content resource (such as a song) by the same user within the same millisecond under high concurrency, the resource identifier is designed as a combination of "content resource identifier (such as songId) + submission time identifier (such as a sequence within milliseconds)," effectively "salting" the data. If the same user submits the same content resource multiple times within the same millisecond, it is considered a single playback action, generating only one primary index key to avoid data duplication and redundancy, thus ensuring data accuracy for business applications.

[0057] Step 130: For at least one filterable attribute dimension included in the playback behavior data, generate corresponding auxiliary index keys respectively, and store the value of each auxiliary index key with the associated primary index key.

[0058] Traditional solutions typically use a coarse-grained key of "user ID (uid) + song ID (songId) + time interval" for deduplication, which makes it impossible to distinguish playback devices, query multiple playbacks of the same song, or support flexible searches by artist or other dimensions. Step 130 completely breaks this limitation by creating an independent auxiliary index for each filterable dimension.

[0059] The auxiliary index key pre-builds ordered index entries for each query dimension (such as device, song, artist), transforming the computational complexity (traversal, filtering) of traditional technology into the storage complexity (storage space occupation) and reading complexity (efficient scanning) of this application embodiment, so that even when facing trillions of data, millisecond-level multidimensional filtering can still be achieved.

[0060] In some embodiments, the filterable attribute dimensions include at least one of the following: resource identifier dimension, resource creator identifier dimension, and playback device identifier dimension.

[0061] In some embodiments, the method further includes: establishing an association between the secondary index key and the primary index key corresponding to each filterable attribute dimension by storing the primary index key as the value of the secondary index key in the database; wherein each secondary index key includes a user identifier, the identifier information of the filterable attribute dimension, and behavior time information arranged in reverse order.

[0062] Among them, the filterable attribute dimensions include at least one of the following: resource identifier dimension, resource creator identifier dimension, and playback device identifier dimension.

[0063] Resource identifier dimension (e.g., songId): The resource identifier is used to uniquely identify the specific audio or video resource being played by the user. Generating a secondary index key using this dimension facilitates quick subsequent queries about the playback status of specific resources, such as those corresponding to "filter by single track" (e.g.,...). Figure 5 (The application scenarios shown) and song search.

[0064] Resource creator identifier dimension (e.g., artistId): The resource creator identifier is associated with the individual or team that created the audio or video content. For example, it corresponds to "search by artist dimension" (e.g., Figure 6 (The application scenarios shown).

[0065] Playback device identifier dimension (e.g., deviceId): The playback device identifier clarifies the type of device from which the playback action occurs. For example, it corresponds to "filtering by device source" (e.g., ...). Figure 4 (The application scenarios shown).

[0066] Each secondary index key consists of a user identifier (uid), identifiers for filterable attribute dimensions (such as artistId / deviceId / songId), and reverse-ordered behavior time information. For example, the user identifier identifies the user to whom the data belongs; the identifiers for filterable attribute dimensions specify the corresponding filter dimension for the secondary index key; and the reverse-ordered behavior time information facilitates data retrieval over time series, such as finding playback behavior data for a specific attribute dimension within a given time period. This structure ensures that the secondary index key contains crucial filter information while also facilitating time-based queries. The reverse-order time information in the secondary index key remains consistent with the primary index key, ensuring that results retrieved via the secondary index, after obtaining complete data by the primary key, are sorted in reverse chronological order by default, conforming to users' habit of viewing recent historical data.

[0067] This approach establishes a relationship between the primary index key and the primary index key by storing the primary index key as the secondary index key in the database. For example, the "value" corresponding to the secondary index key in the database (such as HBase) does not store specific playback behavior data, but rather the associated "primary index key (primary rowkey)". When data queries need to be performed based on a filterable attribute dimension, such as querying playback data for all works by a specific creator, the associated primary index key can be quickly retrieved using the creator's secondary index key. Detailed playback behavior data can then be obtained based on the primary index key, significantly improving the efficiency of data retrieval. The association between the secondary and primary index keys ensures data integrity, allowing playback behavior data from different dimensions to be organically combined, facilitating multi-dimensional data querying and analysis.

[0068] Step 140: In response to the playback history query request initiated by the client, if the playback history query request includes filtering conditions for the target filterable attribute dimension, then a search is performed based on the target auxiliary index key corresponding to the target filterable attribute dimension to obtain at least one associated primary index key, and the corresponding target playback behavior data is obtained based on the at least one primary index key.

[0069] In some embodiments, the step of retrieving at least one associated primary index key based on the target auxiliary index key corresponding to the target filterable attribute dimension includes: determining the scanning range of the target auxiliary index key based on the filtering conditions; and scanning the database for target auxiliary index keys that satisfy the scanning range to obtain at least one associated primary index key value.

[0070] For example, when responding to a playback history query request initiated by any client, if the request includes filtering conditions for a target filterable attribute dimension, the system first searches based on the target secondary index key corresponding to that filterable attribute dimension. Specifically, the scanning range of the target secondary index key is determined based on the filtering conditions. For example, if the filtering condition is a specific resource creator identifier, then the relevant range of the secondary index key corresponding to that creator identifier in the database is determined. Subsequently, the database is scanned for target secondary index keys that meet this scanning range. Once a matching secondary index key is found, at least one associated primary index key value can be obtained. Then, based on these obtained primary index keys, the corresponding target playback behavior data is retrieved from the database, thereby satisfying the client's query requirements. This process fully utilizes the association between secondary index keys and primary index keys, enabling rapid location of the required data and improving query efficiency.

[0071] In some embodiments, the method further includes: in response to a playback record query request containing pagination parameters, determining a query range based on the sorting rules of the primary index key; and using a batch query method to retrieve no more than N playback behavior data records per query, where N is the preset pagination size indicated by the pagination parameters.

[0072] For example, in practical applications, a playback history query request initiated by the client may include pagination parameters. When responding to a playback history query request containing pagination parameters, the query range is determined based on the sorting rules of the primary index key. The sorting rules of the primary index key are usually related to the time attributes of the data, such as sorting by the chronological order of playback actions (ascending or descending). Based on the preset page size N indicated by the pagination parameters, a batch query method is used, retrieving no more than N playback action data entries per query. This avoids the problems of excessive system load and long response times caused by retrieving a large amount of data at once, resulting in smoother data display and an improved user experience.

[0073] In some embodiments, the batch query method includes: initiating a batch query request to the database based on multiple primary index keys to obtain multiple corresponding playback behavior data.

[0074] For example, when using batch queries, a single batch query request is sent to the database based on multiple previously obtained primary index keys to retrieve the corresponding multiple playback behavior data entries. Compared to traditional one-by-one queries, batch queries reduce the number of interactions with the database, lowering network transmission and database processing overhead. For instance, if 100 playback behavior data entries need to be retrieved, the traditional method might require 100 interactions with the database, while batch queries might only require one or a few interactions, significantly improving data retrieval efficiency, especially when handling large amounts of data queries.

[0075] In some embodiments, the method further includes: if the playback record query request does not contain filtering conditions for the target filterable attribute dimension, then performing data retrieval based on the sorting rules of the primary index key to obtain the target playback behavior data.

[0076] For example, when a client initiates a playback history query request without specifying any particular filterable attribute dimensions (such as resource identifier, resource creator identifier, playback device identifier, etc.), a data retrieval process based on the primary index key sorting rule will be triggered. For instance, if a user simply wants to view their recent playback history without explicitly requesting filtering for specific resources or creators, the system will use this method for data retrieval and display. Figure 3 The application scenario shown displays all playback behavior data in chronological order, including song information, playback source device, and time information. The primary index key contains rich information, among which the reverse-ordered behavior timestamps are the key basis for data retrieval. Behavior timestamps record the specific time when the playback behavior occurred, and their reverse order means that recent playback behavior data can be prioritized during data storage and retrieval. This design aligns with users' usual habit of focusing on recent playback records and also facilitates quick location of data within the relevant time period during data retrieval.

[0077] In some embodiments, the data retrieval based on the sorting rules of the primary index key includes: performing a range scan or prefix matching scan based on the reverse-ordered behavior timestamps contained in the primary index key to obtain target playback behavior data.

[0078] Among them, based on the reverse-ordered behavior timestamps in the primary index key, the target playback behavior data is mainly obtained by range scanning or prefix matching scanning.

[0079] Range scanning: This feature allows scanning based on a user-specified time range. For example, if a user wants to view playback history from the past week, the system will calculate the time point from one week ago based on the current time, then scan the database for the primary index key of the behavior timestamp within that time range to retrieve the corresponding playback behavior data. Range scanning can accurately retrieve data within a specific time period, satisfying users' needs for time-range queries.

[0080] Prefix matching scan: If the user doesn't explicitly specify a time range but wants to retrieve the latest playback records, prefix matching scan can be used. Since behavior timestamps are arranged in reverse chronological order, the prefix part corresponds to a relatively later time. Prefix matching scan can quickly retrieve playback behavior data within a recent period, such as the latest 10 playback records. This method efficiently provides recent data that the user might be interested in when the user doesn't have a specific time range requirement.

[0081] When a playback history query request does not contain specific filtering conditions, the target playback behavior data can be obtained flexibly and efficiently by using range scanning or prefix matching scanning based on the primary index key sorting rules, thus meeting the diverse query needs of users.

[0082] In some embodiments, the method further includes: determining whether the playback behavior data is valid playback data, wherein playback behavior data with a playback duration exceeding a preset threshold is determined to be valid playback behavior data; The step of generating a primary index key to uniquely identify each playback behavior data includes: generating a primary index key to uniquely identify each valid playback behavior data. The step of generating corresponding auxiliary index keys for at least one filterable attribute dimension contained in the playback behavior data includes: generating corresponding auxiliary index keys for at least one filterable attribute dimension contained in the valid playback behavior data.

[0083] In the data processing flow, the validity of playback behavior data is determined, and subsequent index keys are generated based on valid data. At the same time, abnormal data is processed to ensure data quality and the accuracy of subsequent data applications.

[0084] The key to determining whether playback behavior data is valid lies in whether the playback duration exceeds a preset threshold. For example, if the preset threshold is set to 30 seconds, only playback behavior data with a duration exceeding 30 seconds will be considered valid. This standard is based on considerations of the user's actual viewing or listening behavior. Shorter playback durations may indicate that the user only triggered playback accidentally and is not genuinely interested in the content, while playback exceeding a certain duration better reflects the user's attention to and consumption of the content. Therefore, it is considered valid data for subsequent more valuable data analysis and application.

[0085] For each valid playback behavior data point, a primary index key is generated to uniquely identify it. The primary index key includes information such as a hash prefix obtained by hashing the user identifier, the user identifier, the data status identifier (in which case the data status is valid), the device identifier, the resource identifier, and the behavior timestamp in reverse order. By generating the primary index key, a unique identifier is established for each valid playback behavior data point, facilitating accurate storage and retrieval in the database.

[0086] Specifically, for each of the at least one filterable attribute dimension contained in the valid playback behavior data, a corresponding secondary index key is generated. Filterable attribute dimensions include resource identifier, resource creator identifier, and playback device identifier. The value of the secondary index key is stored in relation to the associated primary index key. This establishes a relationship between the secondary and primary index keys, supporting flexible data queries based on different attribute dimensions.

[0087] If the playback record query request includes a time range filter condition, then when performing data retrieval based on the primary index key or the secondary index key, the time range condition is encoded into the retrieval key or used as the scanning range condition for the retrieval.

[0088] This process involves batch processing and analyzing stored playback behavior data using offline computing tasks to identify and remove abnormal playback behavior data, and updating the mapping relationship between the corresponding primary and secondary index keys. Abnormal data may include invalid or erroneous playback behavior data generated due to system failures, malicious click fraud, or other reasons. After identifying abnormal playback behavior data, it is removed from the dataset, and the mapping relationship between the corresponding primary and secondary index keys is updated. This is because the presence of abnormal data can affect the accuracy and reliability of data analysis. For example, when calculating the playback volume of popular content, abnormal data may lead to statistical results that are too high or too low. By performing offline batch processing analysis and updating the index key mapping relationship, the consistency and accuracy of the data can be guaranteed, ensuring that subsequent queries and analyses based on this data yield reliable results.

[0089] In some embodiments, the method further includes: storing the playback behavior data and its corresponding primary index key and secondary index key into a first storage table via a real-time processing link; performing anomaly detection and correction on the playback behavior data in the first storage table via an offline processing link, generating corrected playback behavior data and storing it into a second storage table; setting a version identifier for each playback behavior data to indicate whether each playback behavior data has been corrected by the offline processing link, the version identifier including a first identifier and a second identifier, the first identifier corresponding to the first storage table and the second identifier corresponding to the second storage table; and determining whether to read data from the first storage table or the second storage table based on the version identifier in response to the playback record query request.

[0090] In the process of processing playback behavior data, a combination of real-time and offline processing links is used, and version identifiers are introduced to manage data storage and retrieval. At the same time, abnormal data is processed, which can effectively ensure the accuracy and reliability of the data.

[0091] Real-time processing pipeline: Playback behavior data and its corresponding primary and secondary index keys are stored in the first storage table. This pipeline can quickly respond to the reporting of playback behavior data, storing it promptly to ensure data timeliness. For example, when a user plays audio or video, the playback behavior data can be quickly entered into the first storage table (real-time table) through the real-time processing pipeline, providing a data foundation for subsequent real-time queries and analysis.

[0092] Offline processing pipeline: This pipeline performs anomaly detection and correction on the playback behavior data in the first storage table, generates corrected playback behavior data, and stores it in the second storage table (the offline table). The offline processing pipeline typically runs during relatively idle periods of the system, such as starting an offline task on T+1 (the next day). It can perform complex analysis and processing on large amounts of data. It is not limited by real-time requirements, enabling a more comprehensive and in-depth examination of anomalies in the data and corresponding corrections, thereby improving data quality.

[0093] Each playback behavior data point is assigned a version identifier to indicate whether it has been corrected by the offline processing link. The version identifier includes a first identifier and a second identifier; the first identifier corresponds to a first storage table, and the second identifier corresponds to a second storage table. When responding to a playback record query request, the version identifier determines whether data should be read from the first or second storage table. If the data carries the first identifier, it indicates that the data exists only in the real-time table or has not yet been corrected by the offline processing link, and is read from the first storage table. If it carries the second identifier, it indicates that the data has been processed by the offline link, and a more accurate version already exists in the offline table, and is read from the second storage table. This ensures that the queried data has been processed and optimized, improving data accuracy and reliability.

[0094] In some embodiments, the step of detecting and correcting anomalies in the playback behavior data in the first storage table through an offline processing link includes: associating the first storage table with a risk control data table through an offline processing link to identify abnormal playback behavior data in the first storage table; marking or correcting the abnormal playback behavior data in the first storage table to generate a corrected playback behavior dataset.

[0095] The offline processing mechanism links the first storage table with the risk control data table to identify abnormal playback behavior data. The risk control data table contains various known anomaly patterns and risk information. By linking it with the first storage table, it can quickly compare and analyze whether playback behavior data contains anomalies. For example, if a user's playback behavior is found to frequently switch between different devices within a short period with extremely short playback durations, matching the malicious traffic-boosting pattern in the risk control data table, this can be identified as abnormal playback behavior data. The identified abnormal playback behavior data is then marked or corrected, generating a corrected playback behavior dataset. Marking allows subsequent operations to identify this abnormal data, while correction directly adjusts the data to conform to the characteristics of normal data.

[0096] In some embodiments, the method further includes: after storing the corrected playback behavior dataset in the second storage table, synchronously deleting the corresponding abnormal playback behavior data in the first storage table, and deleting the primary index key and secondary index key related to the abnormal playback behavior data.

[0097] Specifically, after storing the corrected playback behavior dataset in the second storage table, the corresponding abnormal playback behavior data in the first storage table is simultaneously deleted, along with the primary and secondary index keys related to the abnormal playback behavior data. This frees up storage resources and avoids wasting storage space. Furthermore, it ensures that only accurate and valid data is retained in the data storage system, preventing abnormal data from interfering with subsequent analysis and queries, thus improving the overall quality and consistency of the data. Reducing unnecessary data storage allows for faster location of valid data during queries, improving query efficiency.

[0098] like Figure 8 The real-time processing flow shown includes the following main processes in the real-time processing chain: (1) Client playback log reporting: The playback behavior data of users on various terminals (such as mobile phones and computers) is recorded and reported to the server as playback logs.

[0099] (2) Local aggregation of consumption engineering to improve consumption throughput: The message engineering in the server performs local aggregation and consumption processing on the reported playback logs to improve the efficiency and throughput of data processing.

[0100] (3) Determine if it is a valid playback: Check if the playback duration of each playback behavior data exceeds the preset threshold (e.g., 30 seconds) to determine if it is a valid playback behavior data. If it is not a valid playback, the process ends.

[0101] (4) Construct the main index (main rowkey): For valid playback behavior data, construct the main index (main rowkey) in the format of "user ID (uid) + time", and its value is the specific playback reporting content (i.e. valid playback behavior data).

[0102] (5) Generate a secondary index key: Artist Query (Resource Creator Identification Dimension): The generated secondary index key is related to the artist query. The row key format of the secondary index key is "uid + artistId + time", and its value stores the corresponding primary row key.

[0103] Device Query (Playback Device Identifier Dimension): The generated secondary index key is related to device query. The row key format of the secondary index key is "uid + deviceId + time", and its value stores the corresponding primary row key.

[0104] Song Query (Resource Identifier Dimension): The generated secondary index key is related to song query. The row key format of the secondary index key is "uid + songId + time", and its value stores the corresponding primary row key.

[0105] (6) Write data to HBase storage (real-time table): Write the generated primary index key (primary rowkey) and secondary index key (secondary key) as a whole to the real-time table (first storage table) of the distributed database (such as HBase).

[0106] like Figure 9 The offline processing flow shown includes the following main processes in the offline processing chain: (1) T+1 offline Hive table task statistics: On the day following the playback action (T+1), start an offline batch processing task based on the Hive data warehouse. Hive is a core data storage structure in a data warehouse tool.

[0107] (2) Construct the main index (main rowkey): the format is "uid + time" or "user identifier (uid) + time", and its value is the specific playback reporting content (i.e. playback behavior data).

[0108] (3) Generate a secondary index key: Artist Query (Resource Creator Identification Dimension): The generated secondary index key is related to the artist query. The row key format of the secondary index key is "uid + artistId + time", and its value stores the corresponding primary row key.

[0109] Device Query (Playback Device Identifier Dimension): The generated secondary index key is related to device query. The row key format of the secondary index key is "uid + deviceId + time", and its value stores the corresponding primary row key.

[0110] Song Query (Resource Identifier Dimension): The generated secondary index key is related to song query. The row key format of the secondary index key is "uid + songId + time", and its value stores the corresponding primary row key.

[0111] (4) Write data to HBase storage (offline table refresh): Write the generated primary index key (primary rowkey) and secondary index key (secondary key) to the offline table (secondary storage table) of the distributed database (such as HBase) for data refresh.

[0112] (5) Delete abnormal data in the real-time table: Delete abnormal data (i.e. abnormal playback behavior data) in the real-time table (second storage table) to ensure the accuracy and consistency of the data.

[0113] By implementing the data processing method provided in this application, the audio and video content platform stably stores approximately 3 trillion playback detail records from the past year. For paginated queries based on the primary index (e.g., retrieving 100 records per request), the average response time (RT) from receiving the query request to returning the query result is approximately 150ms. For filtering query requests based on the target auxiliary index key corresponding to the target filterable attribute dimension, the average response time is around 180ms, showing relatively stable response times. After deployment in the online environment, this function processes over 100,000 user query requests daily, significantly reducing daily user complaints due to unclear playback record sources and providing strong data support for various teams within the platform to analyze and investigate playback behavior-related issues.

[0114] Currently, the storage and efficient retrieval technologies for trillions of data points on massive consumer platforms generally face challenges. Besides the HBase series solutions adopted in this application's embodiments, several other technical paths exist in the industry, but their characteristics are difficult to meet the stringent environmental requirements of high-concurrency writes and real-time queries unique to dubbing and video platforms. This application's embodiments not only achieve a linear display of playback history along the time dimension, but also innovatively provide flexible retrieval and query capabilities for playback behavior data based on multiple dimensions (such as resources, creators, and devices), particularly enabling cross-terminal device playback record association queries.

[0115] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0116] The data processing method provided in this application embodiment receives playback behavior data from multiple terminals; for each playback behavior data, a primary index key is generated to uniquely identify the playback behavior data; for at least one filterable attribute dimension included in the playback behavior data, corresponding secondary index keys are generated, and the value of each secondary index key stores the associated primary index key; in response to a playback record query request initiated by a client, if the playback record query request includes filtering conditions for a target filterable attribute dimension, a retrieval is performed based on the target secondary index key corresponding to the filtering conditions to obtain at least one associated primary index key, and the corresponding target playback behavior data is obtained based on at least one primary index key. This application embodiment, by generating a primary index key for playback behavior data and generating secondary index keys for filterable attribute dimensions and establishing a relationship between the two, enables efficient retrieval and flexible querying of playback behavior data when a client initiates a playback record query request with filtering conditions, by retrieving the associated primary index key based on the target secondary index key.

[0117] To facilitate better implementation of the data processing method of this application embodiment, this application embodiment also provides a data processing apparatus. Please refer to... Figure 10 , Figure 10 A schematic diagram of the structure of a data processing apparatus provided in an embodiment of this application. The data processing apparatus 200 may include: The receiving unit 210 is used to receive playback behavior data from multiple terminals; The first generation unit 220 is used to generate a primary index key for each playback behavior data to uniquely identify the playback behavior data. The second generation unit 230 is used to generate corresponding auxiliary index keys for at least one filterable attribute dimension contained in the playback behavior data, and the value of each auxiliary index key stores the associated primary index key. The processing unit 240 is configured to respond to a playback history query request initiated by the client. If the playback history query request includes filtering conditions for a target filterable attribute dimension, it performs a retrieval based on the target auxiliary index key corresponding to the target filterable attribute dimension to obtain at least one associated primary index key, and obtains the corresponding target playback behavior data based on the at least one primary index key.

[0118] In some embodiments, the filterable attribute dimensions include at least one of the following: resource identifier dimension, resource creator identifier dimension, and playback device identifier dimension.

[0119] In some embodiments, the second generation unit 230 is further configured to: establish an association between the auxiliary index key and the primary index key corresponding to each filterable attribute dimension by storing the primary index key as the value of the auxiliary index key in the database; wherein each auxiliary index key includes a user identifier, the identifier information of the filterable attribute dimension, and behavior time information arranged in reverse order.

[0120] In some embodiments, the processing unit 240 is configured to perform a retrieval based on the target auxiliary index key corresponding to the target filterable attribute dimension to obtain at least one associated primary index key, including: determining the scanning range of the target auxiliary index key based on the filtering conditions; and scanning the database for target auxiliary index keys that satisfy the scanning range to obtain at least one associated primary index key value.

[0121] In some embodiments, the processing unit 240 is further configured to: in response to a playback record query request containing pagination parameters, determine a query range based on the sorting rules of the primary index key; and use a batch query method to obtain no more than N playback behavior data records for each query, where N is the preset pagination size indicated by the pagination parameters.

[0122] In some embodiments, the first generation unit 220 is configured to: perform a hash operation on the user identifier, take the first M bits of the operation result as a hash prefix, where M is a positive integer; concatenate the hash prefix, user identifier, data status identifier, device identifier, resource identifier and reverse-ordered behavior timestamps to generate the primary index key, wherein the data status identifier is used to identify the validity of the playback behavior data.

[0123] In some embodiments, the resource identifier includes a combination of a content resource identifier and a reporting time identifier, used to distinguish multiple reports of playback behavior data of the same content resource by the same user within a millisecond time. The first generation unit 220 is further configured to: if the same user reports playback behavior data of the same content resource multiple times within the same millisecond time, then when generating the primary index key, the multiple reported playback behavior data will be treated as the same playback behavior data.

[0124] In some embodiments, the processing unit 240 is further configured to: if the playback record query request does not contain filtering conditions for the target filterable attribute dimension, perform data retrieval based on the sorting rules of the primary index key to obtain target playback behavior data.

[0125] In some embodiments, the processing unit 240 is configured to perform data retrieval based on the sorting rules of the primary index key, including: performing a range scan or prefix matching scan based on the reverse-ordered behavior timestamps contained in the primary index key to obtain target playback behavior data.

[0126] In some embodiments, the data processing device 200 further includes a judgment unit for judging whether the playback behavior data is valid playback data, wherein playback behavior data with a playback duration exceeding a preset threshold is judged as valid playback behavior data; The first generation unit 220 is used to generate a primary index key for each valid playback behavior data to uniquely identify the valid playback behavior data; The second generation unit 230 is used to generate corresponding auxiliary index keys for at least one filterable attribute dimension contained in the effective playback behavior data.

[0127] In some embodiments, the processing unit 240 is further configured to: store the playback behavior data and its corresponding primary index key and secondary index key in a first storage table via a real-time processing link; perform anomaly detection and correction on the playback behavior data in the first storage table via an offline processing link, generate corrected playback behavior data, and store it in a second storage table; set a version identifier for each playback behavior data to indicate whether each playback behavior data has been corrected by the offline processing link, the version identifier including a first identifier and a second identifier, the first identifier corresponding to the first storage table and the second identifier corresponding to the second storage table; and, in response to the playback record query request, determine whether to read data from the first storage table or the second storage table based on the version identifier.

[0128] In some embodiments, the processing unit 240 is configured to perform anomaly detection and correction on the playback behavior data in the first storage table via an offline processing link, including: associating the first storage table with a risk control data table via an offline processing link to identify abnormal playback behavior data in the first storage table; marking or correcting the abnormal playback behavior data in the first storage table to generate a corrected playback behavior dataset.

[0129] In some embodiments, the processing unit 240 is further configured to: after storing the corrected playback behavior dataset in the second storage table, synchronously delete the corresponding abnormal playback behavior data in the first storage table, and delete the primary index key and secondary index key related to the abnormal playback behavior data.

[0130] Each unit in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each unit.

[0131] The data processing device 200 can be integrated into a terminal or server that has storage and a processor and thus computing power, or the data processing device 200 can be the terminal or server.

[0132] Optionally, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0133] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device may be a terminal or a server. Figure 11 As shown, the computer device 300 includes a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, and a computer program stored in the memory 302 and executable on the processor. The processor 301 is electrically connected to the memory 302. Those skilled in the art will understand that the computer device structure shown in the figures does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0134] The processor 301 is the control center of the computer device 300. It connects various parts of the computer device 300 through various interfaces and lines. By running or loading software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, it performs various functions of the computer device 300 and processes data, thereby performing overall processing of the computer device 300.

[0135] In this embodiment, the processor 301 in the computer device 300 loads the instructions corresponding to the processes of one or more computer programs into the memory 302 according to the following steps, and the processor 301 runs the computer programs stored in the memory 302 to realize various functions: receiving playback behavior data from multiple terminals; generating a primary index key to uniquely identify each playback behavior data; generating corresponding secondary index keys for at least one filterable attribute dimension included in the playback behavior data, with the value of each secondary index key storing the associated primary index key; responding to a playback record query request initiated by the client, if the playback record query request includes filtering conditions for a target filterable attribute dimension, then performing a retrieval based on the target secondary index key corresponding to the target filterable attribute dimension to obtain at least one associated primary index key, and obtaining the corresponding target playback behavior data based on the at least one primary index key.

[0136] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0137] Optional, such as Figure 11 As shown, the computer device 300 also includes: a display screen 303, a radio frequency circuit 304, an audio circuit 305, an input unit 306, and a power supply 307. The processor 301 is electrically connected to the display screen 303, the radio frequency circuit 304, the audio circuit 305, the input unit 306, and the power supply 307. Those skilled in the art will understand that... Figure 11 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0138] The display screen 303 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The display screen 303 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the computer device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 301, and can receive and execute commands from the processor 301. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 301 to determine the type of touch event. Subsequently, the processor 301 provides corresponding visual output on the display panel according to the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the display screen 303 to achieve input and output functions. However, in some embodiments, the touch panel and the display screen 303 can be implemented as two independent components to achieve input and output functions. That is, the display screen 303 can also be used as part of the input unit 306 to achieve input functions.

[0139] The radio frequency circuit 304 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other computer devices, and to transmit and receive signals with network devices or other computer devices.

[0140] Audio circuitry 305 can be used to provide an audio interface between a user and a computer device via a speaker and a microphone. Audio circuitry 305 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 305, converted back into audio data, and output to processor 301 for processing. The audio data is then transmitted via radio frequency circuitry 304 to, for example, another computer device, or output to memory 302 for further processing. Audio circuitry 305 may also include an earphone jack to facilitate communication between peripheral headphones and the computer device.

[0141] The input unit 306 can be used to receive input numbers, characters, or object feature information (such as fingerprints, irises, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0142] Power supply 307 is used to supply power to various components of computer device 300. Optionally, power supply 307 can be logically connected to processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 307 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0143] although Figure 11 As not shown in the diagram, computer equipment 300 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0144] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding flow in the data processing method of the embodiments of this application; for brevity, further details are omitted here.

[0145] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding flow in the data processing method of the embodiments of this application. For simplicity, further details are omitted here.

[0146] This application also provides a computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding flow in the data processing method of this application. For brevity, further details are omitted here.

[0147] It should be understood that the processor in this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0148] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0149] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0151] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0153] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] In addition, the functional units in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0155] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0156] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: Receive playback behavior data from multiple terminals; For each of the playback behavior data, a primary index key is generated to uniquely identify the playback behavior data; For each of the at least one filterable attribute dimension contained in the playback behavior data, a corresponding auxiliary index key is generated, and the value of each auxiliary index key is stored in relation to the associated primary index key. In response to a playback history query request initiated by the client, if the playback history query request includes filtering conditions for the target filterable attribute dimension, then a search is performed based on the target auxiliary index key corresponding to the target filterable attribute dimension to obtain at least one associated primary index key, and the corresponding target playback behavior data is obtained based on the at least one primary index key.

2. The data processing method according to claim 1, characterized in that, The filterable attribute dimensions include at least one of the following: resource identifier dimension, resource creator identifier dimension, and playback device identifier dimension.

3. The data processing method according to claim 1 or 2, characterized in that, The method further includes: By storing the primary index key as the value of the secondary index key in the database, an association relationship is established between the secondary index key and the primary index key corresponding to each filterable attribute dimension; Each of the auxiliary index keys includes a user identifier, the identifier information of the filterable attribute dimension, and the behavior time information arranged in reverse order.

4. The data processing method according to claim 1, characterized in that, The step of retrieving at least one associated primary index key based on the target's filterable attribute dimension and corresponding auxiliary index key includes: The scanning range of the target auxiliary index key is determined based on the filtering criteria; Scan the database for target secondary index keys that satisfy the scan range to obtain at least one primary index key value associated with them.

5. The data processing method according to claim 1, characterized in that, The method further includes: In response to a playback history query request containing pagination parameters, the query range is determined based on the sorting rules of the primary index key; A batch query method is used to retrieve no more than N playback behavior data in each query, where N is the preset page size indicated by the pagination parameter.

6. The data processing method according to claim 1, characterized in that, The generation of the primary index key for uniquely identifying the playback behavior data includes: Perform a hash operation on the user identifier, and take the first M bits of the result as the hash prefix, where M is a positive integer; The primary index key is generated by concatenating the hash prefix, user identifier, data status identifier, device identifier, resource identifier, and reverse-ordered behavior timestamps, wherein the data status identifier is used to identify the validity of the playback behavior data.

7. A data processing apparatus, characterized in that, The device includes: The receiving unit is used to receive playback behavior data from multiple terminals; The first generation unit is used to generate a primary index key for each playback behavior data to uniquely identify the playback behavior data. The second generation unit is used to generate corresponding auxiliary index keys for at least one filterable attribute dimension contained in the playback behavior data, and the value of each auxiliary index key stores the associated primary index key. The processing unit is configured to respond to a playback history query request initiated by the client. If the playback history query request includes filtering conditions for a target filterable attribute dimension, it performs a retrieval based on the target auxiliary index key corresponding to the target filterable attribute dimension to obtain at least one associated primary index key, and obtains the corresponding target playback behavior data based on the at least one primary index key.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the data processing method as described in any one of claims 1-6.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program, and the processor executing the data processing method according to any one of claims 1-6 by calling the computer program stored in the memory.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the data processing method according to any one of claims 1-6.

Images