Data processing method, device and equipment, readable storage medium and program product

Abstract

The embodiment of the invention provides a data processing method and device, equipment, a readable storage medium and a program product. The method comprises the steps that an associated playing data set associated with test time is acquired from a playing data table stored in a fragment node; acquiring an associated data block associated with the test time from a test data conversion table stored in the fragment node; the test data conversion table comprises a plurality of data blocks, and each data block comprises a recommendation strategy identifier, statistical data time and bitmap data; determining statistical data corresponding to the target recommendation strategy according to the associated playing data group and the associated data block; and determining evaluation data of the target recommendation strategy corresponding to the fragment node in the test time according to the statistical data. Through the method provided by the invention, the storage space can be saved by utilizing the bitmap data, the data can be acquired from the same fragment node, the data query efficiency is effectively improved, the data processing efficiency is improved, and the efficiency of determining the evaluation data of the recommendation strategy is further improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, a data processing apparatus, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the advancement of computer technology, video playback applications are becoming increasingly popular. To enhance the user experience, video playback servers can use recommendation strategies to identify videos that meet the user's needs and push them to the user. Once a new recommendation strategy is determined, it is typically evaluated to ensure its accuracy and effectiveness.

[0003] When evaluating a recommendation strategy, the video playback server typically queries related data and processes the resulting recommendation strategy related data to generate the strategy evaluation data. However, current data query efficiency is low, which can lead to inefficient processing of the recommendation strategy related data, thus affecting the efficiency of determining the recommendation strategy evaluation data. Summary of the Invention

[0004] The embodiments of the present application provide data processing methods, devices, equipment, readable storage media and program products, which can use bitmap data to save storage space, and can obtain data from the same shard node, effectively improving data query efficiency and data processing efficiency, thereby facilitating improving the efficiency of evaluation data for determining recommendation strategies.

[0005] In one aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0006] Obtaining an associated playback data group associated with the test time from a playback data table stored in a sharding node; the playback data table includes a plurality of playback data groups, each of which includes a test subject number and video playback data; the sharding node matches the test subject number, the video playback data is playback data generated by playing a test video according to an instruction of the test subject, and the test video is a video recommended to the test subject corresponding to the test subject number according to a recommendation strategy;

[0007] Obtaining an associated data block associated with the test time from a test data conversion table stored in the sharding node; the test data conversion table includes a plurality of data blocks, each of which includes a recommended strategy identifier, a statistical data time, and bitmap data; the test data conversion table is determined based on a test data table stored in the sharding node, the test data table includes a plurality of test data groups, each of which includes a test object number and a recommended strategy identifier; the test data group is associated with a play data group in the play data table; the bitmap data is used to indicate whether an associated test data group corresponding to the test object number is obtained within the statistical data time, the associated test data group being the same as the recommended strategy identifier included in the data block;

[0008] Determining statistical data corresponding to a target recommendation strategy based on the associated playback data group and the associated data block;

[0009] Determine evaluation data of the target recommendation strategy corresponding to the shard node within the test time based on the statistical data.

[0010] In one aspect, an embodiment of the present application provides a data processing device, comprising:

[0011] An acquisition unit is configured to acquire, from a playback data table stored in a sharding node, an associated playback data group associated with a test time; the playback data table includes a plurality of playback data groups, each of which includes a test subject number and video playback data; the sharding node matches the test subject number, the video playback data is playback data generated by playing a test video according to an instruction of the test subject, and the test video is a video recommended to the test subject corresponding to the test subject number according to a recommendation strategy;

[0012] The acquisition unit is further configured to acquire, from a test data conversion table stored in the sharding node, an associated data block associated with the test time; the test data conversion table includes a plurality of data blocks, each of which includes a recommended strategy identifier, a statistical data time, and bitmap data; the test data conversion table is determined based on a test data table stored in the sharding node, the test data table includes a plurality of test data groups, each of which includes a test object number and a recommended strategy identifier; the test data group is associated with a playback data group in the playback data table; the bitmap data is used to indicate whether an associated test data group corresponding to the test object number is acquired within the statistical data time, the associated test data group being the same as the recommended strategy identifier included in the data block;

[0013] a determining unit, configured to determine statistical data corresponding to a target recommendation strategy based on the associated playback data group and the associated data block;

[0014] A processing unit is used to determine evaluation data of the target recommendation strategy corresponding to the shard node within the test time based on the statistical data.

[0015] On the one hand, an embodiment of the present application provides a computer device, comprising: a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores computer instructions, and the processor is used to call the computer instructions to implement the data processing method provided in the embodiment of the present application.

[0016] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer-readable storage medium is executed on a computer device, the computer device implements the data processing method provided in the embodiment of the present application.

[0017] Accordingly, embodiments of the present application further provide a computer program product, comprising a computer program or computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium and executes the computer program or computer instructions, causing the computer device to implement the data processing method provided in embodiments of the present application.

[0018] Through the data processing method provided by the embodiment of the present application, an associated playback data group associated with the test time can be obtained from the playback data table stored in the sharding node, and the playback data table stores the playback data generated by playing the test video according to the instruction of the test object, and the test video is a video recommended to the test object according to the recommendation strategy. Therefore, the video playback data in the playback data table can reflect to a certain extent whether the test video meets the video requirements of the test object, and further reflect whether the recommendation strategy for determining the test video is accurate; the associated data block associated with the test time can be obtained from the test data conversion table stored in the sharding node, and the test data conversion table may include multiple data blocks, and the data blocks may include the recommendation strategy identifier, statistical data time, and bitmap data, wherein the storage space consumed by the bitmap data is much smaller than that of the test data The storage space consumed by the data in the table can be effectively saved; the data of the two tables, the playback data table and the test data conversion table, can be used to comprehensively perform data processing to ensure the accuracy of the evaluation data; the evaluation data of the recommendation strategy within the test time can be determined by obtaining associated playback data groups and associated data blocks from the same shard node, without the need to obtain data across shard nodes for processing, which improves data query efficiency and effectively improves data processing efficiency, thereby improving the efficiency of determining the evaluation data of the recommendation strategy; the test time can be in minutes, and the data processing method provided in the embodiment of the present application can be used to obtain evaluation data of the recommendation strategy within minutes, that is, the processing delay of the evaluation data can be reduced to minutes, which is conducive to the real-time evaluation of the recommendation strategy, thereby improving the update efficiency of the recommendation strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 This is a schematic diagram of the system architecture of a data processing system provided in an embodiment of the present application;

[0021] Figure 2 This is a flow chart of a data processing method provided in an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a data processing method provided in an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a method for determining average playback duration provided by an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of a method for determining the average number of playback times provided in an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of another data processing method provided in an embodiment of the present application;

[0026] Figure 7 This is a flow chart of another data processing method provided in an embodiment of the present application;

[0027] Figure 8 is a schematic diagram of a method for determining evaluation data provided in an embodiment of the present application;

[0028] Figure 9 This is a structural block diagram of a data processing device provided in an embodiment of the present application;

[0029] Figure 10 This is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0031] It should be noted that the terms "first" and "second" in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature designated as "first" or "second" may explicitly or implicitly include at least one such feature.

[0032] To facilitate understanding, we first provide a brief explanation of some nouns:

[0033] Recommendation system: A recommendation system is a personalized information system that solves the matching problem between objects, items, and environments, helps establish connections between objects and items, and recommends corresponding information, products, etc. to objects based on their information needs.

[0034] AB testing: AB testing involves randomly dividing users into two groups within a recommendation system: a control group (Group A) and an experimental group (Group B). Different recommendation strategies are then applied to these two groups, and the behavioral differences between the two groups are compared to determine which strategy is most effective. Through AB testing, the recommendation system can better understand user needs and other information, thereby optimizing recommendation strategies and improving user experience and conversion rates.

[0035] AB experiment metrics: AB experiment metrics are key indicators for measuring AB experiment results. In video recommendation systems, these metrics can include average play time, which refers to the average duration a video is played across all corresponding devices; and average play count, which refers to the average number of times a video is played across all corresponding devices. Both average play time and average play count are important metrics for measuring video quality and the effectiveness of recommendation algorithms. They help recommendation systems evaluate the effectiveness of different videos and algorithms, determining which ones are most effective in increasing play time and play count.

[0036] Real-time attribution: Recommendation results (e.g., recommended videos determined by a recommendation strategy) are typically displayed on the app's homepage. The gains generated by clicking through to multiple pages recommended by the homepage are considered conversion gains from the homepage recommendations. This aggregation of gains is called attribution. Real-time attribution involves calculating attribution directly on the terminal device, then sending each message to be attributed to the server in real time for metric calculation.

[0037] ClickHouse: An open-source column-based database management system designed for real-time query and analysis of large-scale data. Characterized by high performance, scalability, and fault tolerance, ClickHouse can handle massive amounts of data and supports Structured Query Language (SQL) and multiple data formats. Using column-based storage and vectorized querying, ClickHouse can quickly process large amounts of data, supporting real-time query and interactive analysis. It is suitable for scenarios such as data warehousing, log analysis, and business intelligence.

[0038] Flink, also known as Apache Flink, is an open-source stream processing framework for processing and analyzing real-time streaming data. Flink is characterized by high performance, scalability, and fault tolerance. It can handle large-scale real-time data streams and supports real-time, batch, and iterative computations on these streams. Flink utilizes an event-driven stream processing model that can handle unbounded data streams and supports a rich set of stream processing operations and operators, such as windowing, filtering, aggregation, and joins. Flink also provides a rich set of APIs and tools for integration with popular data processing frameworks and tools. Flink is widely used in scenarios such as real-time data processing, streaming machine learning, and real-time recommendations.

[0039] In a recommendation system, once a new recommendation strategy is determined, an A / B experiment can be conducted on it. Evaluation data for the strategy can then be determined based on the A / B experiment metrics, facilitating updating the strategy based on the evaluation data. When evaluating a recommendation strategy, the data associated with the strategy is typically processed to generate the evaluation data. However, currently, efficient processing of data associated with the strategy is not feasible, which can significantly impact the efficiency of determining evaluation data for the strategy.

[0040] Based on this, an embodiment of the present application provides a data processing method, which can obtain an associated playback data group associated with the test time from a playback data table stored in a sharding node; the playback data table includes multiple playback data groups, and the playback data group includes a test object number and video playback data; the sharding node matches the test object number, and the video playback data is the playback data generated by playing the test video according to the instruction of the test object, and the test video is a video recommended to the test object corresponding to the test object number according to the recommendation strategy; the associated data block associated with the test time is obtained from the test data conversion table stored in the sharding node; the test data conversion table includes multiple data blocks, and the data blocks include recommended Strategy identifier, statistical data time, bitmap data; the test data conversion table is determined based on the test data table stored in the sharding node, the test data table includes multiple test data groups, the test data group includes a test object number, a recommended strategy identifier; the test data group is associated with a playback data group in the playback data table; the bitmap data is used to indicate: whether the associated test data group corresponding to the test object number is obtained within the statistical data time, and the associated test data group is the same as the recommended strategy identifier included in the data block; the statistical data corresponding to the target recommendation strategy is determined based on the associated playback data group and the associated data block; the evaluation data of the target recommendation strategy corresponding to the sharding node within the test time is determined based on the statistical data. Through the data processing method provided in the embodiment of the present application, bitmap data can be used to save storage space, and data can be obtained from the same sharding node, effectively improving data query efficiency and data processing efficiency, thereby helping to improve the efficiency of determining the evaluation data of the recommendation strategy.

[0041] The data processing method provided in the embodiment of the present application can be applied to the field of artificial intelligence. Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In an artificial intelligence system, in order to improve the user experience of an object, a recommendation strategy can be used to determine the information pushed to the object; and in order to ensure the accuracy of the recommendation strategy, the data processing method provided in the embodiment of the present application can be used to obtain the associated playback data group associated with the test time from the playback data table stored in the sharding node, and obtain the associated data block associated with the test time from the test data conversion table stored in the sharding node; the statistical data corresponding to the target recommendation strategy can be determined based on the associated playback data group and the associated data block, and the evaluation data of the target recommendation strategy corresponding to the sharding node within the test time can be determined based on the statistical data. The evaluation data can indicate the accuracy of the recommendation strategy to a certain extent. The method provided in the embodiment of the present application can effectively improve the efficiency of determining the evaluation data of the recommendation strategy, thereby helping to improve the update efficiency of the recommendation strategy and enhance the user experience.

[0042] The data processing method provided in the embodiment of the present application can also be applied to the field of cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices. The data processing method provided in the embodiment of the present application can be implemented based on cloud computing technology. Specifically, the sharding node can be a cloud storage node. The cloud computing device can obtain the associated playback data group associated with the test time from the playback data table stored in the sharding node; obtain the associated data block associated with the test time from the test data conversion table stored in the sharding node; determine the statistical data corresponding to the target recommendation strategy based on the associated playback data group and the associated data block; determine the evaluation data of the target recommendation strategy corresponding to the sharding node within the test time based on the statistical data. After the cloud computing device determines the evaluation data, it can send the evaluation data to the policy server so that the policy server adjusts the corresponding recommendation strategy based on the evaluation data. Through the method provided in the embodiment of the present application, the update efficiency of the recommendation strategy can be effectively improved, which is conducive to improving the accuracy of the recommendation strategy.

[0043] The data processing method provided in the embodiment of the present application can also be applied to the field of network media. Network media refers to a type of media that uses network technology to share and interact with information, that is, websites, podcasts, online television, videos, etc. that transmit information through the Internet. Network media is an emerging type of media relative to traditional media (including newspapers, magazines, radio stations, and television stations). It makes information dissemination easier with a lower threshold, faster speed, and a wider audience. The data processing method provided in the embodiment of the present application can be applied to the field of network media to enhance the user experience of network media audiences.

[0044] The following is an example of network media such as video: the server can determine the test video according to the recommendation strategy and send the test video to the terminal device, the test object can interact with the terminal device to make the terminal device play the test video; the terminal device can send the original playback data and the original test data to the server, the server can process the original playback data and the original test data, and determine the playback data table and the test data conversion table based on the processed data. The server can use the method provided in the embodiment of the present application to obtain the associated playback data group from the playback data table stored in the sharding node, and obtain the associated data block from the test data conversion table stored in the sharding node; the statistical data corresponding to the target recommendation strategy can be determined based on the associated playback data group and the associated data block, and the evaluation data of the target recommendation strategy corresponding to the sharding node within the test time can be determined based on the statistical data. The evaluation data can indicate the accuracy of the recommendation strategy to a certain extent. The method provided in the embodiment of the present application can effectively improve the efficiency of determining the evaluation data of the recommendation strategy, thereby improving the update efficiency of the recommendation strategy and enhancing the user experience of the network media audience.

[0045] The architecture of the data processing system provided in the embodiments of the present application will be introduced below with reference to the accompanying drawings.

[0046] See Figure 1 , which is a schematic diagram of the system architecture of a data processing system provided by an embodiment of the present application. The data processing system includes a terminal device 101, a data processing server 102 and a policy adjustment device 103. The data processing server 102 can interact with the terminal device 101 and the policy adjustment device 103 for data. The data processing server 102 includes multiple shard nodes (such as Figure 1 The data processing server 102 includes shard node 1, shard node 2, ..., shard node N, where N is an integer greater than 2.

[0047] The terminal device 101 can interact with the test subject and play the video according to the test subject's instructions. It can also send the original data generated by the video playback (the original data may include original playback data and original test data) to the data processing server 102. The terminal device 101 can be a handheld device with video playback and communication functions (such as a smart phone, tablet computer), a computing device (such as a personal computer (PC), a car terminal, an intelligent voice interaction device, a wearable device, or other smart devices, but is not limited to these.

[0048] The data processing server 102 can receive the original data sent by the terminal device 101, process the original data, and store the processed data in the sharding node. It can also determine the evaluation data of the recommendation strategy based on the data in the sharding node. The data processing server 102 includes multiple sharding nodes, and each sharding node can store a play data table and a test data conversion table. The data processing server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0049] The policy adjustment device 103 can receive the evaluation data of the recommended policy sent by the data processing server 102, and adjust the corresponding recommended policy based on the evaluation data, thereby improving the accuracy of the recommended policy. The policy adjustment device 103 can be a terminal device, an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0050] The following will explain in detail Figure 1 The working principle of the data processing system shown:

[0051] The terminal device 101 can play the test video according to the instruction of the test object, and the test video can be determined according to the recommendation strategy. The terminal device 101 can generate original data based on the data generated when playing the test video, and the original data can include original playback data and original test data. The terminal device 101 can send the original playback data and the original test data to the data processing server 102 at the same time. The data processing server 102 can process the original playback data to obtain a playback data group (the playback data group can include the test object number and video playback data), and store the playback data group in the playback data table of the sharding node ( Figure 1 The data processing server 102 can process the original test data to obtain a test data group, and store the test data group in the test data table of the same shard node ( Figure 1The test data group is stored in shard node 2. The data processing server 102 can also process the test data table to obtain a test data conversion table. The test data conversion table includes multiple data blocks, where each data block can include a recommended strategy identifier, statistical data time, and bitmap data. The bitmap data can be used to indicate whether an associated test data group corresponding to the test object number is obtained within the statistical data time, and the associated test data group is the same as the recommended strategy identifier included in the data block.

[0052] The data processing server 102 can obtain the associated playback data group associated with the test time from the playback data table stored in the shard node 1; it can obtain the associated data block associated with the test time from the test data conversion table stored in the shard node 1; the data processing server 102 can determine the statistical data corresponding to the target recommendation strategy based on the associated playback data group and the associated data block, and determine the evaluation data of the target recommendation strategy corresponding to the shard node 1 during the test time based on the statistical data.

[0053] The same method can be used to determine the evaluation data of the target recommendation strategy corresponding to each of the N shard nodes in the data processing server 102 during the test time. The data processing server 102 can determine the overall evaluation data of the target recommendation strategy during the test time based on the evaluation data of the N target recommendation strategies during the test time, and send the overall evaluation data to the strategy adjustment device 103. The strategy adjustment device 103 can adjust the target recommendation strategy based on the overall evaluation data. Through the data processing method provided in the embodiment of the present application, bitmap data can be used to save storage space, and data can be obtained from the same shard node, which effectively improves data query efficiency and data processing efficiency, thereby helping to improve the efficiency of determining the evaluation data of the recommendation strategy.

[0054] It is understood that the architectural diagram of the data processing system described in the embodiment of the present application is intended to more clearly illustrate the data processing method of the embodiment of the present application, and does not constitute a limitation on the data processing method provided in the embodiment of the present application. For example, the data processing method provided in the embodiment of the present application can be executed by the data processing server 102, as well as by other devices that are different from the data processing server 102 and can communicate with the terminal device 101 and the policy adjustment device 103. It is known to those skilled in the art that Figure 1 The number of terminal devices 101, data processing servers 102, and policy adjustment devices 103 is merely illustrative. Any number of devices and nodes can be configured based on business implementation needs. Furthermore, as system architecture evolves and new business scenarios emerge, the data processing methods provided in the embodiments of this application are equally applicable to similar technical problems.

[0055] It should be noted that the collection and processing of relevant data in this application (for example, test subject number, video playback data, etc.) should be strictly in accordance with the requirements of relevant laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0056] See Figure 2 , Figure 2 This is a flow chart of a data processing method provided in an embodiment of the present application. The data processing method can be implemented by the above-mentioned data processing server 102, or by the above-mentioned policy adjustment device 103, or by other devices capable of implementing the data processing method. The following description uses the data processing method implemented by the above-mentioned data processing server 102 as an example. The flow of the data processing method provided in the embodiment of the present application includes, but is not limited to:

[0057] S201. Obtain an associated playback data group associated with the test time from a playback data table stored in a sharding node; the playback data table includes a plurality of playback data groups, and the playback data group includes a test object number and video playback data; the sharding node matches the test object number, and the video playback data is playback data generated by playing a test video according to an instruction of the test object, and the test video is a video recommended to the test object corresponding to the test object number according to a recommendation strategy.

[0058] In an embodiment of the present application, the data processing server may include a plurality of sharding nodes, which are obtained by dividing the storage space of the data processing server, and each sharding node may store a corresponding playback data table. The playback data table in the sharding node may include a plurality of playback data groups, each playback data group may include a test object number and video playback data, wherein the test object number matches the sharding node, and the video playback data is the playback data generated by playing the test video according to the instruction of the test object, and the test video is the video recommended to the test object corresponding to the test object number according to the recommendation strategy. The data processing server may obtain the associated playback data group associated with the test time from the playback data table stored in the sharding node, and the test time is the evaluation time for the recommendation strategy determined according to the application requirements, for example: if the test time is 1 minute, the data processing server may adopt the data processing method provided in the embodiment of the present application to determine the evaluation data of the target recommendation strategy every minute, thereby realizing the real-time evaluation of the target recommendation strategy. By the method provided in the embodiment of the present application, the associated playback data group can be determined, which is conducive to the subsequent determination of the evaluation data of the target recommendation strategy based on the associated playback data group.

[0059] In one embodiment, the data processing method provided by the present application may further include: receiving a video acquisition request sent by a terminal device, and determining object description information of the test object based on a test object identifier included in the video acquisition request, wherein the video acquisition request is used to request acquisition of a video that meets the video requirements of the test object, and the object description information includes video requirement feature information of the test object; determining a target recommendation strategy from multiple recommendation strategies, and using the target recommendation strategy to process the video requirement feature information to obtain a recommended video to be played; and sending the recommended video to be played to the terminal device. The terminal device may generate and send a video acquisition request to the data processing server in response to the test object's instruction, wherein the video acquisition request may include the test object identifier of the test object.

[0060] The data processing server can determine the object description information of the test object based on the test object identifier. The object description information may include the video requirement characteristics information of the test object, for example, the video requirement characteristics information of the test object may include high click-through rate video types, high play time video types, etc. The data processing server can determine a target recommendation strategy from multiple recommendation strategies to be evaluated. For example, a recommendation strategy can be randomly determined from the multiple recommendation strategies to be evaluated and determined as the target recommendation strategy. The data processing server can use the target recommendation strategy to process the video requirement characteristics information of the test object to obtain a recommended video to be played. The data processing server sends the recommended video to be played to the terminal device, and the terminal device can play the recommended video to be played. If the recommended video to be played can well meet the video requirement of the test object (for example, the terminal device plays the recommended video to be played for a long time), then the target recommendation strategy is relatively accurate. If the recommended video to be played cannot well meet the video requirement of the test object (for example, the terminal device plays the recommended video to be played for a short time), then the target recommendation strategy is not accurate enough and needs to be further adjusted. Through the method provided in the embodiment of the present application, the recommended video to be played determined according to the recommendation strategy can be sent to the terminal device, which is conducive to subsequent adjustment of the recommendation strategy according to the playback data sent by the terminal device, and is conducive to improving the accuracy of the recommendation strategy.

[0061] It should be noted that, when the data processing server receives video acquisition requests sent by multiple terminal devices, it uses the above method to determine the recommended videos to be played, and sends the recommended videos to be played to the corresponding terminal devices.

[0062] In one embodiment, the method provided by the embodiment of the present application further includes: obtaining original playback data from a playback data queue, and performing data extraction processing on the original playback data to obtain a test object identifier, video playback data and a first timestamp information, wherein the original playback data is sent by the terminal device corresponding to the test object at the time indicated by the first timestamp information; if there is a matching number that matches the test object identifier in the number mapping table, the matching number is determined as the test object number corresponding to the test object identifier; if there is no matching number that matches the test object identifier in the number mapping table, a test object number is generated according to the test object identifier, and the test object identifier and the test object number are added to the number mapping table; a playback data group is generated according to the test object number, the video playback data and the first timestamp information; a target shard node is determined from multiple shard nodes corresponding to the test database according to the test object number, and the playback data group is stored in the target shard node.

[0063] After receiving a recommended video (or test video) from a data processing server, the terminal device can play the recommended video according to the test subject's instructions. The terminal device can also generate raw playback data based on the test subject's instructions. For example, after receiving a recommended video from a data processing server, the terminal device can automatically play the recommended video. If the terminal device receives a pause or video close instruction from the test subject, the terminal device can generate raw playback data based on data such as the playback duration of the recommended video.

[0064] The terminal device can send the original playback data to the data processing server, and the data processing server can store the original playback data in the playback data queue. The data processing server can obtain the original playback data from the playback data queue individually or in batches. After obtaining the original playback data, the data processing server can perform data extraction processing on the original playback data for each piece of original playback data to obtain the test object identifier, video playback data and the first timestamp information, and the original playback data is sent by the terminal device corresponding to the test object at the time indicated by the first timestamp information. The test object identifier is used to indicate the test object, and the format of the test object identifier can be a string containing letters. For example: the original playback data may include a lot of irrelevant data, and the original playback data can be subjected to business scenario filtering processing and business field extraction processing to obtain the data of several required fields, thereby reducing the space consumed by subsequent data storage.

[0065] The data processing server includes a number mapping table, which contains the correspondence between the test object identifier and the test object number. The test object identifier can be matched with the number mapping table. If there is no matching number matching the test object identifier in the number mapping table, the matching number can be determined to be the test object number corresponding to the test object identifier; if there is no matching number matching the test object identifier in the number mapping table, a test object number can be generated based on the test object identifier, and the test object identifier and the test object number can be added to the number mapping table. For example: the number mapping table includes test object identifier 1 and its corresponding test object number "1", the number mapping table includes test object identifier 2 and its corresponding test object number "2", and now the test object identifier 3 is determined based on the original playback data, then the test object number "3" corresponding to the test object identifier 3 can be generated, and the test object identifier 3 and its corresponding test object number "3" can be stored in the number mapping table. A playback data group can be generated based on the test object number, video playback data and the first timestamp information. The data processing server may include a test database, which may include multiple shard nodes. The data processing server may determine a target shard node from the multiple shard nodes corresponding to the test database based on the test object number, and store the play data group in the target shard node. In some cases, the storage structure of the play data group in the play data table of the shard node may be as shown in Table 1 below.

[0066] Table 1:

[0067]

[0068] As shown in Table 1 above, the playback data group may include a test object number (the field name is uid, the field description is the test object identifier after the number, for example, the test object number is 1); video playback data (the field name is playduration, the field description is the playback duration of the test video, for example, the video playback data is 100, which means that the playback duration of the test video is 100 seconds); first timestamp information (the field name is timestamp, the field description is the time when the terminal device sends the corresponding original playback data, for example, the first timestamp information is "2023-01-01 10:00", indicating that the time when the terminal device sends the corresponding original playback data is 2023-01-01 10:00). Through the method provided in the embodiment of the present application, the original playback data can be processed to obtain a playback data group, which is conducive to improving the efficiency of determining the evaluation data of the recommendation strategy.

[0069] It should be noted that the tables provided in the embodiments of the present application are all feasible implementation methods. In actual application, the data in the tables can be adaptively adjusted according to application requirements.

[0070] In one embodiment, the implementation method of determining the target shard node from the multiple shard nodes corresponding to the test database according to the test object number can be: calculating the test object number to obtain the shard value; matching the shard value with the number of each shard node in the multiple shard nodes to obtain the target shard node whose number matches the shard value. The test object number can be calculated, and the calculation method can be a hash modulus calculation. Specifically, the test object number can be hashed and then moduloed, and the modulus value can be the number of shard nodes in the test database (the corresponding calculation formula can be: shard value = hash (test object number) % number of shard nodes). For example: the test object number is 8, the number of shard nodes is 5, and the node numbers of these 5 shard nodes are 0-4, then the method provided in the embodiment of the present application can be used to calculate and obtain the shard value of 3, and then the shard node with node number 3 can be determined as the target shard node. After determining the shard value, the target shard node can be determined from the multiple shard nodes. For the original playback data and original test data sent by the same terminal device, the original playback data and the original test data both contain the same test object identifier. After processing the original playback data and the original test data respectively, a playback data group containing the test object number and a test data group containing the same test object number can be obtained. Since the test object numbers are the same, the determined target sharding nodes are the same, that is, the playback data group and test data group corresponding to a terminal device can be stored in the same sharding node, thereby realizing pre-partition storage of data, which is conducive to improving data acquisition efficiency and does not require data acquisition operations across sharding nodes, thereby helping to further improve data processing efficiency.

[0071] See Figure 3 , which is a schematic diagram of a data processing method provided in an embodiment of the present application. Figure 3In the data processing server, a playback data queue is included, and the playback data queue includes original playback data sent by multiple terminal devices. The data processing server can obtain the original playback data from the playback data queue and perform corresponding processing. The data processing server may include a business processing module, an object numbering module and a sharding calculation module. The business processing module can perform data extraction processing on the original playback data to obtain a test object identifier, video playback data and a first timestamp information. The object numbering module can determine the test object number based on the test object identifier. Specifically, if there is a matching number that matches the test object identifier in the number mapping table, the object numbering module can determine the matching number as the test object number corresponding to the test object identifier; if there is no matching number that matches the test object identifier in the number mapping table, the object numbering module can generate a test object number based on the test object identifier, and add the test object identifier and the test object number to the number mapping table. The sharding calculation module can generate a playback data group based on the test object number, the video playback data and the first timestamp information, and determine the target sharding node ( Figure 3 In the embodiment, the data processing server includes a test database, the test database includes N shard nodes, the target shard node is shard node 2 among the N shard nodes, and N is an integer greater than 2); the shard calculation module can be used to store the playback data group into the playback data table of the target shard node.

[0072] In some cases, you can use the Flink stream processing framework to implement Figure 3 In the data processing process shown, the Flink stream processing framework can obtain the original playback data from the playback data queue in batches and process the original playback data in batches to improve data processing efficiency; the ClickHouse column-based database management system can be used to manage each shard node. Through the method provided in the embodiment of the present application, it is possible to process the original playback data, reduce the storage space consumed by the data, and improve data processing efficiency. At the same time, data pre-partitioning can be achieved to avoid the need to obtain data across nodes when subsequently determining the evaluation data, which is conducive to improving the efficiency of determining the evaluation data.

[0073] S202. Obtain an associated data block associated with the test time from the test data conversion table stored in the sharding node; the test data conversion table includes multiple data blocks, and the data blocks include a recommended strategy identifier, statistical data time, and bitmap data; the test data conversion table is determined based on the test data table stored in the sharding node, and the test data table includes multiple test data groups, and the test data group includes a test object number and a recommended strategy identifier; the test data group is associated with a playback data group in the playback data table; the bitmap data is used to indicate: whether an associated test data group corresponding to the test object number is obtained within the statistical data time, and the associated test data group is the same as the recommended strategy identifier included in the data block.

[0074] In an embodiment of the present application, the sharding node includes a playback data table and a test data conversion table. The data processing server can obtain the associated data block associated with the test time from the test data conversion table of the same sharding node. The test data conversion table may include multiple data blocks, and each data block may include a recommendation strategy identifier, statistical data time, and bitmap data, wherein the recommendation strategy identifier is used to indicate a specific recommendation strategy, the statistical data time is used to indicate the time of the statistical test data group, and the bitmap data is used to indicate: whether the associated test data group corresponding to the test object number is obtained within the statistical data time, and the associated test data group is the same as the recommendation strategy identifier included in the data block. For example: the recommendation strategy identifier is "A" and the statistical data time is T, the bitmap data can be used to indicate that the associated test data group corresponding to the test object number is obtained from the test data table within the statistical data time T, and the associated test data group is the same as the recommendation strategy identifier included in the data block.

[0075] The test data conversion table is determined based on the test data table stored in the sharding node. The test data table can include multiple test data groups, each of which includes a test object number and a recommended strategy identifier. Any test data group can be associated with a playback data group in the playback data table of the same sharding node. The method provided in the embodiments of the present application can obtain associated data blocks associated with the test time, which facilitates the subsequent use of the associated data blocks to determine evaluation data for the recommended strategy, thereby improving the efficiency of determining the evaluation data.

[0076] In one embodiment, the data processing method provided by the embodiment of the present application can also: obtain original test data from the test data queue, and perform data extraction processing on the original test data to obtain a test object identifier, a recommended strategy identifier, and a second timestamp information, wherein the original test data is sent by the terminal device corresponding to the test object at the time indicated by the second timestamp information; if a matching number matching the test object identifier exists in the number mapping table, the matching number is determined to be the test object number corresponding to the test object identifier; if a matching number matching the test object identifier does not exist in the number mapping table, a test object number is generated based on the test object identifier, and the test object identifier and the test object number are added to the number mapping table; a test data group is generated based on the test object number, the recommended strategy identifier, and the second timestamp information; a target shard node is determined from multiple shard nodes corresponding to the test database based on the test object number, and the test data group is stored in the target shard node. The terminal device can simultaneously send the original playback data and the original test data to the data processing server, and the data processing server can store the original playback data in the playback data queue and the original test data in the test data queue.

[0077] The data processing server can retrieve raw test data from the test data queue and extract it to obtain a test object identifier, a recommended test identifier, and a second timestamp. The raw test data was sent by the terminal device corresponding to the test object at the time indicated by the second timestamp. The test object identifier is used to identify the test object and can be formatted as an alphanumeric string. For example, if the raw test data may contain a lot of irrelevant data, the raw test data can be filtered for business scenarios and extracted for business fields to obtain data for the required fields, thereby reducing the space consumed by subsequent data storage.

[0078] The data processing server can use the number mapping table and the test object identifier to determine the test object number. A test data set can be generated based on the test object number, the recommended strategy identifier, and the second timestamp information. The data processing server can determine a target shard node from multiple shard nodes based on the test object number and store the test data set in the test data table of the target shard node. In some cases, the storage structure of the test data set in the test data table of the shard node can be as shown in Table 2 below.

[0079] Table 2:

[0080]

[0081] As shown in Table 2 above, the test data group may include a test object number (the field name is uid, the field description is the test object identifier after the number, for example, the test object number is 1); a recommendation strategy identifier (the field name is exp id, the field description is the identifier of the recommendation strategy, for example, the video playback data is 1000, that is, the test video is determined using the recommendation strategy identified as 1000, and the test video is sent to the terminal device); a second timestamp information (the field name is timestamp, the field description is the time when the terminal device sends the corresponding original test data, for example, the second timestamp information is "2023-01-01 10:00", indicating that the time when the terminal device sends the corresponding original test data is 2023-01-01 10:00). Through the method provided in the embodiment of the present application, the original test data can be processed to obtain a test data group, which is conducive to improving the efficiency of determining the evaluation data of the recommendation strategy.

[0082] It should be noted that the terminal device can simultaneously send the original playback data and the original test data to the data processing server, and the test object identification and timestamp information included in the original playback data and the original test data are the same. The process of the data processing server processing the original test data is similar to the process of processing the original playback data, and the corresponding playback data group and test data group can be obtained. Since the test object numbers in the playback data group and the test data group are the same, the target sharding node determined according to the test object number is also the same, that is, the playback data group and test data group corresponding to the same terminal device (or test object) will be stored in the same sharding node, that is, in a sharding node, the test data group in the test data table corresponds to a playback data group in the playback data table.

[0083] In one embodiment, the implementation method for determining the data blocks in the test data conversion table based on the test data table stored in the sharding node can be as follows: obtaining multiple test data groups in the test data table and processing them according to the statistical data time and the second timestamp information in each test data group to obtain multiple test data groups that meet the statistical data time; performing bitmap processing on the multiple test data groups that meet the statistical data time according to the test object number to obtain one or more bitmap data; and determining one or more data blocks based on each bitmap data and its corresponding recommendation strategy identifier and statistical data time. In actual application, the amount of original test data is very large. In order to save storage space and improve data query efficiency, the test data groups in the test data table can be processed to obtain data blocks in the test data conversion table, thereby significantly reducing the data scale. Specifically, the data processing server can obtain multiple test data groups in the test data table, and process them according to the statistical data time and the second timestamp information in each test data group to obtain multiple test data groups that meet the statistical data time. For example: if the statistical data time is "2023-01-01 10:00 to 2023-01-01 10:10", then the test data group whose time indicated by the second timestamp information is within the statistical data time can be determined as the test data group that meets the statistical data time.

[0084] The data processing server can perform bitmap processing on multiple test data groups that meet the statistical data time based on the test object number to obtain one or more bitmap data. For example, four test data groups that meet the statistical data time are determined, and the information format contained in the test data groups is "test object number, recommended strategy identifier, second timestamp information". The first test data group contains information "1, 1000, 2023-01-01 10:00", the second test data group contains information "2, 1000, 2023-01-01 10:01", the third test data group contains information "4, 1000, 2023-01-01 10:02", and the fourth test data group contains information "4, 1001, 2023-01-01 10:00". If the recommended strategy identifiers in the first, second, and third test data groups are the same, bitmap compression processing can be performed to obtain the first bitmap data; the fourth test data group can also be bitmap compressed to obtain the second bitmap data. One or more data blocks can be determined based on each bitmap data and its corresponding recommendation strategy identifier and statistical data time. For example, the first data block can be determined based on the first bitmap data, the recommendation strategy identifier "1000" and the statistical data time, and the second data block can be determined based on the second bitmap data, the recommendation strategy identifier "1001" and the statistical data time. It should be noted that the statistical data time can be adaptively adjusted according to actual application requirements and data processing efficiency, and the length of the statistical data time can be 10 minutes, 1 minute, or other time lengths. It should be noted that the storage space consumed by bitmap data is usually much smaller than the storage space consumed by the test data group. For example, the first test data group, the second test data group and the third test data group mentioned above can be bitmap compressed to obtain the first bitmap data. The test data group includes a test object identifier (usually occupies 4 bytes of storage space, i.e. 32 bits), a recommended strategy identifier (usually occupies 4 bytes of storage space, i.e. 32 bits) and a second timestamp information (usually occupies 8 bytes of storage space, i.e. 64 bits), and the first bitmap data obtained after bitmap compression processing can only occupy 3 bits.

[0085] In some cases, you can use the materialized view function of ClickHouse to convert the test data groups in the test data table to obtain a test data conversion table. Specifically, the materialization process can use the compression to bitmap state (groupBitmapState) function to complete the conversion of the test data table. Assume that the test data groups included in the test data table are as shown in Table 3 below.

[0086] Table 3:

[0087] Serial number uid exp id timestamp 1 0 1 2023-01-01 10:00 2 1 1 2023-01-01 10:01 3 2 1 2023-01-01 10:02 4 0 2 2023-01-01 10:05 5 1 1 2023-01-01 10:12

[0088] As shown in Table 3, the test data table includes 5 test data groups (numbered 1-5 respectively); it can be determined that the first statistical data time is "2023-01-01 10:00--2023-01-01 10:10" and the second statistical data time is "2023-01-01 10:10 to 2023-01-01 10:20". Then, using the data processing method provided by this application, it can be determined that the test data groups numbered 1-4 meet the first statistical data time, and the test data group numbered 5 meets the second statistical data time; among the test data groups numbered 1-4, the test data groups numbered 1-3 contain the same recommended strategy identifier (exp id), then the test data groups numbered 1-3 can be bitmap processed according to the test object number to obtain the corresponding bitmap data. Based on the test data table described in Table 3 above, the test data conversion table shown in Table 4 can be determined.

[0089] Table 4:

[0090]

[0091] As can be seen from Table 4 above, the test data conversion table includes three data blocks, wherein the recommended strategy identifier of the first data block is "1", the statistical data time is the first statistical time, and the bitmap data includes 6 storage spaces (the storage spaces are numbered 0-5, and each storage space can be a bit and can store one bit of data), and the data stored in storage spaces 0-2 is "1". This bitmap data indicates that the associated test data group corresponding to the test object number (i.e., the test data group numbered 1-3 in Table 3 above) was obtained within the first statistical data time, and the associated test data group has the same recommended strategy identifier as that included in the data block. Since the test object numbers in the test data groups numbered 1-3 in Table 3 above are 0, 1, and 2, respectively, these three test data groups can be corresponding to the storage spaces 0, 1, and 2 in the bitmap data of the first data block.

[0092] The recommended strategy identifier for the second data block is "2," the statistical data time is the first statistical time, the bitmap data includes six storage spaces, and the data stored in the storage space at bit 0 is "1." This bitmap data indicates that an associated test data group corresponding to the test object number (i.e., the test data group numbered 4 in Table 3) was obtained during the first statistical data time. This associated test data group has the same recommended strategy identifier as that included in the data block. Since the test object number in the test data group numbered 4 in Table 3 is 0, this test data group can be associated with the storage space at bit 0 in the bitmap data of the second data block.

[0093] The recommended strategy identifier of the third data block is "1", the statistical data time is the above-mentioned second statistical time, the bitmap data includes 6 storage spaces, and the data stored in the first storage space is "1", then the bitmap data indicates that the associated test data group corresponding to the test object number (that is, the test data group with serial number 5 in the above Table 3) is obtained within the second statistical data time, and the associated test data group is the same as the recommended strategy identifier included in the data block. Since the test object number in the test data group with serial number 5 in the above Table 3 is 1, this test data group can be corresponded to the first storage space in the bitmap data of the third data block. It can be seen from Tables 3 and 4 that by converting the test data table, a test data conversion table that occupies less storage space is obtained, which effectively saves storage space. Through the method provided in the embodiment of the present application, bitmap compression processing of the test data table can be realized, which is beneficial to reducing the consumption of storage space and saving resources. It is also beneficial to subsequent data processing according to the test data conversion table, improving data query efficiency, and thereby improving the efficiency of determining the evaluation data of the recommended strategy.

[0094] It should be noted that if the test object number of the test data group obtained in the test data table is large, offset data can be introduced to determine the correspondence between the test object number and the bit in the bitmap data based on the offset data, thereby realizing bitmap compression processing of the test data table. For example: now two test data groups are determined, the test object number in the first test data group is "10000", and the test object number in the second test data group is "10001", then the offset data can be determined to be 10000, and the storage space of the bitmap data corresponding to the first test data group is determined to be the 0th bit based on the offset data, and the storage space of the bitmap data corresponding to the second test data group is determined to be the 1st bit based on the offset data (that is, the data block generated based on these two test data groups includes bitmap data, and the data stored in the 0th and 1st storage spaces in the bitmap data is "1"). In addition, if multiple test data groups with the same test object number and recommended strategy identifier are determined within a statistical data time, they can be treated as one for conversion processing, thereby realizing further merging of test data and reducing the amount of data. For example, Table 3 above also includes a test data group with a serial number of 6, which includes the information "1, 1, 2023-01-01 10:13." Therefore, it can be determined that the test data group with a serial number of 5 and the test data group with a serial number of 6 are both in the second statistical data time, and their test object numbers and recommendation strategy identifiers are the same. Therefore, the data blocks determined based on the test data groups with serial numbers 5 and 6 are the same as the third data block in Table 4 above. The method provided in the embodiment of the present application can significantly compress the number of rows of test data, which is beneficial for saving storage resources, and is also beneficial for the subsequent calculation of evaluation data for the recommendation strategy, thereby improving data processing efficiency.

[0095] S203: Determine statistical data corresponding to a target recommendation strategy according to the associated playback data group and the associated data block.

[0096] In an embodiment of the present application, the associated playback data group can reflect the test subject's playback feedback on the test video during the test time, and the associated data block can reflect the recommendation strategy corresponding to the test subject during the test time. The data processing server can then combine the associated playback data group and the associated data block to determine the relationship between the recommendation strategy and the playback feedback, and determine the statistical data corresponding to the recommendation strategy. The statistical data can reflect the accuracy or quality of the recommendation strategy to a certain extent. The method provided in the embodiment of the present application can determine the statistical data of the target recommendation strategy, facilitate the subsequent determination of the evaluation data of the target recommendation strategy, and improve data processing efficiency.

[0097] In one embodiment, the implementation method for determining the statistical data corresponding to the target recommendation strategy based on the associated playback data group and the associated data block can be as follows: if the recommendation strategy identifier included in the associated data block is the recommendation strategy identifier corresponding to the target recommendation strategy, then the bitmap data included in the associated data block is queried based on the target test object number included in the associated playback data group; if the bitmap data included in the associated data block indicates that the associated test data group corresponding to the target test object number is obtained within the statistical data time, then it is determined that the associated playback data group meets the candidate condition, and the statistical data corresponding to the target recommendation strategy is determined based on the video playback data included in the associated playback data group. Assuming that the recommendation strategy identifier included in the associated data block is the recommendation strategy identifier corresponding to the target recommendation strategy, then the bitmap data included in the associated data block can be queried based on the target test object number included in the associated playback data group. If the bitmap data included in the associated data block indicates that the associated test data group corresponding to the target test object number is obtained within the statistical data time, then it can be determined that the associated playback test data group meets the candidate condition. The statistical data corresponding to the target recommendation strategy can be determined based on the video playback data included in the associated playback data group.

[0098] For example: the test time is "2023-01-01 10:00 to 2023-01-01 10:10", the recommendation strategy identifier of the target recommendation strategy is "1", and the associated data blocks obtained by the data processing server from the shard node are shown in Table 5 below.

[0099] Table 5:

[0100]

[0101] In Table 5, the first data statistical time is "2023-01-01 10:00 to 2023-01-01 10:10". The associated playback data group obtained by the data processing server from the same shard node is shown in Table 6 below.

[0102] Table 6:

[0103]

[0104] Then, the bitmap data included in the associated data block shown in Table 5 can be queried based on the target test object number included in the associated playback data group shown in Table 6. The bitmap data included in the associated data block shown in Table 5 indicates that the associated test data groups with test object numbers 0 and 1 are obtained within the statistical data time. It can be determined that the associated playback data groups with serial numbers 1 and 2 in Table 6 meet the candidate conditions, and the statistical data corresponding to the recommendation strategy with the recommendation strategy identifier "1" can be determined based on the associated playback data groups with serial numbers 1 and 2. Through the method provided in the embodiment of the present application, the data in the playback data table and the data blocks in the test data conversion table can be combined to comprehensively determine the statistical data of the recommendation strategy, thereby ensuring the accuracy of the statistical data and helping to improve the accuracy of the evaluation data of the target recommendation strategy.

[0105] It should be noted that the method provided in this application can use bitmap data to implement query processing, rather than using the test data group in the test data table for query processing. Since the bitmap data is short and the data volume is small, the data query efficiency is very high, which can further improve the data processing efficiency.

[0106] In one embodiment, there are multiple associated playback data groups, and the implementation method for determining the statistical data corresponding to the target recommendation strategy based on the video playback data included in the associated playback data groups can be as follows: determining the number of data groups of the associated playback data groups that meet the candidate conditions in the multiple associated playback data groups; determining the total playback data based on the video playback data included in each associated playback data group that meets the candidate conditions; determining the number of test objects based on the test object numbers included in each associated playback data group that meets the candidate conditions; and determining the statistical data corresponding to the target recommendation strategy based on the number of data groups, the total playback data, and the number of test objects. Assuming that the data processing server determines multiple associated playback data groups associated with the test time from the playback data table, the number of associated playback data groups that meet the candidate conditions in the multiple associated playback data groups can be determined. This number can reflect the number of terminal devices that sent original playback data to the data processing server during the test time, and further can reflect the number of test videos determined by the target recommendation strategy played according to the instructions of the test object during the test time.

[0107] The data processing server can determine the playback aggregate data based on the video playback data included in each associated playback data group that meets the candidate conditions. The playback aggregate data can reflect, to a certain extent, the degree to which the test object receives the test video determined by the target recommendation strategy. The number of test objects can be determined based on the test object numbers included in each associated playback data group that meets the candidate conditions. The number of test objects can indicate how many terminal devices corresponding to the test objects have sent original playback data (the original playback data is associated with the test video determined by the target recommendation strategy) to the data processing server during the test time, thereby reflecting how many terminal devices corresponding to the test objects have received the test video determined according to the target recommendation strategy. The data processing server can determine the statistical data corresponding to the target recommendation strategy based on the number of data groups, the playback aggregate data, and the number of test objects. Through the method provided in the embodiment of the present application, the statistical data of the target recommendation strategy can be determined using the associated playback data groups that meet the candidate conditions and the bitmap data in the associated data blocks, which is conducive to determining accurate evaluation data based on statistical data and improving data processing efficiency.

[0108] In one embodiment, the video playback data includes the video playback time, and the playback total data includes the playback time total data; the implementation method of determining the statistical data corresponding to the target recommendation strategy based on the number of data groups, the playback total data, and the number of test objects can be: calculating the number of data groups and the number of test objects to obtain the average number of playbacks, and the average number of playbacks is used to indicate the average number of times the test video determined according to the target recommendation strategy is played in the terminal device corresponding to the test object; calculating the playback total data and the number of test objects to obtain the average playback time, and the average playback time is used to indicate the average time the test video determined according to the target recommendation strategy is played in the terminal device corresponding to the test object; determining the statistical data corresponding to the target recommendation strategy based on the average number of playbacks and the average playback time. When the video playback data includes the video playback duration, and the playback aggregate data includes the playback duration aggregate data, the number of data groups and the number of test objects can be calculated (for example, by dividing the number of data groups by the number of test objects) to obtain the average number of playbacks, which can be used to indicate the average number of times the test video determined according to the target recommendation strategy is played in the terminal device corresponding to the test object; the playback aggregate data and the number of test objects can be calculated (for example, by using the playback duration aggregate data in the playback aggregate data in the number of test objects) to obtain the average playback duration, which can be used to indicate the average duration of the test video determined according to the target recommendation strategy is played in the terminal device corresponding to the test object. The data processing server can determine the statistical data corresponding to the target recommendation strategy based on the average number of playbacks and the average playback duration. Through the method provided in the embodiment of the present application, the average number of playbacks and the average playback duration can be determined, thereby determining the statistical data corresponding to the target recommendation strategy, which is conducive to the subsequent determination of accurate evaluation data for the target recommendation strategy based on the statistical data.

[0109] See Figure 4 , which is a schematic diagram of a method for determining average playback duration provided by an embodiment of the present application. The data processing server can obtain the following information from the playback data group of the shard node: Figure 4 The four associated playback data groups associated with the test time (2023-01-01 10:00 to 2023-01-01 10:10) are numbered 1-4. The four associated playback data groups all include the test object number, video playback duration, and first timestamp information. The data processing server can obtain the following information from the same shard node: Figure 4The associated data block associated with the test time shown in the figure has a recommendation strategy identifier of "1" and a statistical data time of the first statistical data time (the first statistical data time may include the test time or may be the same as the test time). Assuming that the recommendation strategy identifier of the target recommendation strategy is 1, the four play data groups are processed in sequence.

[0110] Specifically, the bitmap data of the associated data block can be queried according to the test object number (1) of the playback data group with sequence number 1 (i.e., the first storage space in the bitmap data is queried, and the value stored in the storage space is "1"). At this time, the test object number matches the value stored in the bitmap data, then the number of test objects = 0+1, the total playback time data = 0+the video playback time (10) in the playback data group with sequence number 1; the bitmap data of the associated data block can be queried according to the test object number (2) of the playback data group with sequence number 2 (i.e., the second storage space in the bitmap data is queried, and the value stored in the storage space is "1"). At this time, the test object number matches the value stored in the bitmap data, then the number of test objects = 1+1, the total playback time data = 10+sequence number The video playback duration (5) in the playback data group with the sequence number 2 can be queried according to the test object number (3) of the playback data group with the sequence number 3 (i.e., the third storage space in the bitmap data is queried, and the value stored in the storage space is "0"). At this time, the test object number does not match the value stored in the bitmap data, and the corresponding calculation is not performed; the bitmap data of the associated data block can be queried according to the test object number (4) of the playback data group with the sequence number 4 (i.e., the fourth storage space in the bitmap data is queried, and the value stored in the storage space is "1"). At this time, the test object number matches the value stored in the bitmap data, and the number of test objects = 2 + 1, and the total playback duration data = 15 + the video playback duration (3) in the playback data group with the sequence number 4.

[0111] After completing the processing of the above four playback data groups, it can be determined that the number of test objects is 3 and the total playback time data is 18. Then, based on the number of test objects and the total playback time data, the average playback time can be determined to be 18 / 3=6, that is, the average playback time corresponding to the recommendation strategy with the recommendation strategy identifier of "1" during the test time is 6 (the unit of the average playback time can be seconds). Through the method provided in the embodiment of the present application, the average playback time of the target recommendation strategy can be determined, which can reflect the effect of the target recommendation strategy and is conducive to the subsequent determination of the evaluation data of the target recommendation strategy based on the average playback time.

[0112] It should be noted that the data processing server can determine the statistical data of the target recommendation strategy based on the test time, or it can determine the statistical data of the target recommendation strategy based on the number of playback data groups in the playback data table. For example: when the accumulated data in the playback data table in the sharding node reaches a first number, the data processing server can obtain the first number of playback data groups from the playback data table, and process them according to the first number of playback data groups and the test data conversion table in the same sharding node to obtain the statistical data of the target recommendation strategy.

[0113] See Figure 5 , which is a schematic diagram of a method for determining the average number of plays provided by an embodiment of the present application. The data processing server can obtain the following information from the play data group of the shard node: Figure 5 The five associated playback data groups associated with the test time (2023-01-01 10:00 to 2023-01-01 10:10) are numbered 1-5. Each of the five associated playback data groups includes the test object number, video playback duration, and first timestamp information. The data processing server can obtain the following information from the same shard node: Figure 5 The associated data block associated with the test time is shown, the recommendation strategy identifier in the associated data block is "1", and the statistical data time is the first statistical data time. Assuming that the recommendation strategy identifier of the target recommendation strategy is 1, the four play data groups are processed in sequence.

[0114] Specifically, the bitmap data of the associated data block can be queried according to the test object number (1) of the playback data group with the serial number 1 (i.e., the first storage space in the bitmap data is queried, and the value stored in the storage space is "1"). At this time, the test object number matches the value stored in the bitmap data, and the number of test objects = 0+1, and the number of data groups = 0+1; the bitmap data of the associated data block can be queried according to the test object number (2) of the playback data group with the serial number 2 (i.e., the second storage space in the bitmap data is queried, and the value stored in the storage space is "1"). At this time, the test object number matches the value stored in the bitmap data, and the number of test objects = 1+1, and the number of data groups = 1+1; the bitmap data of the associated data block can be queried according to the test object number (3) of the playback data group with the serial number 3 (i.e., the third storage space in the bitmap data is queried, and the value stored in the storage space is The value is "1"), at this time, the test object number matches the value stored in the bitmap data, then the number of test objects = 2+1, the number of data groups = 2+1; the bitmap data of the associated data block can be queried according to the test object number (1) of the playback data group with sequence number 4 (i.e., query the first storage space in the bitmap data, the value stored in the storage space is "1"). Since the number of test objects corresponding to the first storage space has been calculated before, the test object number matches the value stored in the bitmap data at this time, then the number of test objects = 3, the number of data groups = 3+1; the bitmap data of the associated data block can be queried according to the test object number (4) of the playback data group with sequence number 5 (i.e., query the fourth storage space in the bitmap data, the value stored in the storage space is "0"). At this time, the test object number does not match the value stored in the bitmap data, and the corresponding calculation is not performed.

[0115] After completing the processing of the above five playback data groups, it can be determined that the number of test objects is 3 and the number of data groups is 4. Then, based on the number of test objects and the number of data groups, the average number of playbacks can be determined to be 4 / 3=1.3, that is, the average number of playbacks corresponding to the recommendation strategy identified as "1" during the test time is 1.3. Through the method provided in the embodiment of the present application, the average number of playbacks of the target recommendation strategy can be determined, which can reflect the effectiveness of the target recommendation strategy and is conducive to the subsequent determination of evaluation data for the target recommendation strategy based on the average number of playbacks.

[0116] S204: Determine evaluation data of the target recommendation strategy corresponding to the shard node within the test time based on the statistical data.

[0117] In an embodiment of the present application, the data processing server can determine the evaluation data of the target recommendation strategy corresponding to the sharding node within the test time based on statistical data. The evaluation data can reflect the recommendation effect of the target recommendation strategy within the test time to a certain extent, that is, the acceptance of the test video determined by the target recommendation strategy by the test subject. The evaluation data can be used to indicate adjustments to the target recommendation strategy, thereby improving the accuracy of the target recommendation strategy and enhancing the recommendation effect of the target recommendation strategy. Through the method provided in the embodiment of the present application, the evaluation data of the target recommendation strategy can be determined, which is conducive to optimizing the target recommendation strategy and thus improving the user experience.

[0118] See Figure 6 , which is a schematic diagram of another data processing method provided in an embodiment of the present application. Figure 6 In the embodiment of the present invention, the terminal device can play a test video (the test video is determined according to the recommendation strategy) according to the instruction of the test object, and send the original play data and the original test data to the data processing server. The data processing server sends the original play data to the play data queue and the original test data to the test data queue. The data processing server can obtain data from the play data queue and the test data queue respectively for data processing to obtain a play data group and a test data group. The data processing server can determine the target slice node from multiple slice nodes according to the test object number in the play data group and the test data group, and store the play data group in the play data table of the target slice node and the test data group in the test data table of the target slice node. In order to further save storage resources, the data processing server can convert the test data table to obtain a test data conversion table with a smaller data volume. The test data conversion table includes multiple data blocks, each of which includes a recommendation strategy identifier, statistical data time and bitmap data. The data processing server can perform data processing based on the test data conversion table and the play data table to obtain statistical data of the target recommendation strategy within the test time, and can determine the evaluation data of the target recommendation strategy based on the statistical data. The method provided in the embodiment of the present application can effectively save storage resources, and at the same time, data can be obtained from the same shard node for processing, which effectively improves data processing efficiency and is conducive to improving the update efficiency of the recommendation strategy.

[0119] Through the data processing method provided in the embodiment of the present application, associated test data groups and associated data blocks can be obtained from the same shard node for data processing, avoiding resource consumption caused by obtaining data across shard nodes and improving the efficiency of data processing; the test data table containing the test data group can be converted and processed to obtain a test data conversion table with a small data volume, effectively saving storage resources; data can be obtained and processed according to the test time to obtain evaluation data of the recommendation strategy, which is conducive to using the evaluation data to adjust the recommendation strategy, further improving the accuracy of the recommendation strategy, so that the video determined by the recommendation strategy can better meet the video needs of the object and enhance the user experience; when the test time is short, the data processing method provided in the embodiment of the present application can realize real-time evaluation of the recommendation strategy, effectively improving the update efficiency of the recommendation strategy.

[0120] See Figure 7 , Figure 7 This is a flow chart of another data processing method provided in an embodiment of the present application. This data processing method can be implemented by the above-mentioned data processing server 102 or by other devices. The process of the data processing method provided in the embodiment of the present application includes but is not limited to:

[0121] S701. Obtain an associated playback data group associated with the test time from a playback data table stored in a sharding node; the playback data table includes multiple playback data groups, and the playback data group includes a test object number and video playback data; the sharding node matches the test object number, and the video playback data is playback data generated by playing a test video according to an instruction of the test object, and the test video is a video recommended to the test object corresponding to the test object number according to a recommendation strategy, and the sharding node is any one of the multiple sharding nodes corresponding to the test database.

[0122] In an embodiment of the present application, the data processing server may include a plurality of sharding nodes, which are obtained by dividing the storage space of the data processing server, and each sharding node may store a corresponding playback data table. The playback data table in the sharding node may include a plurality of playback data groups, each playback data group may include a test object number and video playback data, wherein the test object number matches the sharding node, and the video playback data is the playback data generated by playing the test video according to the instruction of the test object, and the test video is the video recommended to the test object corresponding to the test object number according to the recommendation strategy. The data processing server may obtain the associated playback data group associated with the test time from the playback data table stored in the sharding node, and the test time is the evaluation time for the recommendation strategy determined according to the application requirements, for example: if the test time is 1 minute, the data processing server may adopt the data processing method provided in the embodiment of the present application to determine the evaluation data of the target recommendation strategy every minute, thereby realizing the real-time evaluation of the target recommendation strategy. By the method provided in the embodiment of the present application, the associated playback data group can be determined, which is conducive to the subsequent determination of the evaluation data of the target recommendation strategy based on the associated playback data group.

[0123] In one embodiment, the data processing method provided by the present application can also: receive a video acquisition request sent by a terminal device, and determine the object description information of the test object based on the test object identifier included in the video acquisition request, the video acquisition request is used to request to obtain a video that meets the video requirements of the test object, and the object description information includes the video requirement feature information of the test object; determine a target recommendation strategy from multiple recommendation strategies, and use the target recommendation strategy to process the video requirement feature information to obtain a recommended video to be played; and send the recommended video to be played to the terminal device. Through the method provided in the embodiment of the present application, the recommended video to be played determined according to the recommendation strategy can be sent to the terminal device, which is conducive to the subsequent adjustment of the recommendation strategy based on the playback data sent by the terminal device, and is conducive to improving the accuracy of the recommendation strategy.

[0124] In one embodiment, the method provided by the embodiment of the present application further includes: obtaining original playback data from the playback data queue, and performing data extraction processing on the original playback data to obtain a test object identifier, video playback data and a first timestamp information, wherein the original playback data is sent by the terminal device corresponding to the test object at the time indicated by the first timestamp information; if there is a matching number that matches the test object identifier in the number mapping table, the matching number is determined to be the test object number corresponding to the test object identifier; if there is no matching number that matches the test object identifier in the number mapping table, a test object number is generated according to the test object identifier, and the test object identifier and the test object number are added to the number mapping table; a playback data group is generated according to the test object number, the video playback data and the first timestamp information; a target shard node is determined from a plurality of shard nodes corresponding to the test database according to the test object number, and the playback data group is stored in the target shard node. Through the method provided by the embodiment of the present application, the original playback data can be processed to obtain a playback data group, which is conducive to improving the efficiency of determining the evaluation data of the recommendation strategy.

[0125] S702. Obtain an associated data block associated with the test time from the test data conversion table stored in the sharding node; the test data conversion table includes multiple data blocks, and the data blocks include a recommended strategy identifier, statistical data time, and bitmap data; the test data conversion table is determined based on the test data table stored in the sharding node, and the test data table includes multiple test data groups, and the test data group includes a test object number and a recommended strategy identifier; the test data group is associated with a playback data group in the playback data table; the bitmap data is used to indicate: whether an associated test data group corresponding to the test object number is obtained within the statistical data time, and the associated test data group is the same as the recommended strategy identifier included in the data block.

[0126] In an embodiment of the present application, a sharding node includes a playback data table and a test data conversion table. The data processing server can obtain an associated data block associated with the test time from the test data conversion table of the same sharding node. The test data conversion table may include multiple data blocks, each of which may include a recommendation strategy identifier, statistical data time, and bitmap data, wherein the recommendation strategy identifier is used to indicate a specific recommendation strategy, the statistical data time is used to indicate the time of the statistical test data group, and the bitmap data is used to indicate whether the associated test data group corresponding to the test object number is obtained within the statistical data time, and the associated test data group is the same as the recommendation strategy identifier included in the data block.

[0127] The test data conversion table is determined based on the test data table stored in the sharding node. The test data table can include multiple test data groups, each of which includes a test object number and a recommended strategy identifier. Any test data group can be associated with a playback data group in the playback data table of the same sharding node. The method provided in the embodiments of the present application can obtain associated data blocks associated with the test time, which facilitates the subsequent use of the associated data blocks to determine evaluation data for the recommended strategy, thereby improving the efficiency of determining the evaluation data.

[0128] S703: Determine statistical data corresponding to a target recommendation strategy according to the associated playback data group and the associated data block.

[0129] In an embodiment of the present application, the associated playback data group can reflect the test subject's playback feedback on the test video during the test time, and the associated data block can reflect the recommendation strategy corresponding to the test subject during the test time. The data processing server can then combine the associated playback data group and the associated data block to determine the relationship between the recommendation strategy and the playback feedback, and determine the statistical data corresponding to the recommendation strategy. The statistical data can reflect the accuracy or quality of the recommendation strategy to a certain extent. The method provided in the embodiment of the present application can determine the statistical data of the target recommendation strategy, facilitate the subsequent determination of the evaluation data of the target recommendation strategy, and improve data processing efficiency.

[0130] S704: Determine evaluation data of the target recommendation strategy corresponding to the shard node within the test time based on the statistical data.

[0131] In an embodiment of the present application, the data processing server can determine the evaluation data of the target recommendation strategy corresponding to the sharding node within the test time based on statistical data. The evaluation data can reflect the recommendation effect of the target recommendation strategy within the test time to a certain extent, that is, the acceptance of the test video determined by the target recommendation strategy by the test subject. The evaluation data can be used to indicate adjustments to the target recommendation strategy, thereby improving the accuracy of the target recommendation strategy and enhancing the recommendation effect of the target recommendation strategy. Through the method provided in the embodiment of the present application, the evaluation data of the target recommendation strategy can be determined, which is conducive to optimizing the target recommendation strategy and thus improving the user experience.

[0132] S705. After determining the evaluation data of the target recommendation strategy corresponding to each of the shard nodes within the test time, determine the overall evaluation data of the target recommendation strategy within the test time based on the evaluation data of the target recommendation strategy corresponding to each of the shard nodes within the test time.

[0133] In the embodiment of the present application, after processing the original playback data and the original test data, the data processing server can obtain the playback data group and the test data group, and determine the target sharding node based on the test object number contained in the playback data group and the test data group, and then store the playback data group and the test data group in the target sharding node. Therefore, for the data sent by the terminal device during the test time, the data processing server may store the data in any one of the multiple sharding nodes after processing. Therefore, in order to comprehensively evaluate the recommendation effect of the target recommendation strategy during the test time, the above method can be used to process the data stored in each sharding node in the data processing server. After determining the evaluation data of the target recommendation strategy during the test time corresponding to each sharding node in the test database, the data processing server can determine the overall evaluation data of the target recommendation strategy during the test time based on the evaluation data of the target recommendation strategy during the test time corresponding to each sharding node. The overall evaluation data can comprehensively reflect the recommendation effect of the target recommendation strategy during the test time. Through the method provided in the embodiment of the present application, a comprehensive evaluation of the recommendation strategy can be achieved, which is conducive to improving the accuracy and recommendation effect of the recommendation strategy.

[0134] In one embodiment, the data processing server can determine the adjustment information of the target recommendation strategy based on the overall evaluation data, and adjust the target recommendation strategy based on the strategy adjustment information, thereby improving the accuracy and recommendation effect of the target recommendation strategy, so that the video determined using the adjusted target recommendation strategy is more in line with the video needs of the object, which is conducive to improving the user experience.

[0135] See Figure 8 , this figure is a schematic diagram of an evaluation data determination method provided by an embodiment of the present application. The data processing server includes a test database, and the test database includes multiple shard nodes, each of which stores a playback data table and a test data conversion table. For any one of the multiple shard nodes, the data processing server can perform data processing based on the playback data table, test data conversion table and test time in the shard node, and obtain the evaluation data of the target recommendation strategy corresponding to the shard node within the test time (for example: the time length indicated by the test time can be 10 minutes or 1 minute). The data processing server can perform similar operations based on the data stored in each shard node, thereby obtaining the evaluation data of the target recommendation strategy corresponding to each shard node within the test time (such as Figure 8As shown, according to the data stored in each sharding node, the evaluation data of the target recommendation strategy corresponding to sharding node 1, the evaluation data of the target recommendation strategy corresponding to sharding node 2, ..., the evaluation data of the target recommendation strategy corresponding to sharding node N can be determined). The data processing server can use the evaluation data of the target recommendation strategy corresponding to each sharding node within the test time for processing, and obtain the overall evaluation data of the target recommendation strategy within the test time (for example: according to the evaluation data of the target recommendation strategy corresponding to each sharding node within the test time, the mean is calculated to obtain the average evaluation data, and the average evaluation data is determined as the overall evaluation data). The data processing server can generate policy adjustment information based on the overall evaluation data, and the policy adjustment information can be used to indicate that the target recommendation strategy is adjusted, thereby improving the accuracy of the target recommendation strategy. The method provided by the embodiment of the present application can realize the determination of the evaluation data of the recommendation strategy every minute, that is, the calculation delay of the evaluation data of the recommendation strategy is shortened to the minute level, effectively improving the data processing efficiency, and also effectively improving the update efficiency of the recommendation strategy.

[0136] Through the data processing method provided in the embodiment of the present application, associated test data groups and associated data blocks can be obtained from the same shard node for data processing, avoiding resource consumption caused by obtaining data across shard nodes and improving data processing efficiency; the test data table containing the test data group can be converted and processed to obtain a test data conversion table with a small data volume, effectively saving storage resources; data can be obtained and processed according to the test time to obtain evaluation data of the recommendation strategy, which is conducive to adjusting the recommendation strategy using the evaluation data, further improving the accuracy of the recommendation strategy, so that the video determined by the recommendation strategy can better meet the video needs of the object and enhance the user experience; when the time length indicated by the test time is at the minute level, the data processing method provided in the embodiment of the present application can realize the determination of the evaluation data of the recommendation strategy every minute, that is, shortening the calculation delay of the evaluation data of the recommendation strategy to the minute level, effectively improving the data processing efficiency, and can also effectively recommend the update efficiency of the strategy, which is conducive to quickly improving the user experience.

[0137] See Figure 9 , Figure 9 This is a block diagram of a data processing device provided in an embodiment of the present application. The device includes:

[0138] An acquisition unit 901 is configured to acquire, from a playback data table stored in a sharding node, an associated playback data group associated with a test time; the playback data table includes multiple playback data groups, each of which includes a test subject number and video playback data; the sharding node matches the test subject number, the video playback data is playback data generated by playing a test video according to an instruction of the test subject, and the test video is a video recommended to the test subject corresponding to the test subject number according to a recommendation strategy;

[0139] The acquisition unit 901 is further configured to acquire, from a test data conversion table stored in the sharding node, an associated data block associated with the test time; the test data conversion table includes a plurality of data blocks, each of which includes a recommendation strategy identifier, a statistical data time, and bitmap data; the test data conversion table is determined based on a test data table stored in the sharding node, the test data table including a plurality of test data groups, each of which includes a test object number and a recommendation strategy identifier; the test data group is associated with a playback data group in the playback data table; the bitmap data is used to indicate whether an associated test data group corresponding to the test object number is acquired within the statistical data time, the associated test data group being the same as the recommendation strategy identifier included in the data block;

[0140] A determining unit 902 is configured to determine statistical data corresponding to a target recommendation strategy based on the associated playback data group and the associated data block;

[0141] The processing unit 903 is configured to determine evaluation data of the target recommendation strategy corresponding to the shard node within the test time based on the statistical data.

[0142] In one embodiment, when determining the statistical data corresponding to the target recommendation strategy based on the associated playback data group and the associated data block, the determination unit 902 is specifically used to: if the recommendation strategy identifier included in the associated data block is the recommendation strategy identifier corresponding to the target recommendation strategy, then query the bitmap data included in the associated data block based on the target test object number included in the associated playback data group; if the bitmap data included in the associated data block indicates that the associated test data group corresponding to the target test object number is obtained within the statistical data time, then determine that the associated playback data group meets the candidate condition, and determine the statistical data corresponding to the target recommendation strategy based on the video playback data included in the associated playback data group.

[0143] In one embodiment, there are multiple associated playback data groups, and when the determination unit 902 determines the statistical data corresponding to the target recommendation strategy based on the video playback data included in the associated playback data groups, it is specifically used to: determine the number of data groups of the associated playback data groups that meet the candidate conditions in the multiple associated playback data groups; determine the total playback data based on the video playback data included in each associated playback data group that meets the candidate conditions; determine the number of test objects based on the test object numbers included in each associated playback data group that meets the candidate conditions; determine the statistical data corresponding to the target recommendation strategy based on the number of data groups, the total playback data, and the number of test objects.

[0144] In one embodiment, the video playback data includes video playback time, and the playback aggregate data includes playback time aggregate data; when the determination unit 902 determines the statistical data corresponding to the target recommendation strategy based on the number of data groups, the playback aggregate data, and the number of test objects, it is specifically used to: calculate the number of data groups and the number of test objects to obtain an average number of playbacks, and the average number of playbacks is used to indicate the average number of times the test video determined according to the target recommendation strategy is played in the terminal device corresponding to the test object; calculate the playback aggregate data and the number of test objects to obtain an average playback time, and the average playback time is used to indicate the average time the test video determined according to the target recommendation strategy is played in the terminal device corresponding to the test object; determine the statistical data corresponding to the target recommendation strategy based on the average number of playbacks and the average playback time.

[0145] In one embodiment, the shard node is any one of the multiple shard nodes corresponding to the test database, and the processing unit 903 is further used to: after determining the evaluation data of the target recommendation strategy corresponding to each of the shard nodes within the test time, determine the overall evaluation data of the target recommendation strategy within the test time based on the evaluation data of the target recommendation strategy corresponding to each of the shard nodes within the test time.

[0146] In one embodiment, the acquisition unit 901 is further used to: acquire original playback data from a playback data queue, and perform data extraction processing on the original playback data to obtain a test object identifier, video playback data, and a first timestamp information, wherein the original playback data is sent by a terminal device corresponding to the test object at the time indicated by the first timestamp information; if there is a matching number matching the test object identifier in the number mapping table, the matching number is determined as the test object number corresponding to the test object identifier; if there is no matching number matching the test object identifier in the number mapping table, a test object number is generated according to the test object identifier, and the test object identifier and the test object number are added to the number mapping table; a playback data group is generated according to the test object number, the video playback data, and the first timestamp information; a target shard node is determined from a plurality of shard nodes corresponding to the test database according to the test object number, and the playback data group is stored in the target shard node.

[0147] In one embodiment, the acquisition unit 901 is further used to: receive a video acquisition request sent by a terminal device, and determine the object description information of the test object based on the test object identifier included in the video acquisition request, wherein the video acquisition request is used to request acquisition of a video that meets the video requirements of the test object, and the object description information includes the video requirement feature information of the test object; determine a target recommendation strategy from multiple recommendation strategies, and use the target recommendation strategy to process according to the video requirement feature information to obtain a recommended video to be played; and send the recommended video to be played to the terminal device.

[0148] It is to be understood that the functions of the various functional units of the data processing apparatus of the embodiment of the present application can be specifically implemented according to the data processing method in the above-mentioned method embodiment, and its specific implementation process can refer to the relevant description in the above-mentioned data processing method embodiment, which will not be repeated here. In the embodiment of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuit or memory) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the module or unit function.

[0149] Through the data processing device provided by the embodiment of the present application, associated test data groups and associated data blocks can be obtained from the same shard node for data processing, avoiding resource consumption caused by obtaining data across shard nodes and improving data processing efficiency; the test data table containing the test data group can be converted and processed to obtain a test data conversion table with a small data volume, effectively saving storage resources; data can be obtained and processed according to the test time to obtain evaluation data of the recommendation strategy, which is conducive to adjusting the recommendation strategy using the evaluation data, further improving the accuracy of the recommendation strategy, so that the video determined by the recommendation strategy can better meet the video needs of the object and enhance the user experience; when the time length indicated by the test time is at the minute level, the data processing method provided by the embodiment of the present application can realize the determination of the evaluation data of the recommendation strategy every minute, that is, shortening the calculation delay of the evaluation data of the recommendation strategy to the minute level, effectively improving the data processing efficiency, and can also effectively recommend the update efficiency of the strategy, which is conducive to quickly improving the user experience.

[0150] See Figure 10 , Figure 10 A structural block diagram of a computer device provided in an embodiment of the present application. Figure 10 The computer device shown can be the above Figure 1 The data processing server 102 shown in FIG. Figure 1 The computer device described in the embodiment of the present application includes: a processor 1001, a communication interface 1002, and a memory 1003. The processor 1001, the communication interface 1002, and the memory 1003 can be connected via a bus or other means. The embodiment of the present application uses the bus connection as an example.

[0151] Processor 1001 (also known as the CPU (Central Processing Unit)) is the computing and control core of the computer device. It can interpret various instructions within the computer device and process various data within the computer device. For example, the CPU can interpret power on / off commands sent by the user to the computer device and control the computer device to perform power on / off operations. Another example is that the CPU can transmit various interactive data between the internal components of the computer device, etc. Communication interface 1002 can optionally include a standard wired interface or a wireless interface (such as Wi-Fi or mobile communication interface), which is controlled by processor 1001 to send and receive data. Memory 1003 (Memory) is a memory device within the computer device that is used to store computer instructions and data. It is understood that memory 1003 here can include both the computer device's built-in memory and, of course, the computer device's supported extended memory. Memory 1003 provides storage space that stores the computer device's operating system, which may include but is not limited to Android, iOS, Windows Phone, etc., although this application is not limited to this. In the embodiment of the present application, the processor 1001 executes the steps in the various method embodiments of the present application by running the computer instructions in the memory 1003 to implement the data processing method provided in the embodiment of the present application.

[0152] In a specific implementation, the processor 1001, communication interface 1002 and memory 1003 described in the embodiment of the present application can execute the implementation method of the data processing server described in a data processing method provided in an embodiment of the present application, and can also execute the implementation method described in a data processing device provided in an embodiment of the present application, which will not be repeated here.

[0153] The present application also provides a computer-readable storage medium having computer instructions stored therein. When executed on a computer device, the computer device executes the steps of each method embodiment of the present application to implement the data processing method provided in the present application. The specific implementation method can be found in the above description and will not be repeated here.

[0154] The present application also provides a computer program product, which includes a computer program or computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, causing the computer device to perform the steps of each method embodiment of the present application to implement the data processing method provided in the embodiment of the present application. The specific implementation method can be referred to the above description and will not be repeated here.

[0155] It should be noted that for the aforementioned various method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0156] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a computer, a server or a network device, etc., specifically a processor in a computer device) to execute all or part of the steps of the above methods in each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (English: Read-Only Memory, abbreviated: ROM) or random access memory (English: Random Access Memory, abbreviated: RAM) and other media that can store program codes.

[0157] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, it should be understood that the technical solutions recorded in the above embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Obtaining an associated playback data group associated with the test time from a playback data table stored in a sharding node; the playback data table includes a plurality of playback data groups, each of which includes a test subject number and video playback data; the sharding node matches the test subject number, the video playback data is playback data generated by playing a test video according to an instruction of the test subject, and the test video is a video recommended to the test subject corresponding to the test subject number according to a recommendation strategy; Obtaining an associated data block associated with the test time from a test data conversion table stored in the sharding node; the test data conversion table includes a plurality of data blocks, each of which includes a recommended strategy identifier, a statistical data time, and bitmap data; the test data conversion table is determined based on a test data table stored in the sharding node, the test data table includes a plurality of test data groups, each of which includes a test object number and a recommended strategy identifier; the test data group is associated with a play data group in the play data table; the bitmap data is used to indicate whether an associated test data group corresponding to the test object number is obtained within the statistical data time, the associated test data group being the same as the recommended strategy identifier included in the data block; Determining statistical data corresponding to a target recommendation strategy based on the associated playback data group and the associated data block; Determine evaluation data of the target recommendation strategy corresponding to the shard node within the test time based on the statistical data.

2. The method according to claim 1, characterized in that The determining of statistical data corresponding to the target recommendation strategy according to the associated playback data group and the associated data block includes: If the recommendation strategy identifier included in the associated data block is the recommendation strategy identifier corresponding to the target recommendation strategy, querying the bitmap data included in the associated data block according to the target test object number included in the associated playback data group; If the bitmap data included in the associated data block indicates that the associated test data group corresponding to the target test object number is obtained within the statistical data time, it is determined that the associated playback data group meets the candidate condition, and the statistical data corresponding to the target recommendation strategy is determined based on the video playback data included in the associated playback data group.

3. The method according to claim 2, characterized in that There are multiple associated playback data groups, and determining the statistical data corresponding to the target recommendation strategy based on the video playback data included in the associated playback data groups includes: Determining the number of associated playback data groups that meet the candidate condition among the multiple associated playback data groups; Determining playback aggregate data based on the video playback data included in each associated playback data group that meets the candidate condition; Determine the number of test objects according to the test object numbers included in each associated playback data group that meets the candidate condition; The statistical data corresponding to the target recommendation strategy is determined based on the number of data groups, the total playback data, and the number of test subjects.

4. The method according to claim 3, characterized in that The video playback data includes video playback duration, and the playback total data includes playback duration total data; The determining of statistical data corresponding to the target recommendation strategy based on the number of data groups, the total playback data, and the number of test subjects includes: Calculating the number of data groups and the number of test subjects to obtain an average number of play times, where the average number of play times indicates the average number of times the test video determined according to the target recommendation strategy is played in a terminal device corresponding to the test subject; Calculating the total playback data and the number of test subjects to obtain an average playback duration, where the average playback duration is used to indicate, in a terminal device corresponding to the test subject, an average duration of playing the test video determined according to the target recommendation strategy; Statistical data corresponding to the target recommendation strategy is determined based on the average number of playback times and the average playback duration.

5. The method according to any one of claims 1 to 4, characterized in that The shard node is any one of a plurality of shard nodes corresponding to the test database, and the method further includes: After determining the evaluation data of the target recommendation strategy corresponding to each of the shard nodes within the test time, the overall evaluation data of the target recommendation strategy within the test time is determined based on the evaluation data of the target recommendation strategy corresponding to each of the shard nodes within the test time.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtaining original playback data from a playback data queue and performing data extraction processing on the original playback data to obtain a test object identifier, video playback data, and first timestamp information, wherein the original playback data is sent by a terminal device corresponding to the test object at a time indicated by the first timestamp information; If a matching number matching the test object identifier exists in the number mapping table, determining the matching number as the test object number corresponding to the test object identifier; If there is no matching number matching the test object identifier in the number mapping table, generating a test object number according to the test object identifier, and adding the test object identifier and the test object number to the number mapping table; Generate a playback data group according to the test object number, the video playback data and the first timestamp information; A target shard node is determined from a plurality of shard nodes corresponding to a test database according to the test object number, and the playback data group is stored in the target shard node.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: receiving a video acquisition request sent by a terminal device, and determining object description information of a test object according to a test object identifier included in the video acquisition request, wherein the video acquisition request is used to request acquisition of a video that meets the video requirement of the test object, and the object description information includes video requirement feature information of the test object; Determining a target recommendation strategy from a plurality of recommendation strategies, and using the target recommendation strategy to process according to the video demand feature information to obtain a recommended video to be played; The recommended video to be played is sent to the terminal device.

8. A data processing device, characterized in that: The device comprises: An acquisition unit is configured to acquire, from a playback data table stored in a sharding node, an associated playback data group associated with a test time; the playback data table includes a plurality of playback data groups, each of which includes a test subject number and video playback data; the sharding node matches the test subject number, the video playback data is playback data generated by playing a test video according to an instruction of the test subject, and the test video is a video recommended to the test subject corresponding to the test subject number according to a recommendation strategy; The acquisition unit is further configured to acquire, from a test data conversion table stored in the sharding node, an associated data block associated with the test time; the test data conversion table includes a plurality of data blocks, each of which includes a recommended strategy identifier, a statistical data time, and bitmap data; the test data conversion table is determined based on a test data table stored in the sharding node, the test data table includes a plurality of test data groups, each of which includes a test object number and a recommended strategy identifier; the test data group is associated with a playback data group in the playback data table; the bitmap data is used to indicate whether an associated test data group corresponding to the test object number is acquired within the statistical data time, the associated test data group being the same as the recommended strategy identifier included in the data block; a determining unit, configured to determine statistical data corresponding to a target recommendation strategy based on the associated playback data group and the associated data block; A processing unit is used to determine evaluation data of the target recommendation strategy corresponding to the shard node within the test time based on the statistical data.

9. A computer device, characterized in that: include: A processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores computer instructions, and the processor is used to call the computer instructions to implement the data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which, when executed on a computer device, enable the computer device to implement the data processing method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the data processing method according to any one of claims 1 to 7 is implemented.

Images