Data processing method and device, equipment, medium and product

Abstract

The embodiment of the invention discloses a data processing method and device, equipment, a medium and a product, and the method comprises the steps: obtaining an object data set corresponding to a data extraction task, and distributing a random filtering value for each piece of object data in the object data set; obtaining a pre-screening proportion corresponding to the data extraction task, and pre-screening from the object data set according to the pre-screening proportion and the filtering value corresponding to each piece of object data to obtain target object data; an optimal filtering value is determined from the filtering values corresponding to the target object data according to the extraction number of the data extraction tasks, and the optimal filtering value is used for screening the object data, so that the number of the screened object data is close to the extraction number; candidate object data are obtained through screening from the target object data according to the preferable filtering value, and data extraction processing is carried out on the candidate object data to obtain extraction object data of the data extraction task. According to the technical scheme provided by the embodiment of the invention, massive concurrence and rapid extraction lottery drawing can be supported while extraction fairness is ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, a data processing device, an electronic device, a computer-readable storage medium, and a computer product. Background Art

[0002] At present, in order to stimulate the activity of existing objects and attract new objects, many platforms will adopt some means, such as setting up data extraction tasks to extract lucky objects from multiple objects for discounts or subsidies.

[0003] However, the data extraction task involving tens of millions of objects has a high concurrency, with a peak object participation rate of about 300,000 queries per second (QPS). Currently, the extraction and prize-giving is carried out by exchanging time for space, that is, the object first registers for the data extraction task, and then the extraction and prize-giving is carried out when the time comes. Although this non-real-time extraction and prize-giving can solve the problem of massive data extraction, the long extraction and prize-giving time leads to poor object experience. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, a data processing device, an electronic device, a computer-readable storage medium, and a computer program product, which support massive concurrency and fast lottery drawing while ensuring fairness in data extraction.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] According to one aspect of an embodiment of the present application, a data processing method is provided, including: obtaining an object data set participating in a data extraction task, and assigning a random filtering value to each object data in the object data set; obtaining a pre-screening ratio corresponding to the data extraction task, and pre-screening target object data from the object data set according to the pre-screening ratio and the filtering value corresponding to each object data; determining a preferred filtering value from the filtering value corresponding to the target object data according to the extraction quantity of the data extraction task, the preferred filtering value being used to screen the object data so that the number of screened object data is close to the extraction quantity; screening candidate object data from the target object data according to the preferred filtering value, and performing data extraction processing on the candidate object data to obtain the extracted object data of the data extraction task.

[0007] According to one aspect of an embodiment of the present application, a data processing device is provided, comprising: an acquisition module, used to acquire an object data set participating in a data extraction task, and assign a random filtering value to each object data in the object data set; a pre-screening module, used to acquire a pre-screening ratio corresponding to the data extraction task, and pre-screen the target object data from the object data set according to the pre-screening ratio and the filtering value corresponding to each object data; a determination module, used to determine a preferred filtering value from the filtering value corresponding to the target object data according to the extraction number of the data extraction task, the preferred filtering value being used to screen the object data so that the number of screened object data is close to the extraction number; an extraction module, used to screen the target object data according to the preferred filtering value to obtain candidate object data, and perform data extraction processing on the candidate object data to obtain the extracted object data of the data extraction task.

[0008] In one embodiment of the present application, the device also includes a writing module for writing the object data set and each filter value into at least two reverse tables according to the filter value corresponding to each object data; the pre-screening module is further used to pull the each filter value from the at least two reverse tables if the number of object data contained in the object data set is greater than a preset number threshold; pre-screen the each filter value according to the screening ratio and the preset filtering threshold, and filter the target object data from the at least two reverse tables according to the filtered filter value.

[0009] In one embodiment of the present application, the pre-screening module is further used to obtain a preset first filtering threshold, and screen to obtain a first target filtering value whose filtering value is greater than the first filtering threshold; pull the first object data corresponding to the first target filtering value from the at least two reverse tables; if the proportion of the first object data exceeds the screening proportion, obtain a preset second filtering threshold, and screen to obtain a second target filtering value whose first target filtering value is greater than the second filtering threshold, and the second filtering threshold is different from the first filtering threshold; screen to obtain the second object data corresponding to the second target filtering value from the first object data, and use the second object data as the target object data, wherein the proportion of the second object data does not exceed the screening proportion.

[0010] In one embodiment of the present application, the device also includes a partitioning module, which is used to obtain a single-table concurrency parameter of a database table used to store object data and the number of object data contained in the object data set; and divide the database table into the at least two reverse tables according to the single-table concurrency parameter and the number of object data.

[0011] In one embodiment of the present application, the writing module is further used to obtain a task identifier corresponding to the data extraction task, and generate a table identifier for each object data based on the task identifier and the filter value of each object data; based on the table identifier of each object data, the object data in the object data set and the filter values ​​corresponding to each object data are written into different reverse tables.

[0012] In one embodiment of the present application, the determination module is further used to obtain the number of target object data corresponding to each filter value in each reverse table; and determine the preferred filter value based on the number of target object data corresponding to each filter value in each reverse table and the extraction number of the data extraction task.

[0013] In one embodiment of the present application, the determination module is further used to obtain a first number of target object data whose filter value is greater than or equal to the preferred filter value and a second number of target object data whose filter value is less than the preferred filter value; and to search for the preferred filter value from each filter value corresponding to the target object data so that the first number is less than or equal to the extraction number, and the sum of the first number and the second number is greater than the extraction number.

[0014] In one embodiment of the present application, the determination module is further used to calculate each filter value and the cumulative number of target object data corresponding to each filter value based on the number of target object data corresponding to each filter value; search for the target filter value from each filter value corresponding to the target object data, and if the target filter value and the cumulative number of target object data corresponding to the target filter value are close to the extraction number, then the target filter value is used as the preferred filter value.

[0015] In one embodiment of the present application, the extraction module is further used to obtain the filter value interval corresponding to the target object data in each reverse table; determine the target reverse table in which the preferred filter value falls according to the filter value interval corresponding to each reverse table; pull the target reverse table in pages to extract the target object data whose filter value matches the preferred filter value from the target reverse table, and use the extracted target object data as the candidate object data.

[0016] In one embodiment of the present application, the extraction module is further used to randomly sort the candidate object data to obtain an object list; from the object list, the extracted object data is randomly extracted according to the extraction quantity of the data extraction task, or the extracted object data is extracted according to the extraction quantity of the data extraction task and the sorting of the object data in the object list.

[0017] In one embodiment of the present application, the device also includes a writing module for writing record information of each object in the object data set participating in the data extraction task into a forward table, and the device also includes a verification module for verifying the extracted object data according to the forward table and generating an extraction result based on the verification result.

[0018] In one embodiment of the present application, the device also includes a storage module for generating an extraction result based on the preferred filter value and the extracted object data; storing the list of objects participating in the data extraction task under a preset key value, and storing the extraction result in an attribute value corresponding to the preset key value; in response to a view request of an object participating in the data extraction task, accessing the preset key value according to the view request to read the extraction status of the object from the attribute value.

[0019] According to one aspect of an embodiment of the present application, an embodiment of the present application provides an electronic device, comprising one or more processors; a storage device for storing one or more computer programs, when the one or more computer programs are executed by the one or more processors, the electronic device implements the data processing method as described above.

[0020] According to one aspect of an embodiment of the present application, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of an electronic device, the electronic device executes the data processing method as described above.

[0021] According to one aspect of an embodiment of the present application, an embodiment of the present application provides a computer program product, including a computer program, wherein the computer program is stored in a computer-readable storage medium, and a processor of an electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device performs the data processing method as described above.

[0022] In the technical solution provided in the embodiment of the present application, an object data set corresponding to a data extraction task is obtained, and a random filtering value is assigned to each object data in the object data set, a pre-screening ratio corresponding to the data extraction task is obtained, and the target object data is pre-screened from the object data set according to the pre-screening ratio and the filtering value corresponding to each object data, and then a preferred filtering value is determined from the filtering value corresponding to the target object data according to the extraction quantity of the data extraction task, and the preferred filtering value is used to filter the object data so that the number of filtered object data is close to the extraction quantity, and candidate object data is filtered from the target object data according to the preferred filtering value, and data extraction processing is performed on the candidate object data to obtain the extracted object data of the data extraction task, that is, by Assign random filtering values, and quickly filter out target object data according to the pre-screening ratio, to ensure that each object has an equal chance in the extraction, and is not affected by the order or data characteristics, and greatly reduce the amount of data that needs to be processed later, reduce the computational burden, and improve the extraction speed. On the basis of the target object data, the preferred filtering value is calculated in combination with the number of extractions, and the number of candidate objects screened is accurately controlled to be close to the number of extractions, avoiding unnecessary repeated calculations, thereby further speeding up the extraction speed. The two-level screening strategy combining the pre-screening ratio and the preferred filtering value provided in the embodiment of the present application shortens the time from participation to generating results, which not only ensures the fairness of the extraction, but also supports massive concurrency and fast drawing and awarding, so that the objects can quickly obtain the extraction information and improve the object experience.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings.

[0025] Figure 1 It is a schematic diagram of an implementation environment involved in this application.

[0026] Figure 2 It is a flow chart of a data processing method shown in an exemplary embodiment of the present application.

[0027] Figure 3 It is a page diagram of a lottery activity scenario shown by an exemplary embodiment of the present application.

[0028] Figure 4 An exemplary embodiment of the present application shows a method based on Figure 3 The overall execution flow chart of the data processing method for the lottery activity scenario.

[0029] Figure 5 It is a schematic diagram of another data processing method shown in an exemplary embodiment of the present application.

[0030] Figure 6 It is a schematic diagram of a data storage shown by an exemplary embodiment of the present application.

[0031] Figure 7 It is a schematic diagram of another data storage shown in an exemplary embodiment of the present application.

[0032] Figure 8 It is a schematic diagram of pre-screening shown in an exemplary embodiment of the present application.

[0033] Fig. 9 It is a schematic diagram of another data processing method shown in an exemplary embodiment of the present application.

[0034] Fig.10 It is a flowchart of another data processing method shown in an exemplary embodiment of the present application.

[0035] Fig.11 It is a flowchart of another data processing method shown in an exemplary embodiment of the present application.

[0036] Fig.12 It is a flowchart of another data processing method shown in an exemplary embodiment of the present application.

[0037] Fig.13 It is a flowchart of another data processing method shown in an exemplary embodiment of the present application.

[0038] Fig.14 It is a structural block diagram of a data processing device shown in an exemplary embodiment of the present application.

[0039] Fig.15 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0040] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0041] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0042] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all contents and operations, nor must they be executed in the order described. For example, some operations may be decomposed, while some operations may be combined or partially combined, so the actual execution order may change according to actual conditions.

[0043] It should also be noted that the "multiple" mentioned in this application refers to two or more than two. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship.

[0044] The technical solutions of the embodiments of the present application are introduced in detail below.

[0045] See also Figure 1 , Figure 1 1 is a schematic diagram of an implementation environment involved in the present application. The implementation environment includes a terminal 10 and a server 20.

[0046] The terminal 10 is used to participate in the data extraction task based on the operation of the object, and record the object data that has participated in the data extraction task.

[0047] The server 20 is used to receive the object data sent by each terminal to generate an object data set for a data extraction task, and assign a random filtering value to each object data in the object data set; then obtain the pre-screening ratio corresponding to the data extraction task, and pre-screen the target object data from the object data set according to the pre-screening ratio and the filtering value corresponding to each object data; further, determine the preferred filtering value from the filtering value corresponding to the target object data according to the extraction quantity of the data extraction task, the preferred filtering value is used to filter the object data so that the number of filtered object data is close to the extraction quantity, and then determine the candidate object data from the target object data according to the preferred filtering value, and perform data extraction processing on the candidate object data to obtain the extracted object data of the data extraction task.

[0048] The server may also send the winning object data to the terminal, so that the terminal displays the intermediate object data to the object.

[0049] In some embodiments, the server 20 may also record the object data that has participated in the data extraction task, obtain the object data set, and then perform pre-screening and determine the preferred filter value, and then determine the candidate object data from the target object data, and perform data extraction processing on the candidate object data to obtain the extracted object data of the data extraction task.

[0050] Among them, the aforementioned terminal 10 can be an electronic device such as a smart phone, a tablet, a laptop computer, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc. The server 20 can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network) and big data and artificial intelligence platforms. There is no restriction on this here.

[0051] The terminal 10 and the server 20 establish a communication connection in advance through a network, so that the terminal 10 and the server 20 can communicate with each other through the network. The network can be a wired network or a wireless network, and there is no limitation here.

[0052] It should be noted that in the specific implementation of this application, the object data involves object-related information. When the embodiments of this application are applied to specific products or technologies, it is necessary to obtain the object's permission or consent, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0053] The various implementation details of the technical solutions of the embodiments of the present application are elaborated in detail below.

[0054] like Figure 2 As shown, Figure 2 is a flowchart of a data processing method according to an embodiment of the present application, which can be applied to Figure 1 In the implementation environment shown, the method can be executed by the terminal or the server, or by the terminal and the server together. In the embodiment of the present application, the method is executed by the server as an example for explanation. The data processing method may include S210 to S240, which are described in detail as follows.

[0055] S210: Acquire an object data set corresponding to the data extraction task, and assign a random filtering value to each object data in the object data set.

[0056] In an embodiment of the present application, a data extraction task refers to a task of extracting part of data from a large amount of data. The data extraction task may be a real-time data extraction task. As long as an object successfully participates in the data extraction task according to the participation conditions of the data extraction task, the object data participating in the data extraction task will be recorded to obtain an object data set including the object data of each participating task, wherein the object data includes a unique identifier of the object, such as an object ID, and the object data also includes participation time, etc.

[0057] In one example, the data extraction task may be a lottery event set by a merchant, such as Figure 3 As shown, it is a page for a lottery activity, which displays the lottery participation conditions and the conditional participation control 310. For example, the lottery participation conditions displayed in the page are "invite friends" and the control "go share", and the page also displays the prize introduction 320 of the lottery reward, which can be a virtual prize or a physical prize. The page also displays the lottery participation control 330, such as "attract wealth immediately". The terminal can jump to the invite friend interface in response to the triggering of the participation condition 310, and display the prize details in a floating window or pop-up window on the page in response to the triggering operation of the prize introduction 320, and respond to the object's triggering operation on the lottery participation control 330, indicating that the object has participated in the lottery activity, and the terminal records the object data of the object at this time.

[0058] In the embodiments of the present application, object data is object-related. When the embodiments of the present application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the object, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0059] After obtaining the object data set, a random filter value filter is assigned to each object data in the object data set. The filter value can be evenly distributed within a preset value range, such as using a random number generator to generate a value between 0 and 10. 6 The random number generator needs to meet the condition of uniform distribution, that is, the probability of each value appearing is equal to ensure the fairness and randomness of the extraction process; among them, since the filter value is random, the filter value corresponding to each object data can be different or the same.

[0060] S220: Obtain a pre-screening ratio corresponding to the data extraction task, and pre-screen the target object data from the object data set according to the pre-screening ratio and the filter value corresponding to each object data.

[0061] It is understandable that there are many objects involved in the data extraction task. In order to increase the speed of data extraction in the embodiment of the present application, the object data is reduced through pre-screening; when calculating the extraction results, it is necessary to use the filter value corresponding to the object data according to the pre-screening ratio corresponding to the data extraction task to screen part of the target object data and continue to calculate the extraction results.

[0062] In one example, the pre-screening ratio can be determined according to the needs of the data extraction task. For example, if the data extraction task is a lottery, the winning ratio is calculated based on the number of winners in the lottery and the total number of objects participating in the lottery, and the pre-screening ratio is greater than the winning ratio. For example, if the winning ratio is 10%, the pre-screening ratio can be a multiple of the winning ratio, such as 40%. In another example, the pre-screening ratio can be a fixed ratio, such as a pre-screening ratio of 50% of the total number of objects participating in the lottery.

[0063] In the embodiment of the present application, after obtaining the pre-screening ratio, the target object data can be pre-screened from the object data set according to the filter value corresponding to each object data, so that the ratio of the target object data in the object data set reaches or is less than the pre-screening ratio. For example, according to the filter value, the object data whose filter value is greater than the preset filter threshold is taken out to obtain the target object data, wherein the preset filter threshold can be determined from the filter values ​​corresponding to all the object data.

[0064] S230. Determine a preferred filter value from the filter values ​​corresponding to the target object data according to the extraction quantity of the data extraction task. The preferred filter value is used to filter the object data so that the number of filtered object data is close to the extraction quantity.

[0065] It is understandable that the number of target object data is still greater than the number of extractions, and it is necessary to filter out as many extraction objects as possible from the target object data at one time to increase the extraction speed. Therefore, it is necessary to determine a preferred filter value winner_filter from the filter values ​​corresponding to the target object data. The preferred filter value is used to filter the target object data so that the number of filtered object data is as close to the number of extractions as possible.

[0066] S240: Screen the target object data to obtain candidate object data according to the preferred filter value, and perform data extraction processing on the candidate object data to obtain extraction object data of the data extraction task.

[0067] In an example of an embodiment of the present application, the number of target object data with a filter value greater than the preferred filter value can be obtained, recorded as the first number, the number of target object data with a filter value less than the preferred filter value can be obtained, recorded as the second number, and the number of target object data with a filter value equal to the preferred filter value can be obtained, recorded as the third number, and the target number closest to the extraction number among the first number, the second number and the third number can be determined, and then the target object data corresponding to the target number can be used as candidate object data.

[0068] For example, the extraction quantity of the data extraction task is the number of winners of a lottery, there are 1,000 target object data, the corresponding filter value is 1 to 99, and the number of winners is 100, then a very small preferred filter value is determined from 1 to 99. Suppose the first quantity obtained through the preferred filter value is 110, the second quantity is 80, and the third quantity is 25, among which the target object data corresponding to the first quantity can be used as candidate object data, and the target object data corresponding to the second quantity and the target object data corresponding to the third quantity can also be used as candidate object data.

[0069] In an embodiment of the present application, if the number of candidate object data is equal to the extraction number, the candidate object data can be directly determined as the extraction object data of the data extraction task; if the number of candidate objects is greater than the extraction number, a corresponding number of object data needs to be extracted from the candidate object data as the extraction object data.

[0070] Optionally, you can extract the object data from the candidate object data through random_shuffle; random_shuffle is used to randomly sort the sequence. It accepts two iterators, indicating the range of the sequence to be sorted, and an optional random number generator as a parameter; random_shuffle will rearrange the elements in the sequence according to the provided random number generator (or the default random number generator), so that the order of the sequence may be different each time the program is run. For example:

[0071]

[0072]

[0073] After random_shuffle, the present embodiment considers that the object data extraction process is fair, the number of participating objects is n, and the number of extractions is k, for example:

[0074] 1. If n=1, the probability of extracting each object is 1, which is fair.

[0075] 2. If n=k, the probability of extracting each object is 1, which is fair.

[0076] 3. Assuming that n objects participate, the extraction probability of each object is It's fair.

[0077] At this time, the execution reaches the n+1th object. For the n+1th object, its participation and extraction is equivalent to randomly selecting a value less than k, so the extraction probability of ta is

[0078] For the first n objects, their probability of being extracted is equal to being extracted first and then not replaced by the n+1th object. Their probability of being extracted is Therefore, after the n+1th object is executed, the extraction probability of all the previous n+1 objects is

[0079] It should be noted that the processes in steps S220 to S240 all belong to the process of data extraction.

[0080] In an embodiment of the present application, an object data set corresponding to a data extraction task is obtained, and a random filtering value is assigned to each object data in the object data set, a pre-screening ratio corresponding to the data extraction task is obtained, and target object data is pre-screened from the object data set according to the pre-screening ratio and the filtering value corresponding to each object data, and then a preferred filtering value is determined from the filtering value corresponding to the target object data according to the extraction quantity of the data extraction task, the preferred filtering value is used to filter the object data so that the number of filtered object data is close to the extraction quantity, candidate object data is filtered from the target object data according to the preferred filtering value, and data extraction processing is performed on the candidate object data to obtain the extracted object data of the data extraction task, that is, by assigning a random filtering value to each object, Filter value, and quickly filter out the target object data according to the pre-screening ratio, to ensure that each object has an equal chance in the extraction, is not affected by the order or data characteristics, and greatly reduces the amount of data that needs to be processed later, reduces the calculation burden, and improves the extraction speed. On the basis of the target object data, the preferred filter value is calculated in combination with the number of extractions, and the number of candidate objects screened out is accurately controlled to be close to the number of extractions, avoiding unnecessary repeated calculations, thereby further speeding up the extraction speed; the two-level screening strategy combining the pre-screening ratio and the preferred filter value provided in the embodiment of the present application shortens the time from participation to the announcement of the results, which not only ensures the fairness of the extraction, but also supports massive concurrency and fast drawing and awarding, so that the objects can quickly obtain the extraction information and improve the object experience.

[0081] Take the data extraction task as a lottery activity as an example. Figure 4 As shown, the object is Figure 3The lottery page of the terminal clicks the lottery button "Attract Fortune Now" to draw a lottery, and the terminal obtains the lottery record of the object, that is, the object data, and sends the lottery record to the server. At this time, the server processes the lottery data of multiple objects, wherein the lottery data processing includes:

[0082] The participation records of the lottery are written into the forward table, and the data of the objects participating in the lottery are written into the reverse table. In the perception of the object, if the forward table can successfully write data, it means that the object has successfully participated in the lottery. For the forward table, each object has its own table, which means that the writing of the forward table will not be a bottleneck. The consistency of the lottery results is ensured by constructing the forward table and the reverse table, and high concurrency is supported by the sub-table of the reverse table.

[0083] Then, step S220 is used to pull out part of the target object data in the reverse table to pre-screen the pulled data, and then the filter value of the target object data pulled out in step S230 is used to calculate the preferred filter value winner_filter, and step S240 is used to filter the target object data based on winner_filter to obtain candidate object data, and then random_shuffle is used to extract the candidate object data to obtain the winning object data to generate the lottery result.

[0084] In the embodiment of the present application, a situation where 100 million people participate in a lottery and there are 10 million winning places is simulated, and the average lottery opening time is about 815ms.

[0085] Among them, by constructing the lottery records of the forward table and the reverse table, and then supporting high concurrency by indexing the reverse table, the lottery speed is improved by pre-allocating filters, and the problem of too many winners and not being able to be stored quickly is solved by winner_filter. While ensuring the fairness of the lottery, it supports massive concurrency and fast lottery. Among them, the consistency of the lottery results is guaranteed by constructing the forward table and the reverse table, and high concurrency is guaranteed by sharding the reverse table. The lottery results are calculated by random_shuffle and the value filter is pre-allocated to improve the lottery speed. The speed of the lottery list is improved by winner_filter to ensure the stability and speed of storage. The fairness of the lottery results is guaranteed by random_shuffle and pre-screening.

[0086] It should be noted that the specific process of writing object data into the reverse table and pulling the target object data through pre-screening can be found in Figure 5 For the specific process of calculating winner_filter, please refer to Fig. 9 For the specific process of filtering the target object data to obtain the candidate object data, please refer to Fig.10For the specific process of generating the lottery results, please refer to Fig.11 The embodiment shown.

[0087] In one embodiment of the present application, another data processing method is provided, which can be applied to Figure 1 In the implementation environment shown, the method can be executed by a terminal or a server, or by a terminal and a server together. In the embodiment of the present application, the method is described by taking the server as an example. Figure 5 As shown, the data processing method is Figure 2 On the basis of S210 to S240 shown in FIG, S510 is added after S210, and S220 is expanded to S520 to S530; S510 to S530 are described in detail as follows.

[0088] S510: Write the object data set and each filter value into at least two reverse tables according to the filter value corresponding to each object data.

[0089] It is understandable that after an object participates in a data extraction task, the object's participation record needs to be stored for subsequent tracing and calculation of extraction results. Therefore, the object data set needs to be written into a reverse table, which is used to store the object data participating in the data extraction task. Since there is a lot of object data participating in the task, writing one reverse table at the same time will cause failure to write to the reverse table due to high concurrency. Therefore, in an embodiment of the present application, the object data set and each filter value are written into at least two reverse tables according to the filter value corresponding to each object data, and each reverse table stores a portion of the object data, wherein the filter value corresponding to each object data is used to determine which reverse table the object data and its corresponding filter value are written into. The filter value is written into the reverse table in order to perform pre-screening more quickly than pulling the object data and only pulling the filter value from the reverse table.

[0090] In one example, before writing the object data set into at least two reverse tables, the number of reverse tables can also be determined. The data processing method provided in the embodiment of the present application also includes: obtaining a single-table concurrency parameter of a database table used to store object data and the number of object data contained in the object data set; dividing the database table into at least two reverse tables according to the single-table concurrency parameter and the number of object data.

[0091] A database table is provided for storing object data. The single-table concurrency parameter of the database table refers to the ability of the database table to process concurrent requests. For example, the single-table concurrency of the database table is 4k / 5kqps, which means that the number of queries or requests that can be processed by the single table per second is not less than 4000 or 5000 times. The number of object data contained in the object data set corresponds to the number of concurrent write requests. If each table can independently process a certain number of concurrent requests, then the total concurrent processing capability can be improved by distributing the concurrent write requests to at least two tables.

[0092] The number of required reverse tables is determined according to the single-table concurrency parameters and the number of object data. For example, if the single-table concurrency of the database table is 4kqps and the number of object data is 40w, the number of reverse tables is 40w / 4k=100. Then the database table is divided into 100 tables, and the object data sets and various filter values ​​are written into these 100 reverse tables. This can prevent the failure of writing to the reverse table due to high concurrency, and improve the concurrency of data writing while meeting the performance characteristics of the database table.

[0093] In the embodiment of the present application, the table identifier tableuin of the initial reverse table is determined by the task identifier of the data extraction task. After the table is divided, tableuin is composed of the task identifier and the object filter value.

[0094] In one example, a write rule for writing object data into a reverse table is determined based on a filter value corresponding to each object data. For example, object data whose filter values ​​fall within a continuous interval are written into the same reverse table. For another example, object data corresponding to filter values ​​generated in the same continuous time period are written into the same reverse table.

[0095] S520: If the number of object data included in the object data set is greater than a preset number threshold, each filter value is pulled from at least two reverse tables.

[0096] It is worth mentioning that, due to the existence of the law of large numbers, only when the number of samples is large enough, the sample results will tend to be evenly distributed; therefore, when the number of people participating in the task is too small, that is, the number of object data contained in the object data set is less than or equal to the preset number threshold, the embodiment of the present application will not perform pre-screening, but will pull out the full amount of object data as the target object data. On the contrary, if the number of object data contained in the object data set is greater than the preset number threshold, it is required to pull part of the object data for subsequent data extraction; wherein the preset number threshold can be flexibly adjusted according to actual conditions, for example, the preset number threshold can be determined according to the scale of the data extraction task, such as the data extraction task is a national lottery held by a well-known brand or organization, which can often attract tens of thousands or even more objects to participate, and the preset number threshold is relatively large; while for small lottery activities, such as regional lotteries held by communities or businesses, the number of participants is relatively small, and the preset number threshold is relatively small.

[0097] If the number of object data contained in the object data set is greater than a preset number threshold, each filter value is pulled from at least two reverse tables. Compared with pulling object data, pulling filter values ​​from the reverse table is faster, thereby improving the subsequent lottery drawing speed.

[0098] like Figure 6 As shown, it shows the data storage of the lottery activity when the reverse table is not divided. The anchor corresponding to the lottery activity is set to pull all object data from a reverse table. Since the reverse table stores a lot of data, the pulling is slow, and the data field of the lottery is put based on all the object data, and then the lottery is drawn; the candidate list is also all the object data, and all the object data are random_shuffled to obtain the winning object data. Figure 7 In order to store the data of the lottery after dividing the reverse table, the anchor pulls all filter values ​​in batches from 100 reverse tables. Pulling in batches from each reverse table can increase the pulling speed and put it into the lottery data field. When the lottery is conducted, all filter values ​​are pulled in batches, and then data is extracted to obtain the winning object data. Compared with the object data, the winning data is obtained through the filter value, and the lottery opening speed is faster.

[0099] S530: Pre-screen each filter value according to the screening ratio and the preset filter threshold, and filter the target object data from at least two reverse tables according to the filtered filter values.

[0100] In the embodiment of the present application, it is very slow and resource-consuming to pull all objects from the reverse table and calculate the extraction results for them, because all the participating records currently extracted need to be recorded in the memory. Therefore, the embodiment of the present application introduces a pre-allocated filter value. When the participating records are written into the reverse table, the pulled data is reduced by pre-screening and filtering. At the same time, the extraction speed can be improved due to the reduction of the pulled data.

[0101] Among them, a preset filtering threshold is obtained, each filtering value is pre-screened according to the filtering threshold, and the target object data is filtered from at least two reverse tables according to the filtered filtering values, so that the proportion of the target object data reaches the filtering ratio; the filtering threshold can be determined randomly, or it can be determined from each filtering value according to the filtering ratio.

[0102] For example, Figure 8 As shown in the figure, there are 800,000 lottery records in the reverse table. When each object participating in the lottery is written into the reverse table, a random filter value is assigned. This value is in the range [0,10 6 ] are evenly distributed between them, and a preset ratio is determined according to the needs of the lottery. For example, if you need to screen out 25% of the objects as candidates, then the preset screening ratio is 25%, and the screening threshold is calculated. For example, if the preset ratio is 25%, the threshold is 750,000, and the filter values ​​of all objects are pulled from the reverse table, and the object data with filter values ​​less than or equal to the threshold are screened out as the target object data, and the records with filter values ​​greater than 750,000 are removed, such as removing the object data corresponding to the filter values ​​of 953924, 823627, 910486, etc. After removal, the remaining nearly 200,000 are used as the target object data, and the screening ratio is achieved. Among them, nearly 200,000 means that the absolute value of the difference between the number of screened target object data and 200,000 is less than the preset difference threshold, such as less than 100.

[0103] It is understandable that if there are too many object data in the object data set, at least two rounds of pre-screening may be performed to screen the target object data, including:

[0104] A preset first filtering threshold is obtained, and a first target filtering value whose filtering value is greater than the first filtering threshold is obtained by screening; first object data corresponding to the first target filtering value is pulled from at least two reverse tables; if the proportion of the first object data exceeds the screening proportion, a preset second filtering threshold is obtained, and a second target filtering value whose first target filtering value is greater than the second filtering threshold is obtained by screening, and the second filtering threshold is different from the first filtering threshold; second object data corresponding to the second target filtering value is obtained by screening from the first object data, and the second object data is used as the target object data, wherein the proportion of the second object data does not exceed the screening proportion.

[0105] In an embodiment of the present application, since the filter values ​​are evenly distributed within a preset value range, an intermediate value can be taken from the preset value range as the first filter threshold, and a first round of pre-screening is performed based on the first filter threshold. A first target filter value greater than the first filter threshold is screened from each filter value, and then the first object data corresponding to the first target filter value is pulled from at least two reverse tables.

[0106] It can be understood that the same filter value can correspond to at least two object data, so the proportion of the first object data pulled may exceed the screening ratio. If the proportion of the first object data in the total object data in the object data set exceeds the screening ratio, such as the difference between the proportion and the screening ratio is greater than the proportion threshold, such as 5%, then based on the first target object data, a second round of pre-screening will be carried out.

[0107] Among them, when conducting the second round of pre-screening, it is necessary to first obtain the second filtering threshold. The second filtering threshold is different from the first filtering threshold. The second filtering threshold can take an intermediate value from the value range of the first target filtering value as the second filtering threshold, or it can be the number of target object data corresponding to each first target filtering value. Then, the second filtering threshold is determined by the screening ratio and the number of target object data corresponding to each first target filtering value, and then the second filtering target value greater than the second filtering threshold is further screened from each first target filtering value, and then the second object data corresponding to the second target filtering value is pulled from the first object data. The proportion of the second object data does not exceed the screening ratio, so the second object data is used as the target object data.

[0108] In other embodiments of the present application, when performing pre-screening, according to actual conditions, a first target filter value whose filter value is less than a first filter threshold can be screened out, and then the first object data can be pulled out, and a second target filter value whose first target filter value is less than a second filter threshold can be screened out, and then the target object data can be screened out.

[0109] It is hereby stated that the extraction remains fair after pre-screening. Suppose the embodiments of this application want to pre-screen object data of p (0 < p ≤ 1) to participate in random_shuffle, and suppose the number of object data screened out by the embodiments of this application is m. Then the value of filter adopted by the embodiments of this application here is (1 - p) × u, and the probability that the filter value of the object data can enter the pre-screening is p. Then the extraction probability of each object is

[0110] where u represents the total number of object data or the maximum range of the filter values of the object data. For example, if the filter value of the object data is between 0 and 10,000, then u is 10,000, and (1 - p) × u represents the range of the filter values of the screened object data. Suppose p is 0.8, then (1 - p) is 0.2, and (1 - p) × u is 0.2 × 10,000 = 2,000. This means that the filter values of the screened objects are between 0 and 2,000. The function of this formula is to reduce the amount of data to be processed through pre-screening, thereby improving the extraction speed and efficiency; the set of object data after pre-screening will be smaller, but still be able to represent the entire object data population, thus ensuring the fairness and accuracy of the lottery. When the number of participants is large enough, there is At this time, the entire extraction process is fair.

[0111] It should be noted that Figure 5 For other detailed introductions of S210, S230 - S240 shown in Figure 2 please refer to S210, S230 - S240 shown in

[0112] In the embodiments of this application, the object data and the filter values are written into at least two reverse tables according to the filter values corresponding to the object data to support high-concurrency data writing, and filtering is performed by pulling the filter values, and then the target object data is filtered through the obtained filter values to improve the pre-screening speed and further improve the extraction and lottery-drawing speed.

[0113] The embodiments of this application provide another data processing method, which can be applied to the Figure 1 implementation environment shown in Fig. 9 This method can be executed by the terminal or the server, or jointly executed by the terminal and the server. In the embodiments of this application, taking the execution of this method by the server as an example for illustration, as Figure 5 shown, on the basis of what is shown in Figure 5 S510 shown in

[0114] S910: Obtain a task identifier corresponding to the data extraction task, and generate a table identifier for each object data according to the task identifier and a filter value of each object data.

[0115] The task identifier corresponding to the data extraction task can be a task ID. For each object data, a combination of the task identifier and the filter value filter of the object data is used to calculate the table identifier tableuin of the object data. The table identifier is used to determine in which table the data should be stored. For example, there is a hash function hash(filter value, task ID), which can generate a value ranging from 0 to N-1 (N is the number of reverse tables). For example, tableuin = hash(task ID + filter) % N.

[0116] S920. According to the table identifier of each object data, write the object data in the object data set and the filter value corresponding to each object data into different reverse tables.

[0117] It is understandable that after determining the number of reverse tables, a table identifier needs to be assigned to each reverse table, and then the object data is assigned to different reverse tables according to the value of tableuin. For example, if the value of tableuin of object data 1 is 0, object data 1 is stored in reverse table table_0; if the value of tableuin of object data 2 is 1, the data is stored in reverse table table_1, and so on. Because the tableuins calculated by the hash function for data with similar filter values ​​will also be similar, this ensures that records with filter values ​​falling within a continuous range are placed in the same table.

[0118] In an embodiment of the present application, the process of calculating tableuin combines the filter value and the task ID to generate a unique table identifier for determining the storage location of the data, so that records whose filter values ​​fall within a continuous interval are placed in the same table, thereby optimizing the physical storage structure of the data; in this way, when querying data within a specific filter value range, only one table needs to be accessed instead of traversing all tables, so as to improve the efficiency of subsequent queries.

[0119] Continue to see Fig. 9 ,exist Fig. 9 Based on the above, in one example, S230 is expanded into steps S930 to S940, which correspond to a method of determining a preferred filtering value; wherein S930 to S940 are described in detail as follows:

[0120] S930: Obtain the number of target object data corresponding to each filter value in each reverse table.

[0121] S940: Determine a preferred filter value according to the number of target object data corresponding to each filter value in each reverse table and the number of extractions of the data extraction task.

[0122] In one example, the filter values ​​of all target object data in each reverse table may be sorted to generate a filter value sequence in descending order, which is represented as follows: V1, V2, V3, ... V n , V1≥V2≥V3≥…≥V n , merge the same filter value, and get the number of target object data corresponding to each filter value in each reverse table, which can be expressed as (V1, K1), (V2, K2), (V3, K3), ... (V x , K x ), where K i Indicates that the filter value is V i The number of target object data.

[0123] V1≥V2≥V3≥…≥V x ,So The frequency of this sequence appearing in all extraction spaces is n is the total number of target object data.

[0124] In one example, determining the preferred filtering value in S930 includes:

[0125] Obtain a first quantity of target object data whose filter value is greater than or equal to the preferred filter value and a second quantity of target object data whose filter value is less than the preferred filter value; search for the preferred filter value from each filter value corresponding to the target object data so that the first quantity is less than or equal to the extraction quantity, and the sum of the first quantity and the second quantity is greater than the extraction quantity.

[0126] Exemplary:

[0127] k1=∑I record.filter ≥winner_filter(record)

[0128] k2=∑I record.filter =winner_filter-1(record)

[0129] Among them, winner_filter is the preferred filter value, k1 represents the first number of target object data whose filter value is greater than or equal to winner_filter, and k2 represents the second number of target object data whose filter value is equal to winner_filter-1.

[0130] The number of extractions is k, k1+k2>k, k1≤k, k<n, then we need to find a minimum winner_filter so that the number of target object data that meets the conditions just exceeds k, so winner_filter must be greater than V x , which is the minimum filter value.

[0131] From k1+k2>k, we know that winner_filter>V x , and because winner_filter is a minimum value, we can set winner_filter = V i +1, now there is K i =k2.

[0132] According to the condition (k1+k2>k), it can be inferred that winner_filter must be greater than V x This is because: Since k1+k2>k, we need to find a minimum winner_filter so that the number of objects that meet the conditions just exceeds k; if winner_filter≤V x , then k1 will contain all filter values ​​greater than or equal to V x In this case, the value of k1 will be too large and cannot satisfy the condition k1≤k.

[0133] Therefore, winner_filter must be greater than V x To ensure that the value of k1 is not too large. x Select the smallest winner_filter and set it equal to (V i +1), which satisfies K i = k2, that is, the filter value of the object is in V i The number below it is k1, in V i The number of objects with +1 is k2.

[0134] In this way, it is ensured that the selection of winner_filter satisfies the condition of k1≤k and k1+k2>k, thereby ensuring the fairness and accuracy of the extraction process.

[0135] The probability of an object being drawn from this sequence is P1+P2; where P1 is the probability of an object randomly picking a value not less than winner_filter in this sequence. Since winner_filter is V i +1, so only consider V i +1, so P1 is P2 is the object in this sequence, randomly to V i After that, the probability of being drawn; first, the object is randomly selected to V i The probability of Then, among these objects, the probability of being selected by random_shuffle is Because k-k1 winning objects need to be selected from k2 objects, P2 is

[0136] Therefore, the probability of object extraction is So in any sequence, the extraction is fair.

[0137] Assume that there are 10 target object data and the filter values ​​are sorted as (V1=9, V2=8, V3=7, ... V 10 =0); after merging the same filter values, assume that ((V1, K1) = (9, 1), (V2, K2) = (8, 2), ... (V x , K x )=(0,1)); If the number of extractions in the data extraction task is 5, it is necessary to find a winner_filter such that k1+k2>5.

[0138] Through calculation, assuming V5=5, winner_filter must be greater than 5. Select the smallest winner_filter and set it equal to 6.

[0139] In the embodiment of the present application, it is necessary to quickly calculate the winner_filter. Obviously, it can be calculated by binary search. Then the number of RPCs is 27X the number of sub-tables, which is unacceptable.

[0140] In the embodiment of the present application, in order to improve the concurrency performance and extraction speed, the object data involved in the object is processed in a separate table, and the filter of the object is assigned in advance. When writing the reverse table, the embodiment of the present application uses a combination of the filter value and the task ID to calculate tableuin. According to tableuin, it can be determined in which table the object data is stored, and then the records whose filter values ​​fall within a continuous range are placed in the same table.

[0141] In this way, when searching for winner_filter from each filter value corresponding to the target object data, you only need to perform table-by-table number rpcs to know which table winner_filter falls in, and then you can accurately find winner_filter by paging. Since the records have been divided into tables according to the filter value, you only need to query each table once (i.e., table-by-table number rpcs) to determine which table the winner_filter value falls in. After that, the embodiment of the present application pulls out all the object data of filter=winner_filter-1, and performs data extraction processing on the selected object data. The method of selecting data extraction here is still random_shuffle.

[0142] In another example, it is necessary to determine a preferred filter value from the filter values ​​of each target object data so that the number of object data obtained by filtering the preferred filter value is as close as possible to the extracted number. The determining of the preferred filter value in S230 includes:

[0143] According to the number of target object data corresponding to each filter value, calculate each filter value and the cumulative number of target object data corresponding to each filter value that is less than each filter value; search for the target filter value from each filter value corresponding to the target object data; if the target filter value and the cumulative number of target object data corresponding to the target filter value that is less than the target filter value are close to the extraction number, then use the target filter value as the preferred filter value.

[0144] For example, assuming that the filter value corresponding to the target object data ranges from 0 to 100, it is necessary to find the preferred filter value within this range, where the target object data with a filter value of 0 is 1000, the target object data with a filter value of 1 is 1500, and so on; then calculate the cumulative number of target objects for each filter value and all the target objects below it, for example, there are 2000 target object data with a filter value of 2, then the cumulative number is the sum of the number of all target object data from 0 to 2, which is 4500.

[0145] Traverse the filter values ​​0 to 100 corresponding to the target object data to find the target filter value from the filter values ​​0 to 100 corresponding to the target object data. Suppose the extraction quantity is 10,000, the target filter value is 12, and the cumulative number of all target object data from 0 to 12 is 10,100; the target filter value is 11, and the cumulative number of all target object data from 0 to 11 is 9,100; the target filter value is 13, and the cumulative number of all target object data from 0 to 13 is 12,000. Therefore, the cumulative number corresponding to the target filter value is closest to the extraction quantity, so the target filter value 12 is taken as the preferred filter value.

[0146] Optionally, after determining the preferred filter value, find the target reverse table containing the preferred filter value according to the filter value range corresponding to each reverse table. For example, the preferred filter value is 12, and the filter value range of the reverse table table_0 is 0-10, and the filter value range of the reverse table table_1 is 11-20, then the preferred filter value falls in the reverse table table_1.

[0147] After determining that the preferred filter value falls in the reverse table table_1, it is necessary to pull the records in the reverse table table_1 by paging, and accurately find the final preferred filter value from the reverse table table_1. Among them, when accurately finding the final preferred filter value, pull each filter value in the reverse table table_1, and for each filter value in the reverse table table_1, calculate the filter value in the reverse table table_1 and the cumulative number of target object data corresponding to the filter value, and select the filter value with the closest cumulative number to the number of winning places as the final preferred filter value. It can be understood that the preferred filter value can be the same as the final preferred filter value, and the accuracy of the preferred filter value is ensured by paging.

[0148] In other embodiments of the present application, the preferred filter value is determined according to the number of target object data corresponding to each filter value in each reverse table and the number of data extraction tasks, including:

[0149] Obtain the number of target object data written in each reverse table; determine the target reverse table where the preferred filter value falls according to the number of target object data in each reverse table and the number of data extraction tasks; and pull the target reverse table by pages to determine the preferred filter value from the filter value interval corresponding to the target reverse table.

[0150] In other embodiments of the present application, the corresponding number of target object data written in each reverse table is obtained. It is assumed that the number of target object data written in reverse table table_0 is 150, the number of target object data written in reverse table table_1 is 270, and the number of target object data written in reverse table table_2 is 240. The extraction number of the data extraction task is assumed to be 200. The number of target object data written in reverse table table_2 is closest to the extraction number. Therefore, the candidate object data can be directly screened from reverse table table_2, and the target reverse table into which the preferred filter value falls is determined to be table_2.

[0151] After determining that the target reverse table where the preferred filter value falls is table_2, it is necessary to pull the records in the reverse table table_2 by paging, and accurately find the final preferred filter value from the reverse table table_2, wherein the filter value interval corresponding to the target object data in the reverse table table_2 is first determined, and in the filter value interval, each filter value and the cumulative number of target object data corresponding to each filter value that is less than each filter value are calculated. If there is a target filter value and the cumulative number of target object data corresponding to the target filter value that is less than the target filter value is close to the extraction number, the target filter value is used as the preferred filter value. For example, the filter value range corresponding to the target object data in the reverse table table_2 is 11-20, there are 20 target object data with a filter value of 11, 30 target object data with a filter value of 12, and 50 target object data with a filter value of 13. The sum of the number of all target object data from 11 to 12 is 50, and the sum of the number of all target object data from 11 to 13 is 100, and so on. Assuming that the filter value is 15, the sum of the number of all target object data from 11 to 15 is 200, then the filter value 15 is used as the final preferred filtering value, and all target object data with filter values ​​11 to 15 are used as candidate object data.

[0152] In an embodiment of the present application, the preferred filter value is calculated based on the number of draws and the number of target object data corresponding to each filter value in each reverse table, and the number of screened candidate objects is accurately controlled to be close to the number of draws, thereby avoiding unnecessary repeated calculations and further speeding up the drawing and awarding of prizes.

[0153] It is understandable that if the number of extractions is not large, the embodiment of the present application does not need to use the preferred filtering value strategy. If the number of extractions is large, the embodiment of the present application needs to use the preferred filtering value solution to quickly calculate the extraction results.

[0154] It should be noted that Fig. 9 For detailed introduction of S210, S520~S530, S240 shown in Figure 5 S210, S520~S530, and S240 shown are not described in detail here.

[0155] The present application also provides another data processing method, which can be applied to Figure 1 In the implementation environment shown, the method can be executed by a terminal or a server, or by a terminal and a server together. In the embodiment of the present application, the method is described by taking the server as an example. Fig.10 As shown, in Figure 5 Based on the basis shown in Figure 2The process of screening and obtaining candidate object data in S240 shown in FIG. 1 is expanded to S1010 to S1030. S1010 to S1030 are described in detail as follows.

[0156] S1010. Obtain the filter value interval corresponding to the target object data in each reverse table.

[0157] S1020. Determine the target reverse table into which the preferred filter value falls according to the filter value interval corresponding to each reverse table.

[0158] S1030, paging and pulling the target reverse table to extract target object data whose filter value matches the preferred filter value from the target reverse table, and using the extracted target object data as candidate object data, performing data extraction processing on the candidate object data to obtain extraction object data of the data extraction task.

[0159] As described above, in the same reverse table, the filter values ​​are recorded in a continuous range. Therefore, you only need to query each reverse table once (that is, the number of sub-tables rpc), query the maximum and minimum filter values ​​in each sub-table, and you can get the filter value range corresponding to the target object data in each reverse table. For example, the filter value range corresponding to the reverse table table_0 is 0-10, and the corresponding filter value range in the reverse table table_1 is 11-20, and so on.

[0160] After determining the preferred filter value, the target reverse table into which the preferred filter value falls can be determined according to the filter value interval corresponding to each reverse table. For example, if the preferred filter value is 5, it falls into the reverse table table_0 with a filter value interval of 0-10.

[0161] In the target reverse table, each object data may be stored in pages, such as storing object data corresponding to filter values ​​0 to 3 in the first page, storing object data corresponding to filter values ​​4 to 7 in the second page, and so on.

[0162] In one example, if Fig. 9 The k1 and k2 in the example determine the preferred filter value. If the preferred filter value is 5, the object data stored in the second page of the target reverse table can be pulled. Paged pulling can avoid reading a large amount of data at one time, reduce memory consumption, and improve query efficiency. The target object data whose filter value matches the preferred filter value is the target object data whose filter value is 5, and then the target object data whose filter value is 5 is accurately found, and then the target object data whose filter value is 5 is used as the candidate object data; or the target object data whose filter value is 5-1=4 is used as the candidate object data, and then the candidate object data is subjected to data extraction processing to obtain the extracted object data of the data extraction task.

[0163] In another example, if Fig. 9 The preferred filter value is determined by each filter value in and the cumulative number of target object data corresponding to each filter value. Then the object data stored in the first page and the second page can be pulled, and the target object data with a filter value matching the preferred filter value is the target object data with a filter value of 0 to 5. Then, the target object data with a filter value of 0 to 5 is accurately found as the candidate object data, and then data extraction is performed on the candidate object data to obtain the extracted object data of the data extraction task.

[0164] It should be noted that Fig.10 For more detailed information about S210, S510~S530, see Figure 2 S210, S510-S530 shown in the figure are not described in detail here.

[0165] In the embodiment of the present application, the filter value interval corresponding to the reverse table is first obtained, so that the target reverse table where the winner_filter is located can be found quickly and accurately, and then the number of queries and the amount of data are greatly reduced through paging and pulling, thereby improving the speed of drawing prizes and the concurrent performance of the system.

[0166] In one embodiment of the present application, another data processing method is also provided, which can be applied to Figure 1 In the implementation environment shown, the method can be executed by a terminal or a server, or by a terminal and a server together. In the embodiment of the present application, the method is described by taking the server as an example. Fig.11 As shown, the data processing method is Figures 2 to 9 Based on the basis shown in Figure 2 The process of determining to extract the object data in S240 is extended to S1110 to S1120. The S1110 to S1120 are described in detail as follows.

[0167] S1110 . Filter the target object data to obtain candidate object data according to the preferred filter value, and randomly sort the candidate object data to obtain an object list.

[0168] S1120. Extract object data from the object list by randomly extracting according to the extraction quantity of the data extraction task, or extracting according to the extraction quantity of the data extraction task and the sorting of the object data in the object list.

[0169] In the embodiment of the present application, in order to ensure that the extraction probability of each candidate object data is the same, the candidate object data needs to be randomly sorted to obtain an object list, and then a final extraction list is generated from the object list according to the actual required number of extractions.

[0170] In one example, if the extraction number is 5, 5 candidate object data are randomly selected from the object list as the extraction object data.

[0171] In one example, if the extraction quantity is 5, after the candidate object data are randomly sorted, the candidate object data in the object list have been sorted, so the first 5 candidate object data can be taken as the extraction object data, or the last 5 candidate object data can be taken as the extraction object data.

[0172] It should be noted that Fig.11 For more detailed information about S210~S230 shown in Figure 2 S210 to S230 shown are not described in detail here.

[0173] In the embodiment of the present application, the screened candidate object data is randomly arranged to ensure that the extraction probability of each object is the same, and to screen out qualified objects to ensure the fairness and efficiency of the extraction results.

[0174] It is worth noting that in one embodiment of the present application, another data processing method is also provided, which can be applied to Figure 1 In the implementation environment shown, the method can be executed by a terminal or a server, or by a terminal and a server together. In the embodiment of the present application, the method is described by taking the server as an example. Fig.12 As shown, the data processing method is Figures 2 to 9 On the basis of that shown in , S1210 is added after S210, and S1220 is added after S240. S1210 to S1220 are described in detail as follows.

[0175] S1210. Write the record information of each object in the object data set participating in the data extraction task into the forward table.

[0176] In an embodiment of the present application, the record information of each object participating in the data extraction task is written into the forward table to ensure that the participation record of each object is successfully saved; as long as the forward table can successfully write the record information, it means that the object participates in the data extraction task successfully. For the forward table, each object has its own independent forward table, and the write operation will not become a system bottleneck, supporting high concurrency operations.

[0177] In one example, the recorded information of the object's participation in the task includes the object identification, the object's participation time, conditions, and the like.

[0178] S1220. Verify the extraction object data according to the forward table, and generate an extraction result according to the verification result.

[0179] As described above, when an object participates in a data extraction task, the object data will be written into the reverse table. The consistency of the extraction results can be guaranteed through the record information of the object participation in the forward table and the object data in the reverse table. That is, either the forward table and the reverse table are successfully written at the same time, or neither is written, to avoid data inconsistency.

[0180] After determining the extraction object data through the reverse table, the extraction object data can be verified according to the forward table to determine whether there is record information in the forward table that the extraction object to which the extraction object data belongs participates in the data extraction task, and determine whether the record information in the forward table that the extraction object participates in the data extraction task corresponds to the extraction object data.

[0181] If the forward table contains record information corresponding to the extraction object, and the record information corresponds to the extraction object data, then the extraction object data is verified and the extraction object data is used as the extraction result.

[0182] If the forward table does not contain record information corresponding to the extracted object, or the record information does not correspond to the extracted object data, the extracted object data verification fails, and the extracted object data is screened out, and new extracted object data is re-determined from the candidate object data, and the new extracted object data is verified until the verification passes and the extraction result is generated.

[0183] It should be noted that Fig.12 For more detailed information about S210~S240 shown in Figure 2 S210 to S240 shown are not described in detail here.

[0184] In the embodiment of the present application, the forward table and the reverse table can work together effectively, and the data of the winning objects can be verified through the forward table to ensure the consistency and reliability of the lottery results.

[0185] It is worth noting that in one embodiment of the present application, another data processing method is also provided, which can be applied to Figure 1 In the implementation environment shown, the method can be executed by a terminal or a server, or by a terminal and a server together. In the embodiment of the present application, the method is described by taking the server as an example. Fig.13 As shown, the data processing method is Figures 2 to 9 Based on the above, S1310 to S1330 are added after S240. S1310 to S1330 are described in detail as follows.

[0186] S1310. Generate an extraction result according to the preferred filter value and the extraction object data.

[0187] S1320. Store the list of objects participating in the data extraction task under a preset key value, and store the extraction result in the attribute value corresponding to the preset key value.

[0188] S1330. In response to a viewing request of an object participating in the data extraction task, access a preset key value according to the viewing request to read an extraction status of the object from an attribute value.

[0189] Storing the extraction results of each object takes up a very large storage space. In the embodiment of the present application, in order to reduce storage space and improve efficiency, only one preferred filter value winner_filter value and object data of a small number of objects need to be saved. Therefore, the extraction result is generated based on the extracted object data that meets the winner_filter value condition, and then the extraction result is saved instead of all the object data involved in the extraction.

[0190] Assume that there are 1,000 objects participating in the lottery, and after calculation, it is determined that the winner_filter value is 10. Then, only this value and the object IDs (assuming there are 50 objects) whose filter value is 10 need to be stored. When the lottery is drawn, only these 50 objects need to be randomly shuffled to select the final winner, ensuring the efficiency and fairness of the lottery process. Only one winner_filter value and 50 object IDs are stored, which greatly reduces the storage space required.

[0191] It should be understood that after the extraction is completed, the extraction result and the object list must be consistent, and the extraction status cannot be rolled back. The object list includes the objects participating in the data extraction task. Therefore, the tinned list is stored under a key, and the extraction result is stored in the vlaue value corresponding to the key. Then, when the object queries the extraction status, if the data extraction task is completed, there is no need to read the extraction status from other storage, which improves the query efficiency. The extraction status includes being drawn, such as winning the prize, and the lottery status also includes not being drawn, such as not winning the prize.

[0192] In one example, only objects participating in a lottery can query the lottery status. In response to a view request, the key value is accessed to obtain the lottery list, and then the lottery list is used to detect whether the object that initiated the view request participated in the lottery. If so, the lottery list is extracted from the value, and then it is detected whether the lottery list includes the object data of the object. If so, it is determined that the object has won the prize. If not, it is determined that the object has not won the prize.

[0193] It should be noted that Fig.13 For more detailed information about S210~S240 shown in Figure 2 S210 to S240 shown are not described in detail here.

[0194] In the embodiment of the present application, only a winner_filter value and a small number of object IDs need to be saved to reduce storage space, and the lottery list is stored under a key, and the drawing results are recorded in the value. Only the key-value is needed to read the drawing status, which improves query efficiency.

[0195] Here, the device embodiment of the present application is introduced, which can be used to execute the data processing method in the above embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the above embodiment of the data processing method of the present application.

[0196] The present application embodiment provides a data processing device, such as Fig.14 As shown, the device includes.

[0197] An acquisition module 1410 is used to acquire an object data set participating in a data extraction task, and assign a random filtering value to each object data in the object data set;

[0198] A pre-screening module 1420 is used to obtain a pre-screening ratio corresponding to the data extraction task, and pre-screen the target object data from the object data set according to the pre-screening ratio and the filter value corresponding to each object data;

[0199] A determination module 1430, configured to determine a preferred filter value from the filter values ​​corresponding to the target object data according to the extraction quantity of the data extraction task, wherein the preferred filter value is used to filter the object data so that the number of the filtered object data is close to the extraction quantity;

[0200] The extraction module 1440 is used to filter the target object data to obtain candidate object data according to the preferred filter value, and perform data extraction processing on the candidate object data to obtain extraction object data of the data extraction task.

[0201] In one embodiment of the present application, based on the aforementioned scheme, the device also includes a writing module for writing the object data set and each filter value into at least two reverse tables according to the filter value corresponding to each object data; the pre-screening module is further used to pull the each filter value from the at least two reverse tables if the number of object data contained in the object data set is greater than a preset number threshold; pre-screen the each filter value according to the screening ratio and the preset filtering threshold, and filter the target object data from the at least two reverse tables according to the filtered filter value.

[0202] In one embodiment of the present application, based on the aforementioned scheme, the pre-screening module is further used to obtain a preset first filtering threshold, and screen to obtain a first target filtering value whose filtering value is greater than the first filtering threshold; pull the first object data corresponding to the first target filtering value from the at least two reverse tables; if the proportion of the first object data exceeds the screening proportion, obtain a preset second filtering threshold, and screen to obtain a second target filtering value whose first target filtering value is greater than the second filtering threshold, and the second filtering threshold is different from the first filtering threshold; screen to obtain the second object data corresponding to the second target filtering value from the first object data, and use the second object data as the target object data, wherein the proportion of the second object data does not exceed the screening proportion.

[0203] In one embodiment of the present application, based on the aforementioned scheme, the device also includes a partitioning module, which is used to obtain a single-table concurrency parameter of a database table used to store object data and the number of object data contained in the object data set; and divide the database table into the at least two reverse tables according to the single-table concurrency parameter and the number of object data.

[0204] In one embodiment of the present application, based on the aforementioned scheme, the writing module is further used to obtain the task identifier corresponding to the data extraction task, and generate a table identifier for each object data according to the task identifier and the filter value of each object data; according to the table identifier of each object data, the object data in the object data set and the filter values ​​corresponding to each object data are written into different reverse tables.

[0205] In one embodiment of the present application, based on the aforementioned scheme, the determination module is further used to obtain the number of target object data corresponding to each filter value in each reverse table; and determine the preferred filter value based on the number of target object data corresponding to each filter value in each reverse table and the extraction number of the data extraction task.

[0206] In one embodiment of the present application, based on the aforementioned scheme, the determination module is further used to obtain a first number of target object data whose filter value is greater than or equal to the preferred filter value and a second number of target object data whose filter value is less than the preferred filter value; and to search for the preferred filter value from each filter value corresponding to the target object data so that the first number is less than or equal to the extraction number, and the sum of the first number and the second number is greater than the extraction number.

[0207] In one embodiment of the present application, based on the aforementioned scheme, the determination module is further used to calculate each filter value and the cumulative number of target object data corresponding to each filter value according to the number of target object data corresponding to each filter value; search for the target filter value from each filter value corresponding to the target object data, and if the target filter value and the cumulative number of target object data corresponding to the target filter value are close to the extraction number, the target filter value is used as the preferred filter value.

[0208] In one embodiment of the present application, based on the aforementioned scheme, the extraction module is further used to obtain the filter value interval corresponding to the target object data in each reverse table; determine the target reverse table in which the preferred filter value falls according to the filter value interval corresponding to each reverse table; pull the target reverse table in pages to extract the target object data whose filter value matches the preferred filter value from the target reverse table, and use the extracted target object data as the candidate object data.

[0209] In one embodiment of the present application, based on the aforementioned scheme, the extraction module is further used to randomly sort the candidate object data to obtain an object list; from the object list, the extracted object data is randomly extracted according to the extraction quantity of the data extraction task to obtain the extracted object data, or is extracted according to the extraction quantity of the data extraction task and the sorting of the object data in the object list to obtain the extracted object data.

[0210] In one embodiment of the present application, based on the aforementioned scheme, the device also includes a writing module for writing record information of each object in the object data set participating in the data extraction task into a forward table, and the device also includes a verification module for verifying the extracted object data according to the forward table and generating an extraction result based on the verification result.

[0211] In one embodiment of the present application, based on the aforementioned scheme, the device also includes a storage module for generating an extraction result based on the preferred filter value and the extracted object data; storing the list of objects participating in the data extraction task under a preset key value, and storing the extraction result in an attribute value corresponding to the preset key value; in response to a view request of an object participating in the data extraction task, accessing the preset key value according to the view request to read the extraction status of the object from the attribute value.

[0212] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0213] The device provided in the above embodiment may be arranged in a terminal or in a server.

[0214] An embodiment of the present application also provides an electronic device, comprising one or more processors and a storage device, wherein the storage device is used to store one or more computer programs, and when the one or more computer programs are executed by one or more processors, the electronic device implements the above data processing method.

[0215] Fig.15 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0216] It should be noted that Fig.15 The computer system 1500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0217] like Fig.15 As shown, the computer system 1500 includes a processor (Central Processing Unit, CPU) 1501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (Read-Only Memory, ROM) 1502 or the program loaded from the storage part 1508 to the random access memory (Random Access Memory, RAM) 1503, such as executing the method in the above embodiment. In RAM 1503, various programs and data required for system operation are also stored. CPU 1501, ROM 1502 and RAM 1503 are connected to each other through bus 1504. Input / output (Input / Output, I / O) interface 1505 is also connected to bus 1504.

[0218] In some embodiments, the following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, etc.; an output section 1507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed, so that a computer program read therefrom is installed into the storage section 1508 as needed.

[0219] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1509, and / or installed from a removable medium 1511. When the computer program is executed by a processor (CPU) 1501, various functions defined in the system of the present application are executed.

[0220] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (Erasable Programmable Read Only Memory), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0221] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and a computer program.

[0222] The units or modules involved in the embodiments described in this application may be implemented by software or hardware, and the units or modules described may also be set in a processor. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves.

[0223] Another aspect of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the data processing method as described above is implemented. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0224] Another aspect of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the electronic device executes the data processing method as described above in the above embodiments.

[0225] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0226] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0227] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person of ordinary skill in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. A data processing method, characterized in that: include: Acquire an object data set corresponding to the data extraction task, and assign a random filtering value to each object data in the object data set; Acquire a pre-screening ratio corresponding to the data extraction task, and pre-screen the target object data from the object data set according to the pre-screening ratio and the filter value corresponding to each object data; Determining a preferred filter value from the filter values ​​corresponding to the target object data according to the extraction quantity of the data extraction task, wherein the preferred filter value is used to filter the object data so that the number of the filtered object data is close to the extraction quantity; According to the preferred filter value, candidate object data is obtained from the target object data, and data extraction processing is performed on the candidate object data to obtain extraction object data of the data extraction task.

2. The method according to claim 1, characterized in that After assigning a random filtering value to the object data of each object in the object data set, the method further comprises: Writing the object data set and each filter value into at least two reverse tables according to the filter value corresponding to each object data; The pre-screening from the object data set to obtain the target object data according to the screening ratio and the filter value corresponding to each object data includes: If the amount of object data included in the object data set is greater than a preset amount threshold, pulling each of the filter values ​​from the at least two reverse tables; The respective filter values ​​are pre-screened according to the screening ratio and a preset filter threshold, and the target object data is screened from the at least two reverse tables according to the screened filter values.

3. The method according to claim 2, characterized in that The filtering of the respective filtering values ​​according to the filtering ratio and the preset filtering threshold, and filtering the target object data from the at least two reverse tables according to the filtered filtering values, comprises: Obtaining a preset first filtering threshold, and screening to obtain a first target filtering value whose filtering value is greater than the first filtering threshold; Pulling first object data corresponding to the first target filter value from the at least two reverse tables; If the proportion of the first object data exceeds the screening proportion, a preset second filtering threshold is obtained, and a second target filtering value whose first target filtering value is greater than the second filtering threshold is obtained by screening, and the second filtering threshold is different from the first filtering threshold; Second object data corresponding to the second target filter value is obtained by screening the first object data, and the second object data is used as the target object data, wherein a proportion of the second object data does not exceed the screening proportion.

4. The method according to claim 2, characterized in that: Before writing the object data set and each filter value into at least two reverse tables according to the filter value corresponding to each object data, the method further includes: Acquire a single-table concurrency parameter of a database table for storing object data and the quantity of object data included in the object data set; The database table is divided into the at least two reverse tables according to the single-table concurrency parameter and the amount of the object data.

5. The method according to claim 2, characterized in that: The step of writing the object data set and each filter value into at least two reverse tables according to the filter value corresponding to each object data includes: Acquire a task identifier corresponding to the data extraction task, and generate a table identifier for each object data according to the task identifier and a filter value of each object data; According to the table identifier of each object data, the object data in the object data set and the filter values ​​corresponding to each object data are written into different reverse tables.

6. The method according to claim 5, characterized in that The determining of the preferred filtering value from the filtering values ​​corresponding to the target object data according to the extraction quantity of the data extraction task includes: Get the number of target object data corresponding to each filter value in each reverse table; The preferred filter value is determined according to the number of target object data corresponding to each filter value in each reverse table and the extraction number of the data extraction task.

7. The method according to claim 6, characterized in that Determining the preferred filter value according to the number of target object data corresponding to each filter value in each reverse table and the number of extractions of the data extraction task includes: Acquire a first quantity of target object data whose filter value is greater than or equal to the preferred filter value and a second quantity of target object data whose filter value is less than the preferred filter value; The preferred filter value is searched from each filter value corresponding to the target object data, so that the first number is less than or equal to the extracted number, and the sum of the first number and the second number is greater than the extracted number.

8. The method according to claim 6, characterized in that Determining the preferred filter value according to the number of target object data corresponding to each filter value in each reverse table and the number of extractions of the data extraction task includes: According to the number of target object data corresponding to each filter value, calculate the cumulative number of target object data corresponding to each filter value and smaller than each filter value; A target filter value is searched from each filter value corresponding to the target object data. If the target filter value and the cumulative number of target object data corresponding to the target filter value and the cumulative number of target object data less than the target filter value are close to the extraction number, the target filter value is used as the preferred filter value.

9. The method according to claim 2, characterized in that: The determining the candidate object data from the target object data according to the preferred filtering value comprises: Obtaining the filter value interval corresponding to the target object data in each reverse table; Determine the target reverse table into which the preferred filter value falls according to the filter value interval corresponding to each reverse table; The target reverse table is pulled in pages to extract target object data matching the filter value with the preferred filter value from the target reverse table, and the extracted target object data is used as the candidate object data.

10. The method according to any one of claims 1 to 9, characterized in that: The step of performing data extraction processing on the candidate object data to obtain the extraction object data of the data extraction task includes: Randomly sorting the candidate object data to obtain an object list; The extraction object data is obtained by randomly extracting from the object list according to the extraction quantity of the data extraction task, or by sorting and extracting according to the extraction quantity of the data extraction task and the object data in the object list.

11. The method according to any one of claims 1 to 9, characterized in that: After obtaining the object data set corresponding to the data extraction task, the method further includes: Writing the record information of each object in the object data set participating in the data extraction task into a forward table; After performing data extraction processing on the candidate object data to obtain the task object data of the data extraction task, the method further includes: The extraction object data is verified according to the forward table, and an extraction result is generated according to the verification result.

12. The method according to any one of claims 1 to 9, characterized in that: After the data extraction process is performed on the candidate object data to obtain the extraction object data of the data extraction task, the method further includes: Generate an extraction result according to the preferred filtering value and the extraction object data; storing a list of objects participating in the data extraction task under a preset key value, and storing the extraction result in an attribute value corresponding to the preset key value; In response to a viewing request of an object participating in the data extraction task, the preset key value is accessed according to the viewing request to read the extraction status of the object from the attribute value.

13. A data processing device, characterized in that: include: An acquisition module, used to acquire an object data set participating in a data extraction task, and assign a random filtering value to each object data in the object data set; A pre-screening module, used to obtain a pre-screening ratio corresponding to the data extraction task, and pre-screen the target object data from the object data set according to the pre-screening ratio and the filter value corresponding to each object data; a determination module, configured to determine a preferred filter value from the filter values ​​corresponding to the target object data according to the extraction quantity of the data extraction task, wherein the preferred filter value is used to filter the object data so that the number of the filtered object data is close to the extraction quantity; The extraction module is used to filter the target object data to obtain candidate object data according to the preferred filter value, and perform data extraction processing on the candidate object data to obtain the extraction object data of the data extraction task.

14. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of an electronic device, the electronic device executes the method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The computer program product comprises a computer program, wherein the computer program is stored in a computer-readable storage medium, and a processor of an electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device executes the method according to any one of claims 1 to 12.

Images