Task duplicate removal method and system for human-enterprise algorithm task queue

Abstract

The invention discloses a task deduplication method and system for a human-enterprise algorithm task queue, and belongs to the technical field of computer algorithm application. The method comprises the steps that when it is determined that tasks in a plurality of pre-established task queues need to be processed, the tasks in the task queues are received according to the priority sequence of the task queues, the receiving time of the tasks is recorded, and recording time information is generated; based on a preset human-enterprise algorithm task queue model, according to the recording time information, scoring the received task, and obtaining a score; and according to the score, determining whether the received task is the task with the repeated task name, and if the received task is the task with the repeated task name, removing the received task with the repeated task name. According to the application of the method, the response time to the task is shortened.

Description

Technical Field

[0001] The present invention relates to the field of computer algorithm application technology, and more specifically, to a task deduplication method and system for a human-enterprise algorithm task queue. Background Art

[0002] Human-enterprise algorithm:

[0003] The person-enterprise algorithm calculates based on the relationship between enterprises and between enterprises and names, and concludes which "Zhang San" in each company is the same person;

[0004] Task queue in human-enterprise algorithm:

[0005] Running the human-enterprise algorithm is expensive (consuming memory, CPU resources, and taking a lot of time). Under limited resources, a reasonable task queue design is a key component to ensure the timely update of human-enterprise data.

[0006] The tasks of human-enterprise algorithms have the following characteristics:

[0007] Tasks are heterogeneous and determine the time consumed. The distribution of consumption time follows the Pareto principle:

[0008] For example, "Zhang Wei" is a common name. For a single name, the amount of data involved is nearly one million, and the consumption time (computation time) is up to tens of minutes; while for rare names, the calculation can be completed within hundreds of milliseconds.

[0009] The task is repeated:

[0010] When the same person's name appears in several companies and changes successively, multiple calculations will be triggered. Summary of the invention

[0011] In view of the above problems, the present invention proposes a method for deduplicating tasks in a human-enterprise algorithm task queue, comprising:

[0012] When it is determined that tasks in the pre-established multiple task queues need to be processed, the tasks in the task queues are received according to the priority order of the multiple task queues, and the receiving time of the tasks is recorded to generate recording time information;

[0013] Based on the preset human-enterprise algorithm task queue model, score the received tasks according to the recorded time information and obtain the scoring scores;

[0014] According to the scoring score, it is determined whether the received task is a task with a repeated task name. If it is a repeated task, the task with the repeated task name received this time is removed.

[0015] Optionally, when multiple task queues are pre-established, multiple task queues for storing tasks are established according to the task types of multiple tasks in the human-enterprise algorithm, and priorities are set for the multiple task queues according to priority requirements of the tasks.

[0016] Optionally, task parameters of each task queue in the multiple task queues are complementary and equal.

[0017] Optionally, establish a human-enterprise algorithm task queue model, including:

[0018] Determine the expected inter-arrival time λ of tasks in the human-enterprise algorithm i and expected consumption time μ i ;

[0019] Determine the expected inter-arrival time λ i and expected consumption time μ i Perform a prior distribution, and determine parameter characteristics of different task parameters in the human-enterprise algorithm through the prior distribution results;

[0020] Based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a human-enterprise algorithm task queue model is built in the human-enterprise algorithm.

[0021] Optionally, based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a task queue model of the human-enterprise algorithm is constructed in the human-enterprise algorithm, including:

[0022] Based on the parameter characteristics, the service rules of the original task queuing model of the human-enterprise algorithm are replaced with a deduplication strategy, and the Kendall notation symbol is expanded to build a human-enterprise algorithm task queue model in the human-enterprise algorithm.

[0023] Optionally, extend Kendall notation to include:

[0024] The symbol representing the arrival interval of tasks in the original task queuing model of the human-enterprise algorithm is replaced with the symbol representing the distribution of task arrivals, and the number of consumption values ​​of the task queue is adjusted to 1 as a constant.

[0025] Optional, human-enterprise algorithm task queue model, as follows:

[0026] G1 / G2 / c / ∞1 / ∞2 / dedupe FCFS

[0027] Wherein, G1 is the distribution of task arrival, and G1 is the sum of the distribution intensities of different task arrivals;

[0028] G2 is the time required for the task to be consumed, and G2 follows an exponential distribution;

[0029] c is the number of consumers in the task queue;

[0030] ∞1 is the length of the task queue;

[0031] ∞2 is the total number of tasks;

[0032] dedupe FCFS is a deduplication strategy;

[0033] The deduplication strategy includes: a maximum value strategy, a minimum value strategy and an average value strategy.

[0034] Optionally, based on a preset human-enterprise algorithm task queue model, the received tasks are scored according to the recorded time information, and a scoring score is obtained, including:

[0035] If the received task is received for the first time, the scoring score is determined according to the task receiving time recorded in the recording time information;

[0036] If the received task is received more than twice, a deduplication strategy is used to determine the score based on the current receiving time and the last receiving time in the recorded time information.

[0037] On the other hand, the present invention also proposes a task deduplication system for a human-enterprise algorithm task queue, comprising:

[0038] The recording unit is used for receiving the tasks in the task queues according to the priority order of the multiple task queues when it is determined that the tasks in the pre-established multiple task queues need to be processed, and recording the receiving time of the tasks to generate recording time information;

[0039] A scoring unit, configured to score the received tasks based on a preset human-enterprise algorithm task queue model and the recorded time information, and obtain a scoring score;

[0040] The task deduplication unit is used to determine whether the received task is a task with a duplicate task name according to the scoring score, and if it is a duplicate, remove the task with the duplicate task name received this time.

[0041] Optionally, when pre-establishing multiple task queues, the recording unit establishes multiple task queues for storing tasks according to the task types of multiple tasks in the human-enterprise algorithm, and sets priorities for the multiple task queues according to priority requirements of the tasks.

[0042] Optionally, task parameters of each task queue in the multiple task queues are complementary and equal.

[0043] Optionally, establish a human-enterprise algorithm task queue model, including:

[0044] Determine the expected inter-arrival time λ of tasks in the human-enterprise algorithmi and expected consumption time μ i ;

[0045] Determine the expected inter-arrival time λ i and expected consumption time μ i Perform a prior distribution, and determine parameter characteristics of different task parameters in the human-enterprise algorithm through the prior distribution results;

[0046] Based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a human-enterprise algorithm task queue model is built in the human-enterprise algorithm.

[0047] Optionally, based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a task queue model of the human-enterprise algorithm is constructed in the human-enterprise algorithm, including:

[0048] Based on the parameter characteristics, the service rules of the original task queuing model of the human-enterprise algorithm are replaced with a deduplication strategy, and the Kendall notation symbol is expanded to build a human-enterprise algorithm task queue model in the human-enterprise algorithm.

[0049] Optionally, extend Kendall notation to include:

[0050] The symbol representing the arrival interval of tasks in the original task queuing model of the human-enterprise algorithm is replaced with the symbol representing the distribution of task arrivals, and the number of consumption values ​​of the task queue is adjusted to 1 as a constant.

[0051] Optional, human-enterprise algorithm task queue model, as follows:

[0052] G1 / G2 / c / ∞1 / ∞2 / dedupe FCFS

[0053] Wherein, G1 is the distribution of task arrival, and G1 is the sum of the distribution intensities of different task arrivals;

[0054] G2 is the time required for the task to be consumed, and G2 follows an exponential distribution;

[0055] c is the number of consumers in the task queue;

[0056] ∞1 is the length of the task queue;

[0057] ∞2 is the total number of tasks;

[0058] dedupe FCFS is a deduplication strategy;

[0059] The deduplication strategy includes: a maximum value strategy, a minimum value strategy and an average value strategy.

[0060] Optionally, based on a preset human-enterprise algorithm task queue model, the received tasks are scored according to the recorded time information, and a scoring score is obtained, including:

[0061] If the received task is received for the first time, the scoring score is determined according to the task receiving time recorded in the recording time information;

[0062] If the received task is received more than twice, a deduplication strategy is used to determine the score based on the current receiving time and the last receiving time in the recorded time information.

[0063] In yet another aspect, the present invention further provides a computing device, comprising: one or more processors;

[0064] a processor for executing one or more programs;

[0065] When the one or more programs are executed by the one or more processors, the above-described method is implemented.

[0066] In yet another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, the method described above is implemented.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] The present invention provides a method for deduplicating tasks for a human-enterprise algorithm task queue, comprising: when it is determined that tasks in a plurality of pre-established task queues need to be processed, tasks in the task queues are received according to the priority order of the plurality of task queues, and the receiving time of the tasks is recorded to generate recorded time information; based on a preset human-enterprise algorithm task queue model, the received tasks are scored according to the recorded time information, and a scoring score is obtained; according to the scoring score, it is determined whether the received tasks are tasks with duplicate task names, and if they are duplicate, the tasks with the duplicate task names received this time are removed. The application of the present invention shortens the response time to tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a flow chart of the method of the present invention;

[0070] Figure 2 This is a deduplication flow chart of the method of the present invention for a queue;

[0071] Figure 3 It is a structural diagram of the system of the present invention. DETAILED DESCRIPTION

[0072] Now, exemplary embodiments of the present invention are described with reference to the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to disclose the present invention in detail and completely and to fully convey the scope of the present invention to those skilled in the art. The terms used in the exemplary embodiments shown in the accompanying drawings are not intended to limit the present invention. In the accompanying drawings, the same units / elements are marked with the same reference numerals.

[0073] Unless otherwise specified, the terms (including technical terms) used herein have the commonly understood meanings to those skilled in the art. In addition, it is understood that the terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

[0074] Embodiment 1:

[0075] The present invention proposes a task deduplication method for a human-enterprise algorithm task queue, such as Figure 1 As shown, including:

[0076] Step 1: When it is determined that tasks in a plurality of pre-established task queues need to be processed, the tasks in the task queues are received according to the priority order of the plurality of task queues, and the receiving time of the tasks is recorded to generate recording time information;

[0077] Step 2: Based on the preset human-enterprise algorithm task queue model, score the received tasks according to the recorded time information, and obtain the score;

[0078] Step 3: Determine whether the received task is a task with a repeated task name based on the scoring score. If it is a repeated task, remove the task with the repeated task name received this time.

[0079] When multiple task queues are pre-established, multiple task queues for storing tasks are established according to the task types of multiple tasks in the human-enterprise algorithm, and priorities are set for the multiple task queues according to priority requirements of the tasks.

[0080] Among them, the task parameters of each task queue in the multiple task queues are complementary and equal.

[0081] Among them, the human-enterprise algorithm task queue model is established, including:

[0082] Determine the expected inter-arrival time λ of tasks in the human-enterprise algorithm i and expected consumption time μ i ;

[0083] Determine the expected inter-arrival time λ i and expected consumption time μi Perform a prior distribution, and determine parameter characteristics of different task parameters in the human-enterprise algorithm through the prior distribution results;

[0084] Based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a human-enterprise algorithm task queue model is built in the human-enterprise algorithm.

[0085] Among them, based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a task queue model of the human-enterprise algorithm is built in the human-enterprise algorithm, including:

[0086] Based on the parameter characteristics, the service rules of the original task queuing model of the human-enterprise algorithm are replaced with a deduplication strategy, and the Kendall notation symbol is expanded to build a human-enterprise algorithm task queue model in the human-enterprise algorithm.

[0087] Among them, the Kendall notation symbols are expanded to include:

[0088] The symbol representing the arrival interval of tasks in the original task queuing model of the human-enterprise algorithm is replaced with the symbol representing the distribution of task arrivals, and the number of consumption values ​​of the task queue is adjusted to 1 as a constant.

[0089] Among them, the human-enterprise algorithm task queue model is as follows:

[0090] G1 / G2 / c / ∞1 / ∞2 / dedupe FCFS

[0091] Wherein, G1 is the distribution of task arrival, and G1 is the sum of the distribution intensities of different task arrivals;

[0092] G2 is the time required for the task to be consumed, and G2 follows an exponential distribution;

[0093] c is the number of consumers in the task queue;

[0094] ∞1 is the length of the task queue;

[0095] ∞2 is the total number of tasks;

[0096] dedupe FCFS is a deduplication strategy;

[0097] The deduplication strategy includes: a maximum value strategy, a minimum value strategy and an average value strategy.

[0098] in,

[0099] Based on the preset human-enterprise algorithm task queue model, the received tasks are scored according to the recorded time information, and the scoring scores are obtained, including:

[0100] If the received task is received for the first time, the scoring score is determined according to the task receiving time recorded in the recording time information;

[0101] If the received task is received more than twice, a deduplication strategy is used to determine the score based on the current receiving time and the last receiving time in the recorded time information.

[0102] The present invention is further described below in conjunction with the application of the model in practice:

[0103] First, the existing queuing model is explained:

[0104] Modern queuing theory studies the random laws in queuing phenomena in service systems.

[0105] Kendall notation uses simple symbols to model queues, the most representative of which are:

[0106] M / M / 1 / ∞ / ∞ / FCFS

[0107] In the above model, the first M means that the arrival interval of the task obeys the exponential distribution Exp(1 / λ), that is, the expected arrival interval is λ, and the arrival of the task obeys the Poisson arrival N(t) with intensity λ, t≥0;

[0108] The second M means that the time required for the task follows the exponential distribution Exp(μ), and the expected time is 1 / μ.

[0109] 1 means that there is only one consumer in the task queue;

[0110] The first ∞ means that there is no limit on the maximum length of the task queue;

[0111] The second ∞ means that the total number of tasks is infinite;

[0112] The present invention takes into account the heterogeneity of human-enterprise algorithm tasks, proposes a new first-come-first-served rule that can remove duplicates, and expands the symbols in the Kendall notation to establish a human-enterprise algorithm task queuing model, as follows:

[0113] G / G / c / ∞ / ∞ / dedupe FCFS

[0114] The meaning of G is that the task arrival time and service time are both generally distributed;

[0115] c means that the number of task queue consumers is a constant;

[0116] The model established for the present invention is as follows:

[0117] Each task has parameters p∈{p i,i=1,2,…}, given parameter p=p i After that, the arrival of the task is subject to the strength λ i Poisson reaches N i (t), the consumption time distribution is subject to Exp(μ i );

[0118] The first G is the arrival distribution of tasks, which is the arrival distribution of tasks of different types that are independent of each other. i (t), that is, Its intensity is the sum of the intensities of each distribution

[0119] The second G is the time required for the task to be consumed by the task parameter p i Determined by, and subject to exponential distribution, that is, the time required for consumption x|p i ~Exp(μ i );

[0120] dedupe FCFS means first come first serve, and the parameter p i The same tasks can be deduplicated;

[0121] Among them, by determining the expected arrival interval λ i , expected consumption time μ i The prior distribution of these two parameters is used to model the characteristics of different parameters in the human-enterprise algorithm;

[0122] Among them, the more common the name of a task is, the shorter the expected arrival interval is and the longer the expected consumption time is, so λ i ,μ i Negatively correlated and subject to Pareto distribution;

[0123] The model established by the present invention includes: the score and deduplication strategy in the service rules

[0124] Among them, dedupe FCFS service rules include: multiple deduplication strategies;

[0125] Assume that the priority queue Q receives 3 tasks with different parameters, namely p1, p2, and p3, and the task scores are the acceptance times $s_i=t_i,t_i\in\R^+,i=1,2,3$ respectively;

[0126] If queue Q receives a task with parameter p2 for the second time, the receiving time of the task is recorded as t2′. After deduplication, the score is s2′.

[0127] Deduplication strategies, including:

[0128] Take the minimum value, that is, s2′=min(t2,t2′);

[0129] Take the maximum value, that is, s2′=max(t2,t2′);

[0130] Take the average value, that is, s2′=(t2+t2′) / 2;

[0131] The present invention uses the Monte Carlo method for simulation to evaluate the improvement brought by the new service rule design in the human-enterprise algorithm task queuing model;

[0132] λ i ,μ i They obey Pareto distribution with parameters of 0.8 and 1.2 respectively, with a total of 2000 groups;

[0133] Starting from 0 seconds, the producer and consumer are started, and the production task is stopped at 100 seconds. The simulation continues until all tasks are consumed.

[0134] The simulation was repeated 100 times to obtain the expected average response time corresponding to each rule, as shown in Table 1 below:

[0135] Table 1

[0136]

[0137] The present invention extends modern queuing theory, takes into account the heterogeneity of tasks, and proposes a new FIFO service rule with deduplication. In the scenario of simulating human-enterprise algorithm, it is found that the new service rule greatly shortens the average response time.

[0138] The present invention achieves the following objectives:

[0139] In view of the heterogeneity and deduplication of human-enterprise algorithm tasks, a human-enterprise algorithm task queuing model is established.

[0140] Propose new service rules that support deduplication to optimize average response time.

[0141] Monte Carlo simulations were used to evaluate the improvements brought by the new cohort design.

[0142] Embodiment 2:

[0143] On the other hand, the present invention can perform deduplication for a queue or a collection, such as Figure 2 As shown, including:

[0144] Each element in a queue or set Q has a value v and a score s. Q will sort all elements internally according to the size of s. The value v of each element is different and will not be repeated. When taking elements from Q, you can take the element according to the smallest / largest s, or you can take the element according to the v value;

[0145] The v value is set as the algorithm task parameter, and the s value is set as a timestamp. When the producer puts a task into Q, it checks whether the task parameter already exists, and then generates a new s value based on the document deduplication strategy. Consumers take tasks from Q in ascending order of s value.

[0146] The above deduplication method further includes:

[0147] Establish the human-enterprise algorithm task queuing model as follows:

[0148] G / G / c / ∞ / ∞ / dedupe FCFS

[0149] The meaning of G is that the task arrival time and service time are both generally distributed;

[0150] c means that the number of task queue consumers is a constant;

[0151] The model established for the present invention is as follows:

[0152] Each task has parameters p∈{p i ,i=1,2,…}, given parameter p=p i After that, the arrival of the task is subject to the strength λ i Poisson reaches N i (t), the consumption time distribution is subject to Exp(μ i );

[0153] The first G is the arrival distribution of tasks, which is the arrival distribution of tasks of different types that are independent of each other. i (t), that is, Its intensity is the sum of the intensities of each distribution

[0154] The second G is the time required for the task to be consumed by the task parameter p i Determined by, and subject to exponential distribution, that is, the time required for consumption x|p i ~Exp(μ i );

[0155] dedupe FCFS means first come first serve, and the parameter p i The same tasks can be deduplicated;

[0156] Among them, by determining the expected arrival interval λ i , expected consumption time μ i The prior distribution of these two parameters is used to model the characteristics of different parameters in the human-enterprise algorithm;

[0157] Among them, the more common the name of a task is, the shorter the expected arrival interval is and the longer the expected consumption time is, so λ i ,μi Negatively correlated and subject to Pareto distribution;

[0158] The established model includes: scores and deduplication strategies in service rules

[0159] Among them, dedupe FCFS service rules include: multiple deduplication strategies;

[0160] Assume that the priority queue Q receives 3 tasks with different parameters, namely p1, p2, and p3, and the task scores are the acceptance times $s_i=t_i,t_i\in\R^+,i=1,2,3$ respectively;

[0161] If queue Q receives a task with parameter p2 for the second time, the receiving time of the task is recorded as t2′. After deduplication, the score is s2′.

[0162] Deduplication strategies, including:

[0163] Take the minimum value, that is, s2′=min(t2,t2′);

[0164] Take the maximum value, that is, s2′=max(t2,t2′);

[0165] Take the average value, that is, s2′=(t2+t2′) / 2;

[0166] The present invention uses the Monte Carlo method for simulation to evaluate the improvement brought by the new service rule design in the human-enterprise algorithm task queuing model;

[0167] λ i ,μ i They obey Pareto distribution with parameters of 0.8 and 1.2 respectively, with a total of 2000 groups;

[0168] Starting from 0 seconds, the producer and consumer are started, and the production task is stopped at 100 seconds. The simulation continues until all tasks are consumed.

[0169] The simulation was repeated 100 times to obtain the expected average response time corresponding to each rule, as shown in Table 1 below:

[0170] Table 1

[0171]

[0172] The present invention extends modern queuing theory, takes into account the heterogeneity of tasks, and proposes a new FIFO service rule with deduplication. In the scenario of simulating human-enterprise algorithm, it is found that the new service rule greatly shortens the average response time.

[0173] Embodiment 3:

[0174] On the other hand, the present invention also proposes a task deduplication system 200 for a human-enterprise algorithm task queue, such as Figure 3 As shown, including:

[0175] The recording unit 201 is used for receiving the tasks in the task queues according to the priority order of the multiple task queues when it is determined that the tasks in the pre-established multiple task queues need to be processed, and recording the receiving time of the tasks to generate recording time information;

[0176] The scoring unit 202 is used to score the received tasks based on the preset human-enterprise algorithm task queue model and the recorded time information, and obtain a scoring score;

[0177] The task deduplication unit 203 is used to determine whether the received task is a task with a duplicate task name according to the scoring score, and if it is a duplicate, remove the task with the duplicate task name received this time.

[0178] When pre-establishing multiple task queues, the recording unit 201 establishes multiple task queues for storing tasks according to the task types of multiple tasks in the human-enterprise algorithm, and sets priorities for the multiple task queues according to the priority requirements of the tasks.

[0179] Among them, the task parameters of each task queue in the multiple task queues are complementary and equal.

[0180] Among them, the human-enterprise algorithm task queue model is established, including:

[0181] Determine the expected inter-arrival time λ of tasks in the human-enterprise algorithm i and expected consumption time μ i ;

[0182] Determine the expected inter-arrival time λ i and expected consumption time μ i Perform a prior distribution, and determine parameter characteristics of different task parameters in the human-enterprise algorithm through the prior distribution results;

[0183] Based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a human-enterprise algorithm task queue model is built in the human-enterprise algorithm.

[0184] Among them, based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a task queue model of the human-enterprise algorithm is built in the human-enterprise algorithm, including:

[0185] Based on the parameter characteristics, the service rules of the original task queuing model of the human-enterprise algorithm are replaced with a deduplication strategy, and the Kendall notation symbol is expanded to build a human-enterprise algorithm task queue model in the human-enterprise algorithm.

[0186] Among them, the Kendall notation symbols are expanded to include:

[0187] The symbol representing the arrival interval of tasks in the original task queuing model of the human-enterprise algorithm is replaced with the symbol representing the distribution of task arrivals, and the number of consumption values ​​of the task queue is adjusted to 1 as a constant.

[0188] Among them, the human-enterprise algorithm task queue model is as follows:

[0189] G1 / G2 / c / ∞1 / ∞2 / dedupe FCFS

[0190] Wherein, G1 is the distribution of task arrival, and G1 is the sum of the distribution intensities of different task arrivals;

[0191] G2 is the time required for the task to be consumed, and G2 follows an exponential distribution;

[0192] c is the number of consumers in the task queue;

[0193] ∞1 is the length of the task queue;

[0194] ∞2 is the total number of tasks;

[0195] dedupe FCFS is a deduplication strategy;

[0196] The deduplication strategy includes: a maximum value strategy, a minimum value strategy and an average value strategy.

[0197] in,

[0198] Based on the preset human-enterprise algorithm task queue model, the received tasks are scored according to the recorded time information, and the scoring scores are obtained, including:

[0199] If the received task is received for the first time, the scoring score is determined according to the task receiving time recorded in the recording time information;

[0200] If the received task is received more than twice, a deduplication strategy is used to determine the score based on the current receiving time and the last receiving time in the recorded time information.

[0201] The application of the present invention shortens the response time to the task.

[0202] Embodiment 4:

[0203] Based on the same inventive concept, the present invention also provides a computer device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of the method in the above embodiment.

[0204] Embodiment 5:

[0205] Based on the same inventive concept, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both a built-in storage medium in a computer device and an extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the method in the above embodiment.

[0206] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0207] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0208] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0209] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0210] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0211] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for deduplicating tasks in a human-enterprise algorithm task queue, characterized in that: include: When it is determined that tasks in a plurality of pre-established task queues need to be processed, the tasks in the task queues are received according to the priority order of the plurality of task queues, and the receiving time of the tasks is recorded to generate recording time information; Based on the preset human-enterprise algorithm task queue model, score the received tasks according to the recorded time information and obtain the scoring scores; According to the scoring score, it is determined whether the received task is a task with a repeated task name. If it is a repeated task, the task with the repeated task name received this time is removed.

2. The task deduplication method according to claim 1, characterized in that: When multiple task queues are pre-established, multiple task queues for storing tasks are established according to the task types of multiple tasks in the human-enterprise algorithm, and priorities are set for the multiple task queues according to priority requirements of the tasks.

3. The task deduplication method according to claim 1, characterized in that: The task parameters of each task queue in the multiple task queues are complementary and equal.

4. The task deduplication method according to claim 1, characterized in that: Establish a human-enterprise algorithm task queue model, including: Determine the expected inter-arrival time λ of tasks in the human-enterprise algorithm i and expected consumption time μ i ; Determine the expected inter-arrival time λ i and expected consumption time μ i Perform a prior distribution, and determine parameter characteristics of different task parameters in the human-enterprise algorithm through the prior distribution results; Based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a human-enterprise algorithm task queue model is built in the human-enterprise algorithm.

5. The task deduplication method according to claim 4, characterized in that: Based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a task queue model of the human-enterprise algorithm is constructed in the human-enterprise algorithm, including: Based on the parameter characteristics, the service rules of the original task queuing model of the human-enterprise algorithm are replaced with a deduplication strategy, and the Kendall notation symbol is expanded to build a human-enterprise algorithm task queue model in the human-enterprise algorithm.

6. The method for deduplication of tasks according to claim 5, characterized in that: The Kendall notation symbols are expanded to include: The symbol representing the arrival interval of tasks in the original task queuing model of the human-enterprise algorithm is replaced with the symbol representing the distribution of task arrivals, and the number of consumption values ​​of the task queue is adjusted to 1 as a constant.

7. The method for deduplication of tasks according to claim 1, characterized in that: The human-enterprise algorithm task queue model is as follows: G1 / G2 / c / ∞1 / ∞2 / dedupe FCFS Wherein, G1 is the distribution of task arrival, and G1 is the sum of the distribution intensities of different task arrivals; G2 is the time required for the task to be consumed, and G2 follows an exponential distribution; c is the number of consumers in the task queue; ∞1 is the length of the task queue; ∞2 is the total number of tasks; dedupe FCFS is a deduplication strategy; The deduplication strategy includes: a maximum value strategy, a minimum value strategy and an average value strategy.

8. The method for deduplication of tasks according to claim 1, characterized in that: The preset human-enterprise algorithm task queue model scores the received tasks according to the recorded time information and obtains the scoring scores, including: If the received task is received for the first time, the scoring score is determined according to the task receiving time recorded in the recording time information; If the received task is received more than twice, a deduplication strategy is used to determine the score based on the current receiving time and the last receiving time in the recorded time information.

9. A task deduplication system for human-enterprise algorithm task queue, characterized in that: include: The recording unit is used for receiving the tasks in the task queues according to the priority order of the multiple task queues when it is determined that the tasks in the pre-established multiple task queues need to be processed, and recording the receiving time of the tasks to generate recording time information; A scoring unit, configured to score the received tasks based on a preset human-enterprise algorithm task queue model and the recorded time information, and obtain a scoring score; The task deduplication unit is used to determine whether the received task is a task with a duplicate task name according to the scoring score, and if it is a duplicate, remove the task with the duplicate task name received this time.

10. The task deduplication system according to claim 9, characterized in that: When the recording unit pre-establishes multiple task queues, it establishes multiple task queues for storing tasks according to the task types of multiple tasks in the human-enterprise algorithm, and sets priorities for the multiple task queues according to priority requirements of the tasks.

11. The task deduplication system according to claim 9, characterized in that: The task parameters of each task queue in the multiple task queues are complementary and equal.

12. The task deduplication system according to claim 9, characterized in that: Establish a human-enterprise algorithm task queue model, including: Determine the expected inter-arrival time λ of tasks in the human-enterprise algorithm i and expected consumption time μ i ; Determine the expected inter-arrival time λ i and expected consumption time μ i Perform a prior distribution, and determine parameter characteristics of different task parameters in the human-enterprise algorithm through the prior distribution results; Based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a human-enterprise algorithm task queue model is built in the human-enterprise algorithm.

13. The task deduplication system according to claim 12, characterized in that: Based on the parameter characteristics and the original task queuing model of the human-enterprise algorithm, a task queue model of the human-enterprise algorithm is constructed in the human-enterprise algorithm, including: Based on the parameter characteristics, the service rules of the original task queuing model of the human-enterprise algorithm are replaced with a deduplication strategy, and the Kendall notation symbol is expanded to build a human-enterprise algorithm task queue model in the human-enterprise algorithm.

14. The task deduplication system according to claim 13, characterized in that: The Kendall notation symbols are expanded to include: The symbol representing the arrival interval of tasks in the original task queuing model of the human-enterprise algorithm is replaced with the symbol representing the distribution of task arrivals, and the number of consumption values ​​of the task queue is adjusted to 1 as a constant.

15. The task deduplication system according to claim 9, characterized in that: The human-enterprise algorithm task queue model is as follows: G1 / G2 / c / ∞1 / ∞2 / dedupe FCFS Wherein, G1 is the distribution of task arrival, and G1 is the sum of the distribution intensities of different task arrivals; G2 is the time required for the task to be consumed, and G2 follows an exponential distribution; c is the number of consumers in the task queue; ∞1 is the length of the task queue; ∞2 is the total number of tasks; dedupe FCFS is a deduplication strategy; The deduplication strategy includes: a maximum value strategy, a minimum value strategy and an average value strategy.

16. The task deduplication system according to claim 9, characterized in that: Said Based on the preset human-enterprise algorithm task queue model, the received tasks are scored according to the recorded time information, and the scoring scores are obtained, including: If the received task is received for the first time, the scoring score is determined according to the task receiving time recorded in the recording time information; If the received task is received more than twice, a deduplication strategy is used to determine the score based on the current receiving time and the last receiving time in the recorded time information.

17. A computer device, characterized in that: include: one or more processors; a processor for executing one or more programs; When the one or more programs are executed by the one or more processors, the method according to any one of claims 1 to 8 is implemented.

18. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the method according to any one of claims 1 to 8 is implemented.

Images