Queue data processing method

Abstract

The application relates to a queue data processing method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: taking received to-be-processed data as queue data, and writing the queue data into a preset queue table asynchronously; identifying each queue data in the queue table, and determining the type of the each queue data; respectively distributing the each queue data to a corresponding thread pool according to the type of the each queue data, and consuming the each queue data through a consumption mode of the corresponding thread pool. The method can ensure the data consumption sequence, improve the data processing efficiency, optimize the overall system performance, and improve the stability of a business function.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a queue data processing method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] In modern computing systems, with the rapid growth of data volume and the complexity of business scenarios, it is particularly important to use an efficient queue data processing method.

[0003] The common queue data consumption method at present is to asynchronously process messages through a thread pool. Using multiple threads can improve the throughput of data processing, but in actual application, this method may cause disorder of queue data consumption. Especially in some business functions that have strict requirements on processing order, the disordered consumption of queue data will lead to data anomalies or invalid business logic. For example, some businesses may rely on continuous event order for correct state update, and the disorder of consumption order will not only increase the subsequent data checking cost, but also may lead to system instability and decline of user experience. SUMMARY

[0004] Therefore, it is necessary to provide a queue data processing method, device, computer equipment, computer readable storage medium and computer program product to solve the above technical problems.

[0005] In a first aspect, the present application provides a queue data processing method. The method comprises:

[0006] Asynchronously writing received to-be-processed data as queue data into a preset queue table;

[0007] Identifying each queue data in the queue table and determining the type of each queue data;

[0008] According to the type of each queue data, respectively assigning each queue data to a corresponding thread pool, and consuming each queue data through the consumption mode of the corresponding thread pool.

[0009] In one embodiment, the identifying each queue data in the queue table and determining the type of each queue data comprises:

[0010] In the case where each queue data has no type identifier, obtaining each document identifier in the queue table; the document identifier is the identifier of the document corresponding to the queue to which each queue data belongs;

[0011] According to the number of queue data under each document identifier, determining the type of each queue data.

[0012] In one of the embodiments, the determining the type of each queue data according to the number of queue data under the document identifier comprises:

[0013] The number of queue data under the document identifier is determined in real time for each document identifier;

[0014] In the case that the number of queue data under the document identifier is greater than the preset number, all the queue data under the document identifier is marked as sequential queue data.

[0015] In one of the embodiments, the method further comprises:

[0016] After traversing the queue list, if the number of queue data under the document identifier is less than the preset number, the queue data under the document identifier is marked as unordered queue data.

[0017] In one of the embodiments, the identifying each queue data in the queue list and determining the type of each queue data further comprises:

[0018] In the case that each queue data has a type identifier, the type of each queue data is determined according to the type identifier of each queue data.

[0019] In one of the embodiments, the distributing each queue data to the corresponding thread pool according to the type of each queue data comprises:

[0020] For each queue data, if the type of the queue data is sequential queue data, the queue data is distributed to an ordered thread pool; the ordered thread pool comprises a single thread;

[0021] If the type of the queue data is unordered queue data, the queue data is distributed to an unordered thread pool; the unordered thread pool comprises multiple threads.

[0022] In a second aspect, the application further provides a queue data processing device. The device comprises:

[0023] A data writing module is configured to write the received to-be-processed data as queue data into a preset queue list asynchronously;

[0024] A type identifying module is configured to identify each queue data in the queue list and determine the type of each queue data;

[0025] A data distributing module is configured to distribute each queue data to the corresponding thread pool according to the type of each queue data, and consume each queue data through the consumption mode of the corresponding thread pool.

[0026] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0027] The received to-be-processed data is written into a preset queue list as queue data in an asynchronous manner;

[0028] Each queue data in the queue list is identified to determine the type of each queue data;

[0029] Each queue data is respectively allocated to a corresponding thread pool according to the type of each queue data, and each queue data is consumed by a consumption mode of the corresponding thread pool.

[0030] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0031] The received to-be-processed data is written into a preset queue list as queue data in an asynchronous manner;

[0032] Each queue data in the queue list is identified to determine the type of each queue data;

[0033] Each queue data is respectively allocated to a corresponding thread pool according to the type of each queue data, and each queue data is consumed by a consumption mode of the corresponding thread pool.

[0034] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program, and the computer program is executed by a processor to implement the following steps:

[0035] The received to-be-processed data is written into a preset queue list as queue data in an asynchronous manner;

[0036] Each queue data in the queue list is identified to determine the type of each queue data;

[0037] Each queue data is respectively allocated to a corresponding thread pool according to the type of each queue data, and each queue data is consumed by a consumption mode of the corresponding thread pool.

[0038] The aforementioned queue data processing method, apparatus, computer equipment, storage medium, and computer program product asynchronously writes the received data to be processed as queue data into a preset queue list; identifies each queue data in the queue list to determine its type; and allocates each queue data to a corresponding thread pool according to its type, consuming the data through the corresponding thread pool's consumption method. This method, by writing queue data into a queue list and rationally allocating it to appropriate thread pools for consumption based on its type, flexibly adapts to the processing needs of different data types, ensuring that data requiring sequential consumption is effectively processed without affecting the concurrent consumption efficiency of other data types. In other words, it improves data processing efficiency while maintaining the order of data consumption, thereby optimizing overall system performance and enhancing the stability of business functions. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating a queue data processing method in one embodiment;

[0040] Figure 2 This is a flowchart illustrating the queue data processing method in another embodiment;

[0041] Figure 3 This is a flowchart illustrating the queue data processing method in yet another embodiment;

[0042] Figure 4 This is a structural block diagram of a queue data processing device in one embodiment;

[0043] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] In one embodiment, such as Figure 1As shown, a queue data processing method is provided, and the embodiment is exemplified by the method applied to a terminal. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction of the terminal and the server. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server can be implemented by an independent server or a server cluster composed of multiple servers. In the embodiment, the method includes the following steps:

[0046] In step S110, the received to-be-processed data is written into a preset queue table as queue data in an asynchronous manner.

[0047] The queue, abbreviated as Q, is a linear table with limited operations, which allows insertion only at one end of the table and deletion only at the other end of the table. Inserting an element into the queue is called enqueue or in-queue, and deleting an element is called dequeue or out-of-queue.

[0048] The queue table can be stored in a database table, a key-value pair-based memory data structure, or a document-type database.

[0049] In a specific implementation, a database table can be established as a queue table, and the received to-be-processed data is written into the queue table in an asynchronous manner. The queue data in the queue table can correspond to a queue identifier, a document identifier corresponding to the queue, a queue operation method, a queue creation time, a type of queue data, etc. Table 1 below shows a schematic explanation of the structure of the queue table.

[0050] Table 1 Queue table structure

[0051]

[0052] In step S120, each queue data in the queue table is identified to determine the type of each queue data.

[0053] In a specific implementation, when the queue data is written into the queue table, the field value of the fd_order (i.e., type identifier) field in Table 1 can be set to indicate the type of the queue data. When the queue data is allocated, if the queue data has a type identifier, the type of the queue data can be determined according to the type identifier. If the queue data does not have a type identifier, the type of each queue data can be identified by a preset type identification algorithm.

[0054] In step S130, each queue data is allocated to a corresponding thread pool according to the type of each queue data, and each queue data is consumed by the consumption mode of the corresponding thread pool.

[0055] Among them, the thread pool is a management framework that creates a certain number of threads in advance. When the task arrives, the thread pool gets the thread from the pool to execute the task, and the thread is not destroyed after the task is completed, but returned to the pool for next use. The core idea of this mechanism is thread reuse, which avoids frequent creation and destruction of threads. Among them, the thread is the smallest unit that can be scheduled by the operating system for operation. It is contained in the process and is the actual operation unit in the process. A process can have multiple threads running concurrently, each thread executing different tasks in parallel.

[0056] In a specific implementation, a corresponding thread pool can be set according to the type of queue data, and each thread pool corresponds to a specific data type and consumption scenario. After identifying the data type, each queue data is allocated to the corresponding thread pool. For example, for queue data that needs to be consumed in order, it is allocated to an ordered thread pool. For data that can be processed in parallel, it is allocated to an unordered thread pool. Through the respective thread pool, the queue data is consumed asynchronously in the corresponding consumption mode. A message confirmation mechanism can be used in the consumption process to ensure that the data is successfully processed. And it also tracks the consumption state in real time, records the success and failure of data consumption, and provides a feedback mechanism so that the system can adjust the processing strategy in time.

[0057] In the above queue data processing method, the received to-be-processed data is written into the preset queue list as queue data; each queue data in the queue list is identified to determine the type of each queue data; and each queue data is allocated to a corresponding thread pool according to the type of each queue data, and each queue data is consumed by the consumption mode of the corresponding thread pool. This method writes queue data into the queue list, identifies the type of queue data, and reasonably allocates queue data to the corresponding thread pool for consumption. This method can flexibly adapt to the processing needs of different data types, ensure that data that needs to be consumed in order is effectively processed, and at the same time does not affect the concurrent consumption efficiency of other types of data, that is, it can ensure the order of data consumption while improving the efficiency of data processing, thereby optimizing the overall system performance and improving the stability of business functions.

[0058] In an example embodiment, the step S120 of identifying each queue data in the queue list and determining the type of each queue data includes:

[0059] Case one: in the case where each queue data has a type identifier, the type of each queue data is determined according to the type identifier of each queue data.

[0060] Specifically, when the queue data is written into the queue table, the field value of the fd_order (i.e., type identifier) field in Table 1 is set, and then when the queue data is allocated, the type of the queue data can be directly determined according to the correspondence between the type identifier and the type. For example, the type identifier is set to 0 for sequential queue data and 1 for unordered queue data, and when the type identifier of a certain queue data is 0, it can be determined that it is sequential queue data.

[0061] Case two: In the case where each queue data does not have a type identifier, each document identifier in the queue table is obtained; the document identifier is the identifier of the document corresponding to the queue to which each queue data belongs; and the type of each queue data is determined according to the number of queue data under each document identifier.

[0062] In a specific implementation, if the data in the queue table does not have a type identifier, the type of each queue data can be identified according to a preset type identification algorithm. Specifically, each document identifier in the queue table can be identified and obtained, and the document identifier is used to distinguish different data sources or businesses, so the queue data under the same document identifier needs to be consumed sequentially. Therefore, the number of queue data under each document identifier can be determined, and the type of the corresponding queue data can be determined according to the number.

[0063] In this embodiment, the type of each queue data is automatically determined by the provided type identification algorithm, and the queue data is allocated, so that the type identification in the case where each queue data does not have a type identifier is realized, and the sequential consumption or unordered fast consumption is realized according to the characteristics of the queue data, thereby providing flexibility of queue consumption.

[0064] In an example embodiment, the step of determining the type of each queue data according to the number of queue data under each document identifier includes: determining the number of queue data under the document identifier in real time for each document identifier; and in the case where it is detected that the number of queue data under the document identifier is greater than a preset number, marking all queue data under the document identifier as sequential queue data.

[0065] In a specific implementation, the number of queue data under each document identifier can be determined in real time, that is, each time a new queue data is identified, the number of queue data under the corresponding document identifier can be updated in real time. When it is detected that the number of queue data under a certain document identifier is greater than a preset number, the queue data under the document identifier is marked as sequential queue data. For other data in the queue table that has not been identified yet, when it is identified that it belongs to the document identifier, it can be directly marked as sequential queue data without the need for type identification again.

[0066] In actual application, the preset number can be 2, that is, when there are at least two queue data under a certain document identifier, it indicates that there are multiple queues under the data of the document, and then the queue data under the document identifier can be marked as sequential queue data which needs to be consumed in sequence. Taking the value 2 as the basis for dividing the sequential queue and the unordered queue, the identification efficiency can be improved on the basis of ensuring the accuracy of the identification result.

[0067] It should be noted that in some other application scenarios, the preset number can also be determined according to actual needs, and the present application does not make specific limitations thereto.

[0068] In the embodiment, unnecessary repeated identification is avoided by real-time detection and marking of data, the response speed of the system is improved, and the logic of subsequent data processing can be simplified. The subsequent data can directly refer to the existing marking, and the system complexity is reduced.

[0069] In an example embodiment, the method further comprises: after traversing the queue list, if the number of queue data under a document identifier is less than the preset number, marking the queue data under the document identifier as unordered queue data.

[0070] In specific implementation, when identifying each queue data in the queue list, if after traversing all the queue data in the queue list, the number of queue data under a certain document identifier is less than the preset number, for example, less than 2, it indicates that the queue data under the document identifier does not need to be consumed in sequence, and then it can be marked as unordered queue data.

[0071] In the embodiment, by adding the marking mechanism for the queue data under the document identifier which is less than the preset number, not only the strict control of the sequential data is realized, but also the appropriate classification of the unordered data is completed, and the overall design guarantees the accuracy and efficiency of data processing.

[0072] In an example embodiment, the above step S130 respectively allocates each queue data to the corresponding thread pool according to the type of each queue data, specifically including: for each queue data, if the type of the queue data is sequential queue data, then the queue data is allocated to an ordered thread pool; the ordered thread pool includes a single thread; if the type of the queue data is unordered queue data, then the queue data is allocated to an unordered thread pool; the unordered thread pool includes multiple threads.

[0073] In specific implementation, allocating the queue data to the appropriate thread pool for processing according to the data type of each queue data includes: if the queue data is sequential queue data, it is allocated to a specially designed ordered thread pool, and a thread is created in the ordered thread pool to ensure the processing order. The ordered thread pool continuously consumes the queue data marked for sequential consumption, and if no queue data is found to be consumed, the thread is put to sleep for a period of time to reduce idling.

[0074] If the queue data is unordered queue data, the queue data is assigned to an unordered thread pool, and multiple threads are created in the unordered thread pool to achieve efficient concurrent processing. Since the multi-thread execution itself is unordered, only the consumption of the unordered queue data by each thread is required.

[0075] In the embodiment, in the data consumption process, the sequential queue data adopts a single-thread consumption mode to ensure that the data is processed one by one in a fixed order; and other queue data in the multi-thread pool enjoys efficient operation brought by parallel consumption. Thus, through the characteristics of the queue data, unordered fast consumption or sequential consumption can be performed as needed, and flexibility of queue consumption is provided.

[0076] Reference Figure 2 A flowchart of a queue data processing method provided for another embodiment is shown in the embodiment, and the method includes the following steps:

[0077] In step S201, received to-be-processed data is written as queue data in a preset queue table asynchronously;

[0078] In step S202, each queue data in the queue table is identified to determine the type of each queue data;

[0079] In step S203, in a case where each queue data has a type identifier, the type of each queue data is determined according to the type identifier of each queue data;

[0080] In step S204, in a case where each queue data does not have a type identifier, each document identifier in the queue table is obtained;

[0081] In step S205, for each document identifier, the number of queue data under the document identifier is determined in real time;

[0082] In step S206, in a case where the number of queue data under the document identifier is greater than a preset number, all queue data under the document identifier is marked as sequential queue data;

[0083] In step S207, after traversing the queue table, in a case where the number of queue data under the document identifier is less than the preset number, the queue data under the document identifier is marked as unordered queue data;

[0084] In step S208, for each queue data, if the type of the queue data is sequential queue data, the queue data is assigned to an ordered thread pool; the ordered thread pool includes a single thread;

[0085] In step S209, if the type of the queue data is unordered queue data, the queue data is assigned to an unordered thread pool; the unordered thread pool includes multiple threads.

[0086] The method writes queue data into a queue table, reasonably allocates the queue data to corresponding thread pools for consumption by identifying the types of the queue data, and can flexibly adapt to the processing requirements of different data types, ensures that the data requiring sequential consumption is effectively processed, and does not affect the concurrent consumption efficiency of other types of data, that is, the efficiency of data processing can be improved while ensuring the sequentiality of data consumption, thereby optimizing the overall system performance and improving the stability of business functions. Through the provided type identification algorithm, the types of various queue data are automatically judged, so that the queue data is allocated, the type identification in the case where the queue data has no type identifier can be realized, the unordered fast consumption or sequential consumption is realized according to the characteristics of the queue data, and the flexibility of queue consumption is provided.

[0087] In an embodiment, in order to facilitate the understanding of the embodiments of the present application by those skilled in the art, the following will be described in combination with specific examples of the accompanying drawings. Referring to Figure 3 , another flowchart of a queue data processing method is shown, a database table is established as a queue table, business sends messages only need to write data into the queue table asynchronously, two sets of thread pools are started, one is an unordered thread pool and one is an ordered thread pool, and the queue data is processed quickly or sequentially according to the business needs. Specifically, it includes:

[0088] (1) Write queue data: provide an addQueue interface to the business side to call to write data, wherein the queue can set the field fd_order of table 1 as whether to consume the queue sequentially, or not set and give the built-in type identification algorithm of the program to calculate.

[0089] (2) Built-in algorithm: set the queue data under the same document to be executed in sequence, and the design is based on a database table, and a corresponding sql needs to be executed to judge whether the data is sequential queue data.

[0090] 2.1) Execute sql: select distinct fd_doc_id from queue where fd_order!= true, get how many queue data of document id under the current queue table.

[0091] 2.2) Through the fd_doc_id set returned by 2.1), execute the paging sql: select fd_id from queue where fd_doc_id = 'queue data processing' and fd_order!= true limit 2 according to each fd_doc_id.

[0092] 2.3) If the return queue data of the same document is greater than or equal to 2, it indicates that there are multiple queue data under the document, and the data under the document is marked as sequential queue data, and the SQL update queue set fd_order = true where fd_doc_id = 'queue data processing' is executed.

[0093] In the method, the invoker can provide an identifier for the data according to the needs of the invoker, or calculate whether the data needs to be sequentially consumed through a built-in algorithm, and then insert the data into the queue table. Subsequent queue data consumption is automatically completed by the program.

[0094] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0095] Based on the same inventive concept, the embodiments of the present application also provide a queue data processing apparatus for implementing the above-mentioned queue data processing method. The problem-solving implementation scheme provided by the apparatus is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more queue data processing apparatus embodiments provided below can refer to the limitations of the queue data processing method described above, which will not be repeated here.

[0096] In one embodiment, as shown in Figure 4 a queue data processing apparatus is provided, comprising a data writing module 410, a type identification module 420, and a data distribution module 430, wherein:

[0097] The data writing module 410 is configured to write the received to-be-processed data as queue data into a preset queue table asynchronously;

[0098] The type identification module 420 is configured to identify each queue data in the queue table and determine the type of each queue data;

[0099] The data distribution module 430 is configured to distribute each queue data to a corresponding thread pool according to the type of each queue data, and consume each queue data through the consumption mode of the corresponding thread pool.

[0100] In one of the embodiments, the type identifying module 420 is further configured to, in the case that the queue data does not have a type identifier, acquire each document identifier in the queue list; the document identifier is an identifier of a document corresponding to each queue data; and determine the type of each queue data according to the number of queue data under each document identifier.

[0101] In one of the embodiments, the type identifying module 420 is further configured to, for each document identifier, determine the number of queue data under the document identifier in real time; and in the case that the number of queue data under the document identifier is greater than a preset number, mark all queue data under the document identifier as sequential queue data.

[0102] In one of the embodiments, the type identifying module 420 is further configured to, after traversing the queue list, if the number of queue data under the document identifier is less than the preset number, mark the queue data under the document identifier as disordered queue data.

[0103] In one of the embodiments, the type identifying module 420 is further configured to, in the case that the queue data has a type identifier, determine the type of each queue data according to the type identifier of each queue data.

[0104] In one of the embodiments, the data allocating module 430 is further configured to, for each queue data, if the type of the queue data is sequential queue data, allocate the queue data to an ordered thread pool; the ordered thread pool includes a single thread; and if the type of the queue data is disordered queue data, allocate the queue data to a disordered thread pool; the disordered thread pool includes multiple threads.

[0105] The above modules in the queue data processing apparatus can be realized by software, hardware and combinations thereof in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules.

[0106] In one of the embodiments, a computer device is provided, which can be a terminal, and the internal structure diagram of the computer device can be as shown in FIG. 1. Figure 5As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless mode can be achieved through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a queue data processing method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad provided on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0107] Those skilled in the art can understand that, Figure 5 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0108] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above method embodiments.

[0109] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0110] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0111] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0112] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0113] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0114] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method of processing queue data, characterized by, The method comprises: writing received to-be-processed data as queue data into a preset queue list asynchronously; identifying each queue data in the queue list and determining the type of the each queue data; specifically, identifying whether each queue data in the queue list has a type identifier; for queue data without the type identifier, obtaining a document identifier corresponding to a queue to which the queue data belongs; determining the type of the queue data according to the number of queue data under the document identifier; the type includes sequential queue data and unordered queue data; according to the type of the each queue data, distributing the each queue data to a corresponding thread pool respectively, and consuming the each queue data through a consumption mode of the corresponding thread pool; for sequential queue data, the sequential queue data is distributed to an ordered thread pool including only a single thread; for unordered queue data, the unordered queue data is distributed to an unordered thread pool including multiple threads.

2. The method of claim 1, wherein, The method further comprises: determining the number of queue data under each document identifier in real time; in a case where it is detected that the number of queue data under the document identifier is greater than a preset number, marking all queue data under the document identifier as sequential queue data.

3. The method of claim 2, wherein, The method further comprises: after traversing the queue list, if the number of queue data under the document identifier is less than the preset number, marking the queue data under the document identifier as unordered queue data.

4. The method of claim 1, wherein, The identifying each queue data in the queue list and determining the type of the each queue data further comprises: for queue data in the queue list having a type identifier, determining the type of the queue data according to the type identifier of the queue data.

5. The method of claim 1, wherein, The ordered thread pool performs a sleep action when there is no queue data to consume.

6. A queue data processing apparatus characterized by comprising: The apparatus comprises: a data writing module configured to write received to-be-processed data as queue data into a preset queue list asynchronously; a type identifying module configured to identify each queue data in the queue list and determine the type of the each queue data; specifically, the type identifying module is configured to identify whether each queue data in the queue list has a type identifier; for queue data without the type identifier, the type identifying module is configured to obtain a document identifier corresponding to a queue to which the queue data belongs; the type identifying module is configured to determine the type of the queue data according to the number of queue data under the document identifier; the type includes sequential queue data and unordered queue data; a data distributing module configured to distribute the each queue data to a corresponding thread pool according to the type of the each queue data respectively, and consume the each queue data through a consumption mode of the corresponding thread pool; for sequential queue data, the data distributing module is configured to distribute the sequential queue data to an ordered thread pool including only a single thread; for unordered queue data, the data distributing module is configured to distribute the unordered queue data to an unordered thread pool including multiple threads.

7. The apparatus of claim 6, wherein, The type identifying module is further configured to, in a case where the each queue data has a type identifier, determine the type of the each queue data according to the type identifier of the each queue data. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. The computer program is executed by the processor to implement the steps of the queue data processing method in any one of claims 1 to 5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the queue data processing method in any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the queue data processing method in any one of claims 1 to 5.

Images