List processing method and device, equipment, medium and program product

Abstract

The invention provides a list processing method and device, equipment, a medium and a program product, and relates to the technical field of information processing.The list processing method comprises the steps that in response to monitoring that a list is stored in a target message partition, a target list is obtained from the target message partition, and the lists of the same customer are stored in the same message partition; determining a processing state of a target customer to which the target list belongs; if the processing state of the target customer is an unprocessed state, marking the target customer as a processing state, and storing the target list in a target batch; and when the target batch meets a preset condition, performing concurrent processing and warehousing on the list in the target batch. According to the method and the device, the problem of transaction rollback caused by concurrent processing library falling failure or transition failure of different lists of the same client can be avoided, and the library storage efficiency of the lists is improved.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a list processing method, device, equipment, medium and program product. Background Art

[0002] At present, the lists pushed by the upstream system are usually received by concurrent processing of multiple application instances, and the received lists are stored in batches in a single process. Among them, when the concurrent processing of different lists of the same customer in a batch of lists fails to be stored or the transition fails, all the transactions of all the lists in the batch need to be rolled back after the failure, and then retried. Summary of the invention

[0003] The present application provides a list processing method, device, equipment, medium and program product, which can avoid the transaction rollback problem caused by the failure of concurrent processing of different lists of the same customer to be stored in the warehouse or the failure of the jump, and improve the efficiency of the list storage.

[0004] In a first aspect, the present application provides a list processing method, comprising:

[0005] In response to detecting that a list is stored in the target message partition, obtaining the target list from the target message partition, wherein the list of the same customer is stored in the same message partition;

[0006] Determine the processing status of the target customers belonging to the target list;

[0007] If the processing status of the target customer is unprocessed, the target customer is marked as being processed, and the target list is stored in the target batch;

[0008] When the target batch meets the preset conditions, the names of the names in the target batch are processed and stored concurrently.

[0009] In a second aspect, the present application provides a list processing device, comprising:

[0010] An acquisition module, for acquiring a target list from the target message partition in response to detecting that a list is stored in the target message partition, wherein the list of the same customer is stored in the same message partition;

[0011] A determination module, used to determine the processing status of the target customers belonging to the target list;

[0012] The first processing module is used for marking the target customer as being in a processing state if the processing state of the target customer is an unprocessed state, and storing the target list into a target batch;

[0013] The second processing module is used to concurrently process and store the names in the target batch when the target batch meets the preset conditions.

[0014] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0015] Memory stores computer-executable instructions;

[0016] The processor executes the computer-executable instructions stored in the memory to implement the list processing method as described in the first aspect of the present application.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed, the list processing method described in the first aspect of the present application is implemented.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed, implements the list processing method described in the first aspect of the present application.

[0019] The list processing method, device, equipment, medium and program product provided by the present application obtain the target list from the target message partition in response to monitoring that a list is stored in the target message partition, and the list of the same customer is stored in the same message partition, thereby ensuring that the list of the same customer will only be processed in one application instance; determining the processing status of the target customer to whom the target list belongs; if the processing status of the target customer is unprocessed, marking the target customer as being in processing status, and storing the target list in the target batch; when the target batch meets the preset conditions, concurrently processing and warehousing the lists in the target batch, without introducing additional system overhead, can ensure that in the process of processing the list at the whole system cluster level, there will be no concurrent processing of different lists of the same customer, thereby avoiding the transaction rollback problem caused by the failure of concurrent processing of different lists of the same customer to be stored or the failure of the jump, solving the system performance loss caused by batch data rollback, and effectively improving the efficiency of list storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 A flowchart of the list entry process provided for related technologies;

[0022] Figure 2 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0023] Figure 3 A flowchart of a list processing method provided in an embodiment of the present application;

[0024] Figure 4 A flowchart of a list processing method provided in another embodiment of the present application;

[0025] Figure 5 A flowchart of a list processing method provided in another embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of a list processing device provided in one embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] First, some technical terms involved in this application are explained:

[0031] Telemarketing system, also known as telephone sales application system, is a system used to handle the specific process logic of telephone marketing.

[0032] Customers, information of people who are being marketed via telemarketing in the telemarketing system.

[0033] A list is a marketing opportunity for a customer in a telemarketing system. The list contains marketing data on customer information and activity information.

[0034] List reception means receiving the list pushed by the upstream system.

[0035] List transition: when the received list is entered into the library (i.e. database), if different lists of the same customer already exist in the library, a logical decision will be made as to whether to use the new list to replace the old list, and the ignore or replace will be executed based on the decision result.

[0036] The list contains information about potential or existing customers, is the basis of sales activities, and helps improve sales efficiency and effectiveness. For example, for the lists in the telemarketing system, the telemarketing system can reach high-value lists in real time, and conduct secondary marketing for existing customers. Among them, the telemarketing system needs to receive the list pushed by the upstream system for marketing. Business requirements require that only one list can be marketed to a customer at the same time, so when the received list is entered into the database, it is necessary to determine whether the current customer already has a list in the database. If the current customer already has a list in the database, the list will be transitioned to replace the lower-value list with the higher-value list.

[0037] At present, the list pushed by the upstream system is usually received by concurrent processing of multiple application instances, and the received list is stored in batches in a single processing. Due to the concurrent processing transition logic, the consistency of the list needs to be considered. Figure 1 A flowchart of the list storage process provided for related technologies, such as Figure 1 As shown, a unique index of the customer identifier (IDentity, ID) is established for the database list table to ensure that a customer can only market one list at a time. When the list is dropped from the database or a new jump replaces the list, consistency is guaranteed by blocking the execution of database transactions and retrying the business. Specifically, the upstream system sends a message containing the list to the kafka middleware (a distributed message middleware), and the kafka middleware receives the message and stores the list contained in the message in the message partition. Multiple application instances consume lists from the message partition respectively, and process the lists concurrently in batches through the thread pool. Taking application instance 1 and application instance 2 as examples, the following nine steps are used to concurrently process and store different lists of customer a:

[0038] S101. Assume that application instance 1 consumes customer a's list 1 through thread 1 in the thread pool, stores customer a's list 1 in batch 1, queries the database according to customer a's customer ID, and determines that customer a's list is not stored in the database.

[0039] S102. Store the list 1 of customer a in the database through thread 1.

[0040] S103. Assume that application instance 2 consumes list 2 of customer a through thread 2 in the thread pool, stores list 2 of customer a in batch 2, queries the database according to the customer ID of customer a, and determines that list 1 of customer a is stored in the database. At this time, a transition judgment is required to determine whether list 1 is replaced by list 2.

[0041] S104. Thread 1 uses transaction 1 to delete customer a's list 1 from the database, and stores customer a's list 2 in the database.

[0042] S105. While step S103 is being executed, assume that application instance 3 consumes list 3 of customer a through thread 3 in the thread pool, stores list 3 of customer a in batch 3, queries the database based on the customer ID of customer a, and determines that list 1 of customer a is stored in the database. At this time, a transition judgment is required to determine whether list 1 is replaced by list 3.

[0043] S106. Thread 3 uses transaction 2 to start deleting customer a's list 1 from the database, but is blocked by transaction 1, and the deletion of customer a's list 1 from the database fails.

[0044] S107, rolling back transaction 2 through thread 3, all the lists of batch 3 in transaction 2 must be rolled back, and then executing step S108 to retry.

[0045] S108, query the database according to the customer identification of customer a, and determine that list 2 of customer a is stored in the database. At this time, a transition judgment needs to be made to determine to replace list 2 with list 3.

[0046] S109. Thread 3 uses transaction 3 to delete customer a's list 2 from the database, and stores customer a's list 3 in the database.

[0047] refer to Figure 1 , when different lists of the same customer in a batch (such as batch 3) fail to be stored or migrated concurrently, since the business adopts batch processing, as long as one list in a batch fails to be stored and needs to be rolled back, all lists in the batch in the transaction must be rolled back and retried. Therefore, storing the received list in the above manner increases the system performance overhead and has the problem of low efficiency. In addition, storing the received list in the above manner also has the problem of not being able to process different lists of the same customer in a batch. Specifically, when there are different lists of the same customer in a batch of lists consumed, batch migration is impossible, because the migration requires that the new list and the list in the library must be used. Therefore, when there are different lists of the same customer in a batch, one of the lists must be randomly retained and the rest must be discarded. The discarded list is considered to have failed this processing and will be retrieved and retried in the next business bottom-up retry, which increases the system overhead.

[0048] Based on the above problems, the present application provides a list processing method, which ensures that the list of the same customer will only be processed in one application instance by storing the list of the same customer in the same message partition and obtaining the list from the message partition; for the target list obtained from the target message partition, determine whether to put the target list into storage according to the processing status of the target customer to which the target list belongs, and without introducing additional system overhead, ensure that during the list storage process, there will be no concurrent processing of different lists of the same customer, avoiding the transaction rollback problem caused by the concurrent processing of different lists of the same customer failing to be stored or failing to jump, thereby effectively improving the efficiency of the list storage. In addition, if the processing status of the target customer is in the processing state, the list is stored in the retry queue, and priority is given to trying to pull the list from the retry queue and pack it in batches, that is, retrying, which can solve the problem of different lists of the same customer in the same batch, and avoid the performance overhead of inefficient retries caused by discarding the list.

[0049] In the following, the application scenarios of the solutions provided in this application are first illustrated.

[0050] Figure 2 This is a schematic diagram of an application scenario provided by an embodiment of the present application. Figure 2 As shown, in this application scenario, taking the warehousing processing of the list in the telemarketing system as an example, the upstream system 201 sends a message containing the list to the message middleware 202 (such as the kafka middleware), and the message middleware 202 receives the message, locates the message partition according to the customer identifier carried in the message, and stores the list contained in the message in the message partition, so that the list of the same customer is stored in the same message partition. The telemarketing application instance of the telemarketing system 203 consumes the list from the message partition and stores the list in the batch queue; obtains the list from the batch queue, determines the processing status of the customer to whom the list belongs, stores the list of customers who are not marked as being processed, and stores the list of customers marked as being processed in the retry queue for retry warehousing.

[0051] It should be noted that Figure 2 This is only a schematic diagram of an application scenario provided by the embodiment of the present application. Figure 2 does not limit the equipment included in Figure 2 The positional relationship between the devices is limited.

[0052] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0053] Figure 3This is a flowchart of a list processing method provided in an embodiment of the present application. The method of the embodiment of the present application can be applied to an electronic device, which can be a server or a server cluster. Figure 3 As shown, the method of the embodiment of the present application includes:

[0054] S301. In response to detecting that a list is stored in a target message partition, a target list is obtained from the target message partition. The list of the same customer is stored in the same message partition.

[0055] In an embodiment of the present application, the producer of the list message (i.e., the upstream system) sends a message containing a list to the message middleware; the message middleware receives the message, locates the message partition according to the customer ID carried by the message, and stores the list contained in the message in the message partition, so that the list of the same customer is stored in the same message partition, which can ensure that the list of the same customer will only be processed in one application instance. It can be understood that a message partition can store the lists of multiple different customers, and the list of the same customer is stored in the same message partition. Among them, the message middleware is, for example, a kafka middleware, or any message middleware with partitioning function.

[0056] Exemplarily, the electronic device executing the embodiment of the method is, for example, an electronic device where an electric marketing system is located, and an electric marketing application instance of the electric marketing system consumes a message partition. For example, the message partitions include message partition 1, message partition 2, and message partition 3, then the electric marketing application instance 1 consumes message partition 1, the electric marketing application instance 2 consumes message partition 2, and the electric marketing application instance 3 consumes message partition 3. Specifically, the electric marketing application instance can obtain the target list from the target message partition in response to monitoring that a list is stored in the target message partition through the consumption thread, that is, consume the target list from the target message partition.

[0057] S302: Determine the processing status of the target customers in the target list.

[0058] In this step, after obtaining the target list from the target message partition, the processing status of the target customers to whom the target list belongs can be determined. Exemplarily, the processing status of the target customers to whom the target list belongs can be determined based on whether the customer identifier of the target customer is stored in the memory. If the customer identifier of the target customer is not stored in the memory, it can be determined that the processing status of the target customer is an unprocessed state, and step S303 is continued; if the customer identifier of the target customer is stored in the memory, it can be determined that the processing status of the target customer is a processing state, and the target list is stored in the retry queue. For specific information on how to process the target list into storage through the retry queue, please refer to the subsequent embodiments, which will not be repeated here. This step is, for example, executed by a packaging thread.

[0059] S303: If the processing status of the target customer is unprocessed, the target customer is marked as being processed, and the target list is stored in a target batch.

[0060] In this step, after determining that the processing status of the target customer to whom the target list belongs is unprocessed, the target customer is marked as being processed, and the target list is stored in the target batch, and the next list is obtained from the target message partition.

[0061] Optionally, marking the target customer as being in a processing state may include: storing a customer identifier of the target customer in a memory, so as to determine that the target customer is marked as being in a processing state.

[0062] Exemplarily, after determining that the processing status of the target customer is unprocessed, the customer identification of the target customer can be stored in the memory to determine that the target customer is marked as being in processing status, thereby avoiding the concurrent processing problem of different lists of the same customer when the lists are stored in batches.

[0063] S304: When the target batch meets the preset conditions, the names of the names in the target batch are processed and stored concurrently.

[0064] It can be understood that the lists are processed and stored in batches concurrently. Optionally, when the target batch meets the preset conditions, the lists in the target batch are processed and stored concurrently, which may include: when the second number of lists in the target batch is greater than or equal to the second threshold, the lists in the target batch are processed and stored concurrently; or when the waiting time of the target batch is greater than the second preset time, the lists in the target batch are processed and stored concurrently.

[0065] Exemplarily, the second threshold value can be set as needed, and the second preset time length is, for example, 2 seconds. The embodiment of the present application does not limit the second threshold value and the second preset time length. It can be understood that when there are many lists pushed by the upstream system, when the second number of lists in the target batch is greater than or equal to the second threshold value, the target batch is deemed to be packaged and the lists in the target batch are processed and put into storage concurrently. Alternatively, when there are few lists pushed by the upstream system, when the waiting time of the target batch is greater than 2 seconds, the lists in the target batch are processed and put into storage concurrently.

[0066] In this step, illustratively, after the packaging thread determines that the target customer is not marked as being in the processing state, the target customer is marked as being in the processing state, and the target list is stored in the target batch. When the target batch meets the preset conditions, the list in the target batch is processed and stored in the warehouse through the list batch processing thread pool. When the threads in the working state in the list batch processing thread pool are full, the packaging thread is blocked.

[0067] Optionally, after the lists in the target batch are processed concurrently and stored in the warehouse, the list processing method provided in the embodiment of the present application may further include: updating the processing status of the target customers to whom the target list belongs from a processing status to an unprocessed status.

[0068] It can be understood that after all the names in the target batch are stored in the warehouse, the processing status of the target customers to which the target list belongs can be updated from the processing status to the unprocessed status, that is, the processing status mark of the customers corresponding to all the names in the target batch is cleared. For example, if the customer ID of the customer corresponding to the name in the target batch is stored in the memory, the customer ID in the memory can be cleared to continue the storage processing of the name of the customer.

[0069] The list processing method provided in the embodiment of the present application obtains the target list from the target message partition in response to monitoring that a list is stored in the target message partition, and the list of the same customer is stored in the same message partition, thereby ensuring that the list of the same customer will only be processed in one application instance; determining the processing status of the target customer to whom the target list belongs; if the processing status of the target customer is unprocessed, marking the target customer as being in processing status, and storing the target list in the target batch; when the target batch meets the preset conditions, concurrently processing and warehousing the lists in the target batch, without introducing additional system overhead, can ensure that in the process of processing the list at the whole system cluster level, there will be no concurrent processing of different lists of the same customer, thereby avoiding the transaction rollback problem caused by the failure of concurrent processing of different lists of the same customer to be put into the warehouse or the failure of the jump, solving the system performance loss caused by batch data rollback, and effectively improving the efficiency of warehousing of the list.

[0070] Figure 4 This is a flowchart of a list processing method provided by another embodiment of the present application. Based on the above embodiment, the present application embodiment further illustrates the list processing method. Figure 4 As shown, the method of the embodiment of the present application may include:

[0071] S401. In response to detecting that a list is stored in a target message partition, a target list is obtained from the target message partition. The list of the same customer is stored in the same message partition.

[0072] The detailed description of this step can be found in Figure 3 The relevant description of S301 in the illustrated embodiment will not be repeated here.

[0073] In the embodiment of the present application, Figure 3 The step S302 may further include the following two steps S402 and S403:

[0074] S402: Store the target list into a batch queue.

[0075] For example, after the target list is obtained from the target message partition through the consumer thread, the target list can be stored in the batch queue (Blocking Queue) of the memory, waiting to be packaged in batches. When the batch queue is full, the consumer thread is blocked until a list is dequeued from the batch queue, and the target list is stored in the batch queue to ensure the stability of the system.

[0076] S403: Obtain a target list from the batch queue, and determine whether the processing status of the target customers in the target list is in the processing status.

[0077] For example, the target list can be obtained from the batch queue through the packaging thread to determine whether the processing status of the target customers to which the target list belongs is in the processing state. For details on how to determine whether the processing status of the target customers to which the target list belongs is in the processing state, please refer to Figure 3 The relevant description of S302 in the illustrated embodiment. If the processing status of the target customer is unprocessed, then execute steps S404 to S406; if the processing status of the target customer is in processing, then execute steps S407 to S411. It should be noted that the implementation of this application does not limit the order of execution of steps S404 to S406 and steps S407 to S411.

[0078] S404: Mark the target customers as being in processing status, and store the target list in a target batch.

[0079] The detailed description of this step can be found in Figure 3 The relevant description of S303 in the illustrated embodiment will not be repeated here.

[0080] S405. When the target batch meets the preset conditions, the names of the names in the target batch are processed and stored concurrently.

[0081] Exemplarily, when the target batch meets the preset conditions, the names in the target batch can be processed and stored concurrently through the name list batch processing thread pool. When the threads in the name list batch processing thread pool are full, the packaging thread is blocked to ensure the stability of the system.

[0082] S406: If it is determined that the concurrent processing and storage of the target list in the target batch is completed, the processing status of the target customers to whom the target list belongs is updated from the processing status to the unprocessed status.

[0083] It can be understood that after all the lists in the target batch are stored in the warehouse, the processing status of the target customer to whom the target list belongs can be updated from the processing status to the unprocessed status to continue the storage processing of the customer's list.

[0084] S407: Store the target list in the retry queue.

[0085] On the basis of the above embodiment, optionally, before storing the target list in the retry queue, the list processing method provided in the embodiment of the present application may also include: if a first number of lists to be retried in the retry queue is greater than or equal to a first threshold, determining that the target list is a list of failed consumption; when the waiting time is greater than a first preset time, obtaining the list of failed consumption, and storing the list of failed consumption in a batch queue.

[0086] Exemplarily, the first threshold value can be set as needed, the first preset time length is, for example, 10 minutes, and the waiting time length is the time length from the moment when the target list is determined to be a list of failed consumption; the embodiment of the present application does not limit the first threshold value and the first preset time length. Before storing the target list in the retry queue, it can be determined whether the first number of lists to be retried in the retry queue is greater than or equal to the first threshold value, that is, whether the retry queue is full. If the retry queue is full, the target list is determined to be a list of failed consumption; when the waiting time length is greater than 10 minutes, the list of failed consumption is obtained, and the list of failed consumption is stored in the batch queue to continue the warehousing process. This embodiment can be understood as a bottom-up retry of the list of failed consumption.

[0087] S408: Obtain a list of people to be retried in the retry queue at a preset time interval.

[0088] For example, the preset time interval is, for example, every 10 seconds, and the list of pending retry in the retry queue is obtained every 10 seconds. If the list of pending retry is stored in the retry queue, steps S409 to S411 are executed; if the list of pending retry is not stored in the retry queue, step S403 is executed ( Figure 4 (not shown), that is, first try to obtain the list from the retry queue and put it into the warehouse in batches. When there is no list to be retried stored in the retry queue, obtain the list from the batch queue and put it into the warehouse in batches.

[0089] S409: For each acquired list to be retried, determine whether the processing status of the customer on the list to be retried is in the processing status.

[0090] In this step, how to determine whether the processing status of the customers in the waiting retry list is in the processing status is the same as how to determine the processing status of the target customers in the target list. Figure 3 The relevant description of S302 in the illustrated embodiment: If the processing status of the customer to be retried belongs to is unprocessed, then execute step S410; if the processing status of the customer to be retried belongs to is in processing, then execute step S411.

[0091] S410, mark the customers on the waiting-for-retry list as being in the processing state, store the waiting-for-retry list into the target batch, and execute steps S405 and S406.

[0092] For example, the customer identification of the customer on the to-be-retried list is stored in the memory, that is, the customer on the to-be-retried list is marked as being in processing status, and then the to-be-retried list is stored in the target batch, so that when the target batch meets the preset conditions, the lists in the target batch are processed concurrently and put into storage.

[0093] S411, the list to be retried is stored in the retry queue again, and then step S408 can be executed.

[0094] It can be understood that if the customer to whom the to-be-retried list belongs is marked as being processed, in order to ensure that different lists of the same customer are not processed concurrently, the to-be-retried list can be stored back in the retry queue for retry into the warehouse.

[0095] The list processing method provided in the embodiment of the present application obtains the target list from the target message partition in response to monitoring that a list is stored in the target message partition, and the list of the same customer is stored in the same message partition, which can ensure that the list of the same customer will only be processed in one application instance; the target list is stored in a batch queue, the target list is obtained from the batch queue, and the processing status of the target customer to which the target list belongs is determined. By storing the target list in the batch queue, the stability of the system can be guaranteed; if the processing status of the target customer is unprocessed, the target customer is marked as being in processing, and the target list is stored in the target batch. When the target batch meets the preset conditions, the lists in the target batch are processed and stored in the warehouse concurrently, and after the storage is completed, the processing status of the target customer to which the target list belongs is updated from being in processing to being unprocessed, which can avoid the transaction rollback problem caused by the concurrent processing of different lists of the same customer failing to be stored or failing to jump, and effectively improve the storage efficiency of the list; if the processing status of the target customer is in processing, The target list is stored in the retry queue, and the list to be retried in the retry queue is obtained at a preset time interval. For each list to be retried obtained, it is determined whether the processing status of the customer to whom the list to be retried belongs is in the processing state. If the processing status of the customer to whom the list to be retried belongs is unprocessed, the customer to whom the list to be retried belongs is marked as in the processing state, and the list to be retried is stored in the target batch. When the target batch meets the preset conditions, the lists in the target batch are processed and stored concurrently. If the processing status of the customer to whom the list to be retried belongs is in the processing state, the list to be retried is stored again in the retry queue. Based on the dual synchronous queue collaboration mechanism of the batch queue and the retry queue, the list of failed consumption can be directly retried and stored at the appropriate time without affecting the packaging processing of other lists, which can avoid the extra overhead caused by the list being directly treated as a consumption failure and reprocessed by the bottom-up retry process, that is, it can solve the problem of inefficient retries caused by the inability to transition different lists of the same customer in the same batch.

[0096] In addition, when the batch queue is full, the consumer thread is blocked until a list is dequeued from the batch queue and the target list is stored in the batch queue; when the threads in the list batch processing thread pool are full, the packaging thread is blocked. Due to the use of double blocking collaboration, when the list push rate is slow, the packaging job can pull the list pushed to the batch queue in real time and package it and deliver it to the processing job; when the list push rate is fast and the concurrent processing limit of the processing job is reached, the packaging job will be blocked when delivering the list to the processing job. Therefore, when the main queue is full, the consumer thread of the message middleware will eventually be blocked, ensuring the stability of the system.

[0097] Based on the above embodiments, Figure 5A flowchart of a list processing method provided for another embodiment of the present application. In which, the upstream system sends a message containing a list to the Kafka message middleware, that is, pushes the list; the Kafka message middleware receives the message, and locates the message partition according to the customer ID carried by the message, and stores the list contained in the message in the message partition, so that the list of the same customer is stored in the same message partition. An application instance consumes a message partition, and after consuming the list from the message partition, the list is processed concurrently in batches through the thread pool. Taking application instance 2 as an example, the method of the embodiment of the present application may include:

[0098] S501. Application instance 2 consumes lists from message partition 2 through a consumer thread. Taking the example that the consumer thread consumes customer a's list 2, customer b's list 1, customer a's list 3 and customer a's list 4 from message partition 2 in sequence, the above lists consumed are sequentially stored in a batch queue.

[0099] For each list in the batch queue, execute the following steps S502 to S507.

[0100] S502: Obtain a list from a batch queue through a packaging thread.

[0101] S503: Determine whether the customer to whom the list belongs is marked as being processed in the memory.

[0102] If it is not marked as being in process, then execute step S504; if it is marked as being in process, then execute steps S509 and S510.

[0103] S504: Mark the customers to whom the list belongs as being processed, and store the list in a batch.

[0104] S505. When the batch meets the preset conditions, the batch is submitted to the list batch processing thread pool, so that the list in the batch is processed concurrently and stored in the warehouse through the list batch processing thread pool.

[0105] Assuming that batch 1 contains customer a's list 2 and customer b's list 1, customer a's list 2 and customer b's list 1 can be put into storage through the thread pool. Specifically, taking customer a's list 2 as an example, the storage process of customer a's list 2 may include the following steps:

[0106] S506. Query the database according to the customer ID of customer a through thread 1 to determine that customer a's list 1 is stored in the database. At this time, a transition judgment is required to determine to replace list 1 with customer a's list 2.

[0107] S507. Thread 1 uses transaction 1 to delete customer a's list 1 from the database, and stores customer a's list 2 in the database.

[0108] Based on the above steps S506 and S507, since the database does not store the list of customer b, the list 1 of customer b can be directly stored in the database. It can be understood that for batch 2, since both customer a and customer b are marked as being processed in the memory, the list of customer a or customer b will not appear in batch 2.

[0109] S508. If it is determined that the concurrent processing and storage of the lists in batch 1 are completed, the processing status of the customers to whom each list in batch 1 belongs is updated from the processing status to the unprocessed status.

[0110] S509, store the list in the retry queue and execute step S510.

[0111] S510. Obtain a list of people to be retried in the retry queue at a preset time interval, and execute step S503.

[0112] The list processing method provided in the embodiment of the present application stores the list of the same customer in the same message partition, ensuring that different lists of the same customer will only be processed in one application instance. At the same time, the lists of customers that are not marked as being in processing status are processed and put into storage concurrently in batches, which can avoid the transaction rollback problem caused by the concurrent processing of different lists of the same customer failing to be put into storage or failing to jump, and effectively improve the efficiency of list entry; the list of customers marked as being in processing status is stored in the retry queue. Based on the dual synchronous queue collaboration mechanism of the batch queue and the retry queue, the list of failed consumption can be directly retried and put into storage at an appropriate time without affecting the packaging processing of other lists, which can solve the problem of inefficient retries caused by the inability to jump different lists of the same customer in the same batch.

[0113] Based on the above embodiments, it should be noted that the list processing method, device, equipment, medium and program product provided in the embodiments of the present application can be applied to the telemarketing system, and can also be applied to any system other than the telemarketing system. The application system of the list processing method, device, equipment, medium and program product provided in the present application is not limited.

[0114] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0115] Figure 6 A structural diagram of a list processing device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the list processing device 600 of the embodiment of the present application includes: an acquisition module 601, a determination module 602, a first processing module 603 and a second processing module 604. Among them:

[0116] The acquisition module 601 is used to acquire a target list from the target message partition in response to detecting that a list is stored in the target message partition. The list of the same customer is stored in the same message partition.

[0117] The determination module 602 is used to determine the processing status of the target customers belonging to the target list.

[0118] The first processing module 603 is used to mark the target customer as being in a processing state if the processing state of the target customer is an unprocessed state, and store the target list into a target batch.

[0119] The second processing module 604 is used to concurrently process and store the names in the target batch when the target batch meets the preset conditions.

[0120] In some embodiments, the determination module 602 may be specifically used to: store the target list in a batch queue; obtain the target list from the batch queue, and determine the processing status of the target customers to which the target list belongs.

[0121] Optionally, the first processing module 603 may also be used to: if the processing status of the target customer is in the processing status, store the target list in the retry queue.

[0122] Optionally, the first processing module 603 can also be used to: obtain the list of customers to be retried in the retry queue at preset time intervals; for each list of customers to be retried obtained, determine whether the processing status of the customers on the list of customers to be retried is in the processing status; if the processing status of the customers on the list of customers to be retried is in the unprocessed status, then update the processing status of the customers on the list of customers to be retried to the processing status, and store the list of customers to be retried in the target batch; if the processing status of the customers on the list of customers to be retried is in the processing status, then store the list of customers to be retried back in the retry queue.

[0123] Optionally, the first processing module 603 can also be used for: before storing the target list in the retry queue, if the first number of lists to be retried in the retry queue is greater than or equal to a first threshold, determining that the target list is a list of failed consumption; when the waiting time is greater than a first preset time, obtaining the list of failed consumption, and storing the list of failed consumption in a batch queue.

[0124] Optionally, the second processing module 604 may also be used to update the processing status of the target customers to which the target list belongs from the processing status to the unprocessed status after the lists in the target batch are concurrently processed and stored.

[0125] In some embodiments, the second processing module 604 can be specifically used to: when the second number of lists in the target batch is greater than or equal to a second threshold, concurrently process and store the lists in the target batch; or when the waiting time of the target batch is greater than a second preset time, concurrently process and store the lists in the target batch.

[0126] The device of the embodiment of the present application can be used to execute the solution of the list processing method in any of the above method embodiments. Its implementation principle and technical effect are similar and will not be repeated here.

[0127] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 7 As shown, the electronic device 700 may include: at least one processor 701 and a memory 702 .

[0128] The memory 702 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer-executable instructions.

[0129] The memory 702 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0130] The processor 701 is used to execute the computer-executable instructions stored in the memory 702 to implement the list processing method described in the aforementioned method embodiment. The processor 701 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. Specifically, when implementing the list processing method described in the aforementioned method embodiment, the electronic device may be, for example, an electronic device with processing functions such as a server.

[0131] Optionally, the electronic device 700 may further include a communication interface 703. In a specific implementation, if the communication interface 703, the memory 702 and the processor 701 are implemented independently, the communication interface 703, the memory 702 and the processor 701 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0132] Optionally, in a specific implementation, if the communication interface 703, the memory 702 and the processor 701 are integrated on a chip, the communication interface 703, the memory 702 and the processor 701 can communicate through an internal interface.

[0133] The present application also provides a computer-readable storage medium, in which computer program instructions are stored. When a processor executes the computer program instructions, the above list processing method is implemented.

[0134] The present application also provides a computer program product, including a computer program, which implements the above list processing method when executed by a processor.

[0135] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0136] An exemplary readable storage medium is coupled to the processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in a dedicated integrated circuit. Of course, the processor and the readable storage medium can also exist as discrete components in the list processing device.

[0137] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A list processing method, characterized in that: include: In response to detecting that a list is stored in a target message partition, obtaining a target list from the target message partition, wherein the list of the same customer is stored in the same message partition; Determine the processing status of the target customers belonging to the target list; If the processing status of the target customer is unprocessed, the target customer is marked as being processed, and the target list is stored in a target batch; When the target batch meets the preset conditions, the names in the target batch are processed and stored concurrently.

2. The list processing method according to claim 1, characterized in that: The determining of the processing status of the target customers to which the target list belongs includes: storing the target list into a batch queue; The target list is obtained from the batch queue, and the processing status of the target customers to which the target list belongs is determined.

3. The list processing method according to claim 2, characterized in that: The method further comprises: If the processing status of the target customer is in the processing state, the target list is stored in the retry queue.

4. The list processing method according to claim 3, characterized in that: The method further comprises: Obtain a list of people to be retried in the retry queue at a preset time interval; For each acquired list to be retried, determine whether the processing status of the customer to be retried is in the processing status; If the processing status of the customer to be retried belongs to the list is unprocessed, then the processing status of the customer to be retried belongs to the list is updated to the processing status, and the list to be retried is stored in the target batch; If the processing status of the client to which the to-be-retried list belongs is in the processing status, the to-be-retried list is stored again in the retry queue.

5. The list processing method according to claim 3, characterized in that: Before storing the target list in the retry queue, the method further includes: If the first number of the to-be-retried list in the retry queue is greater than or equal to the first threshold, determining that the target list is a list of failed consumption; When the waiting time is longer than the first preset time, the list of failed consumption is obtained, and the list of failed consumption is stored in the batch queue.

6. The list processing method according to any one of claims 1 to 5, characterized in that: After the lists in the target batch are processed and stored concurrently, the method further includes: The processing status of the target customers in the target list is updated from a processing status to an unprocessed status.

7. The list processing method according to any one of claims 1 to 5, characterized in that: When the target batch meets the preset conditions, the names in the target batch are processed and stored concurrently, including: When the second number of the names in the target batch is greater than or equal to the second threshold, the names in the target batch are processed concurrently and stored; or, When the waiting time of the target batch is longer than the second preset time, the names of the target batch are processed concurrently and stored in the warehouse.

8. A list processing device, characterized in that: include: An acquisition module, configured to acquire a target list from the target message partition in response to detecting that a list is stored in the target message partition, wherein the list of the same customer is stored in the same message partition; A determination module, used to determine the processing status of the target customers belonging to the target list; A first processing module, for marking the target customer as being in a processing state if the processing state of the target customer is an unprocessed state, and storing the target list into a target batch; The second processing module is used to concurrently process and store the names in the target batch when the target batch meets a preset condition.

9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the list processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed, the list processing method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed, the list processing method according to any one of claims 1 to 7 is implemented.

Images