Task processing method, apparatus, device, and storage medium

Abstract

The present disclosure provides a task processing method and device, equipment and storage medium, relates to the technical field of computers, in particular to the fields of big data, cloud computing and artificial intelligence. The specific implementation scheme is: a task processing method, comprising: obtaining a to-be-processed task from a batch task; and performing an operation corresponding to a stage of the batch task on the to-be-processed task according to a state of the batch task. The embodiments of the present disclosure are beneficial to performing corresponding operations for different stages, and can more flexibly process tasks.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of big data, cloud computing and artificial intelligence. Background Technology

[0002] When importing data in batches into some systems or platforms, such as importing base images into the face database in a system or platform that uses a face recognition model, multiple registration photos (i.e., base images) can be registered into the face database at once by uploading files. Typically, batch data import can be done serially or in parallel. Summary of the Invention

[0003] This disclosure provides a task processing method, apparatus, device, and storage medium.

[0004] According to one aspect of this disclosure, a task processing method is provided, comprising:

[0005] Retrieve tasks to be processed from batch tasks;

[0006] Based on the status of the batch tasks, perform the operations corresponding to the stage of the batch tasks on the pending tasks.

[0007] According to another aspect of this disclosure, a task processing apparatus is provided, comprising:

[0008] The acquisition module is used to acquire tasks to be processed from batch tasks;

[0009] The processing module is used to perform the corresponding operations for the stage of the batch tasks based on the status of the batch tasks.

[0010] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0011] At least one processor; and

[0012] The memory is communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described above.

[0014] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the methods described above.

[0015] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method.

[0016] In this embodiment of the disclosure, the operation corresponding to the stage of the batch task is performed on the unprocessed task in the batch task according to the status of the batch task. This is beneficial for performing corresponding operations for different stages and can handle tasks more flexibly.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0018] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0019] Figure 1 This is a schematic flowchart of a task processing method according to an embodiment of the present disclosure;

[0020] Figure 2 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure;

[0021] Figure 3 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure;

[0022] Figure 4 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure;

[0023] Figure 5 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure;

[0024] Figure 6 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure;

[0025] Figure 7 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure;

[0026] Figure 8 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure;

[0027] Figure 9 This is a schematic block diagram of a task processing apparatus according to an embodiment of the present disclosure;

[0028] Figure 10 This is a schematic block diagram of a task processing apparatus according to another embodiment of the present disclosure;

[0029] Figure 11 This is a schematic block diagram of a task processing apparatus according to another embodiment of the present disclosure;

[0030] Figure 12 This is a schematic block diagram of a task processing apparatus according to another embodiment of the present disclosure;

[0031] Figure 13 This is a schematic diagram of the system structure of a two-stage parallel import scheme according to an embodiment of the present disclosure;

[0032] Figure 14 This is a flowchart illustrating an example of a user uploading a file;

[0033] Figure 15 This is a flowchart illustrating an example of the worker thread functionality;

[0034] Figure 16 This is a flowchart illustrating another example of the worker thread functionality;

[0035] Figure 17 A schematic block diagram of an example electronic device used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0036] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0037] Figure 1 This is a flowchart illustrating a task processing method according to an embodiment of the present disclosure. The method may include:

[0038] S101. Obtain tasks to be processed from batch tasks;

[0039] S102. Based on the status of the batch tasks, perform the operation corresponding to the stage of the batch tasks on the task to be processed.

[0040] In this embodiment, a batch task may include multiple pending tasks, and one pending task can be retrieved from the batch task at a time. For example, a task receiving thread (also known as a service thread) can add a portion of the pending tasks from the batch task to a task queue, the number of pending tasks added to the task queue being determined by the length of the task queue. Worker threads in a thread pool can sequentially read pending tasks from the task queue. The thread pool may include multiple worker threads; running multiple worker threads can support concurrent task processing, improving the processing speed of batch tasks.

[0041] In this embodiment, the status of batch tasks can be recorded in various ways. For example, a distributed cache such as Redis (Redis Remote Dictionary Server) can be used to record the status of batch tasks. After a worker thread reads the tasks to be processed from the task queue, it can look up the records in Redis to obtain the status of the batch tasks. Then, based on the status of the batch tasks, the worker thread executes the operations corresponding to the stage of the batch tasks it is responsible for.

[0042] In this embodiment, the state of a batch task can include various factors and may change depending on the processing progress. The state of a batch task may be related to its stage. For example, the state of a batch task can change from "importing" to "ready to submit". The stages of a batch task can include a first stage and a second stage, where the first stage can be an import stage and the second stage can be a submission stage. After all pending tasks included in the batch task have been imported, the state of the batch task can change from "importing" to "ready to submit", and correspondingly, the stage of the batch task can also change from the import stage to the submission stage. During the import stage, if the user needs to undo, a rollback operation can be supported to clear the imported data.

[0043] The task processing method in this embodiment performs the operation corresponding to the stage of the batch task on the unprocessed task in the batch task according to the status of the batch task. This is beneficial for performing corresponding operations for different stages, and can process tasks more reasonably and flexibly.

[0044] Figure 2 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure. The business processing method of this embodiment may include one or more features of the above-described method embodiments. In one implementation, based on the state of the batch tasks, the operation corresponding to the stage of the batch tasks is performed on the task to be processed, including: when the batch tasks are in a first state, performing at least one of the following operations:

[0045] S201. Perform the first phase operation on the task to be processed;

[0046] S202. Upon first viewing, perform the first-stage operation on the task to be processed;

[0047] S203. Upon a second inspection, terminate the worker thread;

[0048] The first state indicates that the batch task is in the import process; the first stage operation includes importing the data of the task to be processed as intermediate data.

[0049] In this embodiment, the first state can be represented by information such as "importing", "importing in progress", "importing in progress", "1", "A", etc., without limiting the specific representation format. The first stage operation is the intermediate state data of the task obtained after the import operation, which can represent valid data that has been written into the system but is not yet displayed to the user. The system to which the data is written can include various types of databases, such as a face database used to support face recognition models.

[0050] For example, if a worker thread checks the status of a batch task and finds that the batch task is in the first state, i.e., during the import process, it can perform the first stage operation on the worker thread's pending tasks, that is, import the data of the worker thread's pending tasks into intermediate state data.

[0051] For example, worker threads can check the status of batch tasks multiple times. If a worker thread finds that a batch task is in the import process for the first time, it can import the data of the tasks to be processed by that worker thread as intermediate data.

[0052] For example, if a worker thread checks a second time and finds that the batch tasks are in the import process, it means that the worker thread has already performed the import operation on the tasks it is responsible for and does not need to perform any commit or undo operations; the worker thread can then terminate. Furthermore, if a worker thread checks a second time and finds that all tasks in the batch have been imported successfully, it can change the status of the batch tasks. For example, it can change the status from "importing" to "pending commit".

[0053] By indicating that the batch task is in the first state of the import process, the data of the task to be processed can be imported as intermediate data during the normal import process. The intermediate data is not the final data, which is beneficial to support other flexible requirements such as reversal.

[0054] Figure 3 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure. The business processing method of this embodiment may include one or more features of the above-described method embodiments. In one implementation, based on the state of the batch tasks, the operation corresponding to the stage of the batch tasks is performed on the tasks to be processed, including:

[0055] S301. If the batch task is in the second state, perform the second phase operation on the task to be processed.

[0056] The second state indicates that the batch task has completed all the first-stage operations; the second-stage operations include submitting the intermediate state data of the task to be processed that was successfully imported into the batch task as the completed state data.

[0057] In this embodiment of the disclosure, the second state can be represented by information such as "pending submission", "submitting", "submitting", "2", "B", etc., without being limited to a specific representation form.

[0058] In this embodiment, the first-stage operation may succeed or fail. For example, if a batch task includes 200 import tasks, and a worker thread performs an import operation on each task once, for a total of 200 import operations, then it can be considered that all the pending tasks in the batch task have completed the first-stage operation. Alternatively, fault-tolerance mechanisms can be set up so that if some tasks fail to import on the first attempt, they can be retried one or more times until the import is determined to be successful or failed before recording the completion of the import operation. Then, based on the statistical results, it can be determined whether all the pending tasks in the batch task have completed the first-stage operation.

[0059] For example, if a worker thread checks the status of a batch task for the first and / or second time and finds that the batch task is in a second state, indicating that the first stage of operations has been fully executed, it can change the intermediate state data of the successfully imported task to the completed state (or final state) data. After submission, the worker thread can change the status of the batch task to indicate that the submission is complete.

[0060] By using the first and second states of batch tasks, data can be imported in stages. Combining intermediate and completed state data helps support other flexible needs, such as reversal.

[0061] Figure 4 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure. The business processing method of this embodiment may include one or more features of the above-described method embodiments. In one implementation, based on the state of the batch tasks, performing the operation corresponding to the stage of the batch tasks on the task to be processed further includes:

[0062] S401. If the batch task is in the third state, perform a rollback operation, where the third state indicates that the batch task received a cancellation request during the import process.

[0063] In this embodiment of the disclosure, the third state can be represented by information such as "Undoing", "Undoing", "Rolling Back", "Undoing", "3", "C", etc., without limiting the specific representation form.

[0064] For example, if a worker thread checks the status of a batch task for the first and / or second time and finds that the batch task is in the third state, indicating that it has received a cancellation request during the import process, it can stop importing the tasks it is responsible for and instead perform a rollback operation, clearing the intermediate data that has already been imported. Performing a rollback operation in response to a cancellation request can clear the imported data, avoid leaving data from batch tasks that were not successfully imported, and save storage space.

[0065] In one implementation, the rollback operation includes: sending a notification indicating that the import has been revoked and clearing the imported intermediate data.

[0066] For example, before clearing the imported intermediate data, a notification of cancellation can be sent to the user's device, for instance, via a service thread. Since intermediate data is invisible to the user, notifying the user of cancellation in advance and performing asynchronous clearing helps reduce interaction waiting time and improves the user experience.

[0067] Figure 5 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure. The business processing method of this embodiment may include one or more features of the above-described method embodiments. In one implementation, based on the state of the batch tasks, performing the operation corresponding to the stage of the batch tasks on the task to be processed further includes:

[0068] S501. If the batch task is in the fourth or fifth state, terminate the worker thread, where the fourth state indicates that the batch task has been submitted; and the fifth state indicates that the batch task has been terminated due to cancellation.

[0069] In this embodiment, the fourth state can be represented by information such as "Completed," "Submission Completed," "4," or "D," without limitation on the specific representation. The fifth state can be represented by information such as "Terminated," "Cancellation Completed," "5," or "E," without limitation on the specific representation. The fourth and fifth states can be represented by different information or by the same information.

[0070] For example, if a worker thread checks the status of a batch task for the first and / or second time and finds that the batch task is in a fourth state indicating that the batch task has been completed or a fifth state indicating that the batch task has been terminated, the worker thread can terminate. Controlling the termination of worker threads by indicating whether the batch task is completed or terminated allows for timely release of worker threads, facilitating the rapid execution of other tasks and improving execution efficiency.

[0071] Figure 6This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure. The business processing method of this embodiment may include one or more features of the method embodiments described above. In one implementation, the method further includes:

[0072] S601. Perform status maintenance on this batch of tasks.

[0073] In one implementation, S601 maintaining the status of the batch tasks specifically includes: updating the first number of successfully imported tasks and / or the second number of failed imported tasks when the first phase operation is performed; and updating the first status of the batch tasks to the second status when it is determined, based on the first number and / or the second number, that the number of processed tasks has reached the total number of the batch tasks. The first status indicates that the batch tasks are in the import process, and the second status indicates that all of the batch tasks have completed the first phase operation.

[0074] For example, the first number that was successfully imported and / or the second number that failed to import can each be stored in Redis using a separate key in a character data format. When a worker thread increments the count, it uses an atomic increment command (incr) to add the keys for the first and / or second numbers. This helps ensure the accuracy of the values ​​under high concurrency.

[0075] In one implementation, S601 specifically includes maintaining the status of the batch task: when performing the second phase operation, updating the second status of the batch task to a fourth status, which indicates that the batch task has been submitted.

[0076] In one implementation, S601 performing state maintenance on the batch task specifically includes: in the case of performing a rollback operation, updating the third state of the batch task to the fifth state, wherein the third state is used to indicate that the batch task received a cancellation request during the import process, and the fifth state is used to indicate that the batch task was terminated due to cancellation.

[0077] The methods described above for maintaining the status of batch tasks can be one or more depending on the different states of the batch tasks, and the specific method can be flexibly selected according to actual needs. By maintaining the status of batch tasks, the current state of the batch tasks can be determined, thereby enabling timely and accurate corresponding operations and improving task processing speed.

[0078] In one implementation, different states of the batch task have different switch identifiers.

[0079] For example, batch tasks have multiple different states, such as the first state, second state, third state, fourth state, and fifth state in the above embodiments. Different states can have different switch flags. For example, the switch flag for the first state is key1, the switch flag for the second state is key2, the switch flag for the third state is key3, the switch flag for the fifth state is key4, and the switch flag for the fifth state is key5. Each switch flag can represent on or off with different values. For example, a switch flag value of 1 indicates on, and 0 indicates off. Alternatively, a switch flag value of 0 indicates on, and 1 indicates off. Switch flags can also be called switches, switch flags, or switch quantities, etc. The validity of different states of the batch task can be determined through different switch flags.

[0080] In one implementation, the value of each switch identifier can be modified from a first value to a second value. For example, the first value is 0 and the second value is 1. Alternatively, the first value is 1 and the second value is 0. The first and second values ​​can also be other values, as long as the first and second values ​​are different.

[0081] Each of these switch identifiers can default to the first value. If the switch identifier corresponding to a certain state meets the conditions for change, the worker thread can modify the switch identifier for that state from the first value to the second value. For example, if the state of a batch task is "Undoing," the worker thread can modify the switch identifier for the "Undoing" state from the first value to the second value.

[0082] In one implementation, the value of each of these switch identifiers cannot be modified in reverse.

[0083] By setting a one-way modification of the switch flag value, it is beneficial to ensure the accuracy of the switch status, which is especially suitable for frequent operations on the switch flag under high concurrency conditions.

[0084] In one implementation, the batch task's states have priorities. When multiple different states correspond to the same switch flag value, the valid state is determined according to priority. For example, the priorities from highest to lowest are: fourth state, fifth state, third state, second state, and first state. If key3=1 for the third state and key=1 for the second state, then the current valid state can be determined as the third state, i.e., it is in the process of being cancelled. By setting the priorities of different states of the batch task, it is beneficial to accurately determine the valid state of the batch task when multiple different states correspond to the same switch flag value.

[0085] The states of the batch tasks in this embodiment are merely examples and not limitations. In practical applications, they can be flexibly increased or decreased according to different needs of the batch tasks. For example, start state, stop state, stop state, etc. can also be set according to requirements.

[0086] Figure 7 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure. The business processing method of this embodiment may include one or more features of the above-described method embodiments. In one implementation, step S101 involves obtaining tasks to be processed from a batch of tasks, including:

[0087] S1011. Using the files and images that need to be imported, generate each of the batch tasks to be processed.

[0088] S1012. Add multiple pending tasks to the task queue;

[0089] S1013. Use multiple worker threads in the thread pool to read the task to be processed from the task queue.

[0090] For example, the file to be imported can be an Excel spreadsheet. The Excel spreadsheet can contain various information that needs to be imported. The image to be imported and the file to be imported can be related. For example, the file to be imported might have a column that includes the name or other identifier of the image to be imported.

[0091] In one implementation, the method further includes: uploading the image to be imported from the task to be processed; and receiving the storage path of the uploaded image to be imported from the task to be processed. For example, the storage path can be a URL (Uniform Resource Locator) link address. The steps of uploading the image and parsing the file to be processed to obtain personnel information can be executed asynchronously without time constraints, which is beneficial for rationally arranging parallel processing content to speed up task processing progress.

[0092] For example, a service thread parses the file to be imported to obtain the personnel information and image identifiers, uploads the images to a web server (e.g., a cloud server), and receives the image identifiers and their corresponding storage paths from the cloud server. The service thread then generates pending tasks based on the personnel information, image identifiers, and storage paths. If there are N pieces of personnel information to be imported, N pending tasks can be generated. Depending on the size of the task queue, some pending tasks can be added to the queue. If the task queue size is S, and S is greater than or equal to N, all S pending tasks can be added to the queue. If the task queue size is M, and M is less than N, M of the N pending tasks can be added to the queue, and if a worker thread takes a pending task, a new pending task can be added to the queue. If there are T worker threads in the thread pool, each thread can read T pending tasks from the task queue and process them concurrently.

[0093] By using multiple worker threads in a task queue and thread pool, tasks can be processed concurrently, improving task processing speed.

[0094] In one implementation, each task to be processed includes at least one of the following: personnel information to be imported, image identifiers to be imported, and storage paths for the images to be imported. Including personnel information, image identifiers, and image storage paths in the task to be processed reduces the amount of information included in the task, facilitates its inclusion in the task queue for processing, and improves processing speed.

[0095] In one implementation, the personnel information and image identifier are parsed from the file to be imported; the storage path of the image to be imported includes the Uniform Resource Locator (URL) received after uploading the image. Using the URL after image upload facilitates asynchronous image processing, accelerating the overall data import speed.

[0096] In one implementation, the method further includes:

[0097] S701. Download the image to be imported according to the storage path of the image to be imported in the task to be processed;

[0098] S702. Associate the personnel information to be imported with the images to be imported.

[0099] In this embodiment, S701 and S702 can be considered as part of the first-stage operation or as preparatory steps before the first-stage operation, and can be flexibly set according to the actual application requirements. Downloading the image to be imported via the image's storage path allows for the download of already uploaded images, thereby enabling the related import of more complete data.

[0100] Figure 8 This is a flowchart illustrating a task processing method according to another embodiment of the present disclosure. The business processing method of this embodiment may include one or more features of the method embodiments described above. In one implementation, the method further includes:

[0101] S801. Detect the input information of the task to be processed and determine the error type;

[0102] S802. Based on the error type, perform the corresponding processing on the input information.

[0103] In this embodiment, S801 and S802 can be considered as part of the first-stage operation or as preparatory steps before the first-stage operation, and can be flexibly set according to the actual application requirements. After error detection of the input information of the task to be processed, if there are no problems, normal import can be performed. If errors are detected, they can be flexibly processed according to the error type, thereby improving the accuracy of the imported data.

[0104] In one implementation, the corresponding processing of the input information based on the error type includes at least one of the following:

[0105] If both the error type and the input information are Type I errors, record it as an import failure.

[0106] If the error type is Type I and the input information is Type II, the error information of the task to be processed is cleared and submitted to the thread pool for retry.

[0107] If the error type is Type 2, the task to be processed is submitted to the thread pool for retry.

[0108] For example, the first type of error can include errors caused by invalid input information, such as substandard avatar quality or fields not conforming to rules. The second type of error can include internal server-side errors, such as network fluctuations or service overload. For the first type of error, depending on business needs, if necessary information is invalid (such as an avatar), the worker thread can determine that the import has failed. If it is non-necessary information, the worker thread can replace the invalid information with a null value and then return it to the task queue for retry. The second type of error is often sporadic and can also be returned to the task queue for retry. After a task that needs to be retried is returned to the task queue, the worker thread that executes the retried task again can be the same as the previous worker thread or a different one. Retrying or logging according to different error types helps to reasonably correct errors and process tasks more accurately and quickly.

[0109] Figure 9 This is a schematic block diagram of a task processing apparatus according to an embodiment of the present disclosure. The apparatus may include:

[0110] The acquisition module 901 is used to acquire tasks to be processed from batch tasks;

[0111] The processing module 902 is used to perform the operation corresponding to the stage of the batch task on the task to be processed according to the status of the batch task.

[0112] In the task processing device of this disclosure embodiment, the operation corresponding to the stage of the batch task is performed on the unprocessed task in the batch task according to the status of the batch task. This is beneficial for performing corresponding operations for different stages, and can process tasks more reasonably and flexibly.

[0113] Figure 10 This is a schematic block diagram of a task processing apparatus according to another embodiment of the present disclosure. The service processing apparatus of this embodiment may include one or more features of the apparatus embodiments described above. In one embodiment, the processing module 902 includes:

[0114] The first processing submodule 9021 is specifically used to perform at least one of the following operations when the batch tasks are in the first state:

[0115] Perform the first phase of operations on the pending task;

[0116] Upon initial inspection, perform the first phase of operations on the task to be processed.

[0117] Upon a second inspection, the worker thread will be terminated.

[0118] The first state indicates that the batch task is in the import process; the first stage operation includes importing the data of the task to be processed as intermediate data.

[0119] By indicating that the batch task is in the first state of the import process, the data of the task to be processed can be imported as intermediate data during the normal import process. The intermediate data is not the final data, which is beneficial to support other flexible requirements such as reversal.

[0120] In one embodiment, the processing module 902 includes:

[0121] The second processing submodule 9022 is specifically used to perform a second-stage operation on the task to be processed when the batch task is in a second state. The second state indicates that the batch task has completed all the first-stage operations. The second-stage operation includes submitting the intermediate data of the task to be processed that was successfully imported into the batch task as completed data. By using the first and second states of the batch task, data is imported into the task in stages. Combining the intermediate and completed data facilitates support for other flexible requirements such as reversal.

[0122] In one embodiment, the processing module 902 includes:

[0123] The third processing submodule 9023 is specifically used to perform a rollback operation when the batch task is in the third state, where the third state indicates that the batch task has received a cancellation request during the import process. Performing a rollback operation in response to a cancellation request can clear the imported data, avoid leaving data from batch tasks that were not successfully imported, and save storage space.

[0124] In one implementation, the rollback operation includes: sending a notification indicating that the import has been revoked, and clearing the imported intermediate data. Notifying the user in advance of the revocation and performing the clearing asynchronously helps reduce interaction waiting time and improves the user experience.

[0125] In one embodiment, the processing module 902 includes:

[0126] The fourth processing submodule 9024 is specifically used to terminate the worker thread when the batch task is in the fourth or fifth state. The fourth state indicates that the batch task has been submitted; the fifth state indicates that the batch task has been terminated due to cancellation. By controlling the termination of the worker thread by indicating whether the batch task has been completed or terminated, the worker thread can be released in a timely manner to facilitate the rapid execution of other tasks and improve execution efficiency.

[0127] In one embodiment, the device further includes a state maintenance module 1001, used to maintain the state of the batch tasks, specifically used to perform at least one of the following:

[0128] When performing the first phase operation, update the first number of successfully imported and / or the second number of failed imported; when it is determined from the first number and / or the second number that the number of processed tasks has reached the total number of batch tasks, update the first state of the batch tasks to the second state, wherein the first state is used to indicate that the batch tasks are in the import process, and the second state is used to indicate that the batch tasks have all performed the first phase operation.

[0129] In the case of performing the second phase operation, the second state of the batch task is updated to the fourth state, which indicates that the batch task has been submitted.

[0130] In the event of a rollback operation, the third state of the batch task is updated to the fifth state, where the third state indicates that the batch task received a cancellation request during the import process, and the fifth state indicates that the batch task was terminated due to cancellation.

[0131] By maintaining the status of batch tasks, the current status of the batch tasks can be determined, thereby enabling timely and accurate corresponding operations and improving task processing speed.

[0132] In one implementation, the acquisition module 901 is specifically used to: generate each of the pending tasks for the batch task using the files and images to be imported; add multiple pending tasks to a task queue; and read the pending tasks from the task queue using multiple worker threads in a thread pool. By using the task queue and multiple worker threads in the thread pool, tasks can be processed concurrently, improving task processing speed.

[0133] In one implementation, each task to be processed includes at least one of the following: personnel information to be imported, image identifiers to be imported, and storage paths for the images to be imported. Including personnel information, image identifiers, and image storage paths in the task to be processed reduces the amount of information included in the task, facilitates its inclusion in the task queue for processing, and improves processing speed.

[0134] In one implementation, the personnel information and image identifier are parsed from the file to be imported; the storage path of the image to be imported includes the Uniform Resource Locator (URL) received after uploading the image. Using the URL after image upload facilitates asynchronous image processing, accelerating the overall data import speed.

[0135] Figure 11 This is a schematic block diagram of a task processing apparatus according to another embodiment of the present disclosure. The task processing apparatus of this embodiment may include one or more features of the apparatus embodiments described above. In one embodiment, the apparatus further includes:

[0136] Download module 1101 is used to download the image to be imported according to the storage path of the image to be imported in the task to be processed;

[0137] The association module 1102 is used to associate the personnel information to be imported with the images to be imported.

[0138] Download the image you need to import by specifying its storage path. You can also download images that have already been uploaded, thus importing more complete data in a related manner.

[0139] In one implementation, different states of the batch task have different switch identifiers. The validity of different states of the batch task can be determined by using different switch identifiers.

[0140] In one implementation, the value of each of the switch identifiers can be modified from a first value to a second value.

[0141] In one implementation, the value of each of these switch identifiers cannot be modified in reverse.

[0142] By setting a one-way modification of the switch flag value, it is beneficial to ensure the accuracy of the switch status, which is especially suitable for frequent operations on the switch flag under high concurrency conditions.

[0143] In one implementation, the states of the batch tasks have priorities. If multiple different states correspond to the same switch flag value, the valid state is determined according to the priority. By setting different priorities for the different states of the batch tasks, it is beneficial to accurately determine the valid state of the batch tasks even when multiple different states correspond to the same switch flag value.

[0144] Figure 12 This is a schematic block diagram of a task processing apparatus according to another embodiment of the present disclosure. The task processing apparatus of this embodiment may include one or more features of the apparatus embodiments described above. In one embodiment, the apparatus further includes:

[0145] The detection module 1201 is used to detect the input information of the task to be processed and determine the error type;

[0146] Error handling module 1202 is used to perform corresponding processing on the input information according to the error type.

[0147] In this embodiment, after the detection module performs error detection on the input information of the task to be processed, if there are no problems, normal import can be performed. If an error is detected, the error handling module can flexibly handle it according to the error type, thereby improving the accuracy of the imported data.

[0148] In one implementation, the error handling module 1202 is further configured to perform at least one of the following:

[0149] If both the error type and the input information are Type I errors, record it as an import failure.

[0150] If the error type is Type I and the input information is Type II, the error information of the task to be processed is cleared and submitted to the thread pool for retry.

[0151] If the error type is Type 2, the task to be processed should be submitted to the thread pool for retry. Retrying or logging according to different error types helps to correct errors appropriately and process tasks more accurately and quickly.

[0152] The specific functions and examples of each module of the business processing apparatus disclosed herein can be found in the relevant descriptions of the corresponding steps in the above-described business processing method embodiments, and will not be repeated here.

[0153] In a specific example, on a system or platform that applies a facial recognition model, multiple base images, such as registration photos of individuals, are uploaded to the facial database at once via file upload. Current facial recognition technology is widely used and may require importing large amounts of information into the database at once and then distributing it from the cloud to the necessary devices, or directly providing online facial recognition capabilities. Importing into the facial database requires ensuring both the volume and speed of the imported data, as well as guaranteeing "dual-write consistency" and "facial uniqueness."

[0154] "Dual writing" can include writing the person's registration photo into the face model's face database, and also writing the person's information and the photo's storage address into the database. "Dual-write consistency" ensures that the person information written to the database and the face database remains consistent. "Face uniqueness" ensures that there is a one-to-one correspondence between the face imported into the face database and the person's information. The same person's image cannot be registered as two different people in the face database. Furthermore, by providing a UI (User Interface), users can see the import progress in real time and can cancel the process at any time.

[0155] If a serial import method is used, the face database is searched before importing each image to ensure that the same person is not successfully registered multiple times. This method is suitable for importing face databases from device-side management systems or private face recognition systems, as it helps ensure consistency. However, the number of ongoing batch import tasks needs to be limited, as multiple batch import tasks can cause congestion. Therefore, it is not suitable for cloud platform scenarios with high concurrency and large data volumes.

[0156] Parallel import involves splitting the images in the file directory into many individual image registration tasks and executing them in parallel. This leverages the high performance of multi-core CPUs, improving the speed of batch import. Parallel import has advantages in both throughput and import speed. However, because a batch task is broken down into many atomic subtasks, if the user chooses to cancel during the import process, it's necessary to consider how to roll back the submitted subtasks and revert the imported personnel database.

[0157] This disclosure provides a two-stage parallel import scheme. Based on parallel import, it employs a two-phase commit approach, transforming the entire batch import task (which can be referred to as a batch task) into a series of atomic transactions. These transactions do not necessarily represent actual database transactions, but rather can be understood as the entire batch import task possessing atomicity and consistency. This disclosure supports overall rollback and fully leverages the performance of multi-core CPUs through parallel import, offering significant advantages in throughput and import speed. For example, this two-stage parallel import scheme can be used to quickly import large volumes of face databases in cloud scenarios.

[0158] For example, this two-phase parallel import scheme divides the entire import process into two phases: Import and Submit. Figure 13 As shown, users can choose to import or undo operations through the UI. The task receiving thread (also known as the service thread) interacts with the UI and can respond to user imports by adding a portion of the batch task to the task queue. During the import phase, multiple worker threads from a worker thread pool register individual images in parallel, and the imported data is in an intermediate state at this stage. Only after all images have been imported does the submission phase begin. At this point, the system can transform all intermediate data into the final state at once, and the entire batch import task is truly completed. During the import and submission process, tasks, the number of successful imports, and the number of failed imports can be recorded using methods such as Redis.

[0159] like Figure 14 As shown in this example, the process of a user uploading a file may include the following steps:

[0160] S1401. Users can upload compressed files for batch import of face databases through the UI interface, which can be achieved by uploading compressed files in chunks. The compressed file can contain images such as personnel registration photos and personnel information, such as information saved in an Excel file.

[0161] S1402. The server (or the server-side component) creates a batch task to decompress the compressed package and obtain the images that need to be uploaded.

[0162] S1403. Asynchronously upload an image. After a successful upload, you will receive the image's storage path.

[0163] S1404. Scan the decompressed file directory and parse the files to be processed, such as Excel files. A column in the Excel file can store the identifier of the image associated with each person's information, such as the image name.

[0164] S1405. Redis generates batch task records, which can include the status of each batch task and the number of successful and / or failed imports.

[0165] S1406. Batch submit pending tasks, such as avatar registration tasks, to the worker thread pool. The delegated processing tasks can include personnel information, image identifiers, and image storage paths.

[0166] S1407. Return the task identifier (ID).

[0167] If the user performs an undo operation through the UI, the status of the batch task will be changed to "Undoing".

[0168] like Figure 15As shown, an exemplary flow of a worker thread includes:

[0169] After the asynchronous task begins (S1501), the worker thread checks the task status (S1502). For example, the worker thread can view the task status in a Redis record. The task status can include various states, such as Start (represented by 0), Importing (represented by 1, also known as importing), Pending Commit (represented by 2), Completed (represented by 3), Cancelled (represented by -1, or canceling), and Terminated (represented by -2). If the worker thread sees the batch task status as "Importing," it can execute a transaction, record intermediate member records (S1503), and then perform the insertion operation (S1504). Then, update the task progress (S1505). If all has been imported, change the batch task status in Redis to "Pending Commit." If the worker thread can view the batch task status in Redis and finds that the batch task status is "Pending Commit," it means that the first stage (Import) has been completed, and asynchronous commit can be performed (S1506). Change the member records to the final state (S1507) and send a change message (S1508). Furthermore, the status of the batch task can be updated to "Completed". If the user cancels the import, the status of the batch task in Redis can be updated to "Cancelled", indicating that the user has performed a cancellation operation through the UI interface. It can be asynchronously rolled back (S1509), the intermediate records that have been entered into the database are deleted (S1510), and the status is updated to "Terminated".

[0170] like Figure 16 As shown, another exemplary flow of a worker thread may include:

[0171] S1601. The worker thread first checks the latest status of the batch task and triggers different actions based on the status value: if it is "Importing (Importing in progress)", the import task can be executed. If the current task status is "Completed" or "Terminated", it means that this batch task has ended (normal completion or cancellation termination), and it can directly proceed to S1607 to end.

[0172] S1602. The worker thread performs format verification and face quality checks on the image. Images that fail the verification are counted as failure samples.

[0173] S1603. The worker thread writes personnel information into the database, and then uses the personnel identifier (ID) generated by the database to register the image, such as the headshot, into the face database (which can be referred to as the face database).

[0174] S1604. The worker thread updates the batch task information, including the number of successful tasks. For example, increment the number of successful tasks by 1.

[0175] S1605. The worker thread checks the latest status of the batch tasks again and triggers different steps based on different status values:

[0176] (1) If the previous task status is "pending submission", it means that the first stage (Import) has been completed. Then, perform the "submit" operation and update the status to "completed". The completion status of the first stage can be determined based on the number of people. If the number of successful imports in Redis + the number of failed imports = the total number, it means that the first stage has been completed.

[0177] (2) If the previous task status is "Undoing", it means that the user has performed an undo operation through the UI interface. Then, perform a "rollback" operation to clear the data that has been written and update the status to "Terminated".

[0178] (3) If the previous task status is "Importing", "Completed" or "Terminated", proceed directly to step S1607. For example, if there are a total of 64 worker threads, the first 63 threads will end and the 64th thread will be submitted.

[0179] S1606. If an error occurs in any step of a worker thread, it can be resubmitted to the thread pool for a retry.

[0180] S1607. The worker thread has finished executing.

[0181] In this embodiment of the disclosure, when a user cancels, the imported data can be rolled back. For example, rollback can be achieved through intermediate state data. Intermediate state data can be valid data that has been written into the system but cannot yet be displayed to the user. In the first stage, namely the import stage, all personnel databases and other information successfully written into the system are in an intermediate state. Before the entire batch import task is completed, these successfully imported personnel may be invisible; users cannot see the intermediate state data on the UI interface, nor can they operate on the intermediate state data. However, the intermediate state data is also "valid," so although it is invisible to the user, it is considered successfully written in terms of business logic. Relevant data restriction rules in the database or face database (e.g., face uniqueness, database unique index) are all valid for the intermediate state data, thus ensuring data consistency.

[0182] When a user cancels an action via the UI, the intermediate data is invisible to the user, eliminating the need for time-consuming data rollback. Therefore, from the user's perspective, the cancellation operation is almost instantaneous. Of course, it's still necessary to clear the intermediate data to free up resources (such as storage space and data uniqueness constraints). Clearing intermediate data can be done asynchronously, without consuming valuable user interaction time, thus ensuring a good user experience. For example, after a user indicates cancellation, the service thread can first reply to the user via the UI that cancellation has been successful, and then use a worker thread to clean up the imported data. The user doesn't need to wait for the cancellation to succeed before receiving the message. Specifically, for example, if a user imports 10,000 batch tasks, and after importing 5,000 tasks and receiving the user's cancellation instruction, the service thread can first reply that cancellation was successful, and then use a worker thread to clean up the data of the 5,000 imported tasks.

[0183] This disclosure improves throughput. To enable the system to support multiple batch import tasks of tens of thousands simultaneously, this disclosure employs a multi-threaded task queue approach. At the start of a task, the user-uploaded file decompression directory is scanned, and each image in the directory is submitted as a portrait registration task to the thread pool's task queue. The thread pool then allocates the task to all worker threads within it for parallel execution. This way, even if multiple batch import tasks arrive simultaneously, they are broken down into sub-tasks (i.e., tasks to be processed) and queued in the task queue. Each sub-task carries information including the file path of an image, occupying very little memory. Therefore, the task queue length can be set very large to accommodate a large number of sub-tasks. Furthermore, the number of worker threads in the thread pool is elastically scalable; when tasks accumulate, more threads can be started to work, thereby achieving higher throughput.

[0184] This disclosure embodiment maintains the task status under high concurrency. Since batch tasks require frequent changes before completion, a distributed cache, Redis, can be used to maintain task information, which is then written to the database after task completion. The task information to be maintained includes task status, the number of personnel successfully imported, and the number of personnel whose imports failed. This disclosure embodiment employs a parallel approach, with multiple worker threads frequently performing modifications.

[0185] The data format and storage method used in the cache of this disclosure embodiment help ensure data consistency and high performance. For example, the number of people who were successfully imported / failed is stored in Redis using a separate key in character data format. When worker threads increment the count, they directly use Redis's native atomic increment command (incr). This ensures the correctness of the value even under high concurrency and prevents data loss.

[0186] In this embodiment, the task states in the cache can each have a corresponding switch identifier. For example, the states of batch tasks include "Start," "Importing," "Pending Submission," "Completed," "Cancelling," and "Terminated." A single field can be used to store these states in the database. If a single field is also used to store the state values ​​in the cache, and worker threads modify the states using an overwriting method, then under concurrent conditions, due to the uncertainty of the thread scheduling execution order, a "new state being overwritten by an old state" may occur. This embodiment can use multiple keys to store switch values ​​as switch identifiers for different states. Modification of the state in a worker thread can include setting the value of the corresponding state from an initial value, such as "0," to a value, such as "1," but cannot be modified in reverse. This ensures that modifications to any state are not overwritten by other states. Furthermore, different states are assigned priorities. If multiple states are set to "1," the true state of the task can be determined based on the priority of the states. For example, the priorities from high to low are "Completed," "Terminated," "Cancelling," "Pending Submission," "Importing," and "Start." If the values ​​of "Cancelling" and "Importing" are both "1," then according to the priority, the current task state is "Cancelling."

[0187] This disclosure improves fault tolerance. Employing asynchronous parallelism, this disclosure considers the impact of various errors that may occur during the asynchronous parallel import process. For example, network fluctuations causing connection interruptions, invalid personnel information fields, and excessive load on the face model can all lead to program execution errors. This disclosure categorizes errors into two types: one is errors caused by invalid input information, such as substandard image quality or fields not conforming to rules; the other is server-side errors, such as network fluctuations or service overload. For the first type of error, depending on business needs, if necessary information is invalid (such as an image), the import can be directly judged as failed. If it is non-necessary information, the invalid information is replaced with a null value, and then a retry is performed. The second type of error is often sporadic and can also be retried.

[0188] This disclosed embodiment can significantly improve the data scale, import speed, and concurrency of batch importing base map data, while ensuring data consistency. It meets the needs of online multi-user import of large amounts of data and the ability to undo at any time, effectively supporting the practical application of systems such as facial recognition management in various scenarios. Currently, in real online systems, it can increase the number of images imported in a single batch to 50,000, and the import speed can be increased by 400% to 500% compared to serial import.

[0189] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0190] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0191] Figure 17 A schematic block diagram of an example electronic device 1700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0192] like Figure 17 As shown, device 1700 includes a computing unit 1701, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1702 or a computer program loaded from storage unit 1708 into random access memory (RAM) 1703. The RAM 1703 may also store various programs and data required for the operation of device 1700. The computing unit 1701, ROM 1702, and RAM 1703 are interconnected via bus 1704. Input / output (I / O) interface 1705 is also connected to bus 1704.

[0193] Multiple components in device 1700 are connected to I / O interface 1705, including: input unit 1706, such as a keyboard, mouse, etc.; output unit 1707, such as various types of displays, speakers, etc.; storage unit 1708, such as a disk, optical disk, etc.; and communication unit 1709, such as a network card, modem, wireless transceiver, etc. Communication unit 1709 allows device 1700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0194] The computing unit 1701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1701 performs the various methods and processes described above, such as a task processing method. For example, in some embodiments, a task processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1700 via ROM 1702 and / or communication unit 1709. When the computer program is loaded into RAM 1703 and executed by the computing unit 1701, one or more steps of a task processing method described above may be performed. Alternatively, in other embodiments, computing unit 1701 may be configured to perform a task processing method by any other suitable means (e.g., by means of firmware).

[0195] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0196] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0197] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0198] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0199] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0200] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0201] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0202] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A task processing method, comprising: Retrieve tasks to be processed from batch tasks; Based on the status of the batch tasks, perform the operation corresponding to the stage of the batch tasks on the task to be processed; The status maintenance of the batch tasks includes: updating the first number of successfully imported tasks and / or the second number of failed imported tasks when performing the first stage operation; and updating the first status of the batch tasks to the second status when it is determined that the number of processed tasks has reached the total number of the batch tasks based on the first number and / or the second number; wherein the first number and the second number are accumulated using separate keys. The batch task's state includes a first state and a second state, and the batch task's stage includes a first stage and a second stage. The first state corresponds to the first stage, and the operation corresponding to the first stage is the first stage operation. The first state indicates that the batch task is in the import process, and the first stage operation includes importing the data of the task to be processed as intermediate state data. The intermediate state data represents valid data that has been written into the system but is not yet displayed to the user. The second state corresponds to the second stage, and the operation corresponding to the second stage is the second stage operation. The second state indicates that the batch task has completed all the first stage operations, and the second stage operation includes submitting the intermediate state data of the task to be processed that has been successfully imported into the batch task as completed state data.

2. The method according to claim 1, wherein, Based on the status of the batch tasks, perform the operations corresponding to the stage of the batch tasks on the tasks to be processed, including: When the batch task is in the first state, perform at least one of the following operations: Perform the first phase operation on the task to be processed; Upon initial inspection, the first phase of the operation is performed on the task to be processed. Upon a second inspection, terminate the worker thread.

3. The method according to claim 1 or 2, wherein, Based on the status of the batch tasks, perform the operations corresponding to the stage of the batch tasks on the tasks to be processed, including: When the batch tasks are in the second state, the second phase operation is performed on the tasks to be processed.

4. The method according to any one of claims 1 or 2, wherein, Based on the status of the batch tasks, performing the operations corresponding to the stage of the batch tasks on the tasks to be processed further includes: If the batch task is in the third state, a rollback operation is performed, wherein the third state indicates that the batch task received a cancellation request during the import process.

5. The method according to claim 4, wherein, The rollback operation includes: Send a notification indicating that the import has been revoked and clear the intermediate data that has been imported.

6. The method according to any one of claims 1 or 2, wherein, Based on the status of the batch tasks, performing the operations corresponding to the stage of the batch tasks on the tasks to be processed further includes: When the batch task is in the fourth or fifth state, the worker thread ends, wherein the fourth state indicates that the batch task has been submitted; and the fifth state indicates that the batch task has been terminated due to cancellation.

7. The method according to any one of claims 1 or 2, wherein, Maintaining the state of the batch tasks further includes at least one of the following: When performing the second phase operation, the second state of the batch task is updated to the fourth state, which indicates that the batch task has been submitted. In the event of a rollback operation, the third state of the batch task is updated to the fifth state, wherein the third state indicates that the batch task received a cancellation request during the import process, and the fifth state indicates that the batch task was terminated due to cancellation.

8. The method according to claim 1, wherein, Retrieve tasks to be processed from batch tasks, including: Using the files and images that need to be imported, generate each of the batch tasks to be processed; Add multiple tasks to be processed to the task queue; The tasks to be processed are read from the task queue using multiple worker threads in the thread pool.

9. The method according to claim 8, wherein, Each of the tasks to be processed includes at least one of the following: personnel information to be imported, identifiers of images to be imported, and storage paths of images to be imported; The personnel information and image identifiers are obtained by parsing from the files to be imported; the storage path of the images to be imported includes the Uniform Resource Locator received after uploading the images.

10. The method according to claim 8 or 9, wherein, Also includes: Download the images to be imported according to the storage path of the images to be imported in the task to be processed; Associate the personnel information to be imported with the images to be imported.

11. The method according to claim 1, wherein, The different states of the batch tasks have different switch indicators.

12. The method according to claim 11, wherein, The value of each of the switch identifiers can be modified from a first value to a second value.

13. The method according to claim 11 or 12, wherein, The status of the batch tasks has a priority. When the switch identifier values ​​corresponding to multiple different statuses are the same, the valid status is determined according to the priority.

14. The method according to any one of claims 1 or 2, wherein, The method further includes: The input information of the task to be processed is checked to determine the error type; Based on the error type, perform corresponding processing on the input information.

15. The method according to claim 14, wherein, The step of performing corresponding processing on the input information according to the error type includes at least one of the following: If the error type is a Type I error and the input information is Type I information, it will be recorded as an import failure. If the error type is a first type of error and the input information is a second type of information, the error information of the task to be processed is cleared and submitted to the thread pool for retry. If the error type is a Type 2 error, the task to be processed will be submitted to the thread pool for retry.

16. A task processing apparatus, comprising: The acquisition module is used to acquire tasks to be processed from batch tasks; The processing module is used to perform the operation corresponding to the stage of the batch tasks on the task to be processed according to the status of the batch tasks; The status maintenance module is used to maintain the status of the batch tasks, including: updating the first number of successfully imported tasks and / or the second number of failed imported tasks when performing the first stage operation; and updating the first status of the batch tasks to the second status when it is determined that the number of processed tasks has reached the total number of the batch tasks based on the first number and / or the second number; wherein the first number and the second number are accumulated using separate keys. The batch task's state includes a first state and a second state, and the batch task's stage includes a first stage and a second stage. The first state corresponds to the first stage, and the operation corresponding to the first stage is the first stage operation. The first state indicates that the batch task is in the import process, and the first stage operation includes importing the data of the task to be processed as intermediate state data. The intermediate state data represents valid data that has been written into the system but is not yet displayed to the user. The second state corresponds to the second stage, and the operation corresponding to the second stage is the second stage operation. The second state indicates that the batch task has completed all the first stage operations, and the second stage operation includes submitting the intermediate state data of the task to be processed that has been successfully imported into the batch task as completed state data.

17. The apparatus according to claim 16, wherein, The processing module includes: The first processing submodule is specifically configured to perform at least one of the following operations when the batch tasks are in the first state: Perform the first phase operation on the task to be processed; Upon initial inspection, the first phase of the operation is performed on the task to be processed. Upon a second inspection, terminate the worker thread.

18. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-15.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-15.

20. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-15.

Images