Multi-agent text data processing method and device based on bitmap state tracking, medium and program product

Abstract

The application relates to the technical field of text data processing, and provides a multi-agent text data processing method, equipment, medium and program product based on bitmap state tracking, which comprises the following steps: obtaining a text data set to be processed and determining a data set identifier thereof; constructing a global state word comprising a first bit segment and a second bit segment, the first bit segment storing the data set identifier, and the second bit segment storing a processing state bitmap corresponding to a plurality of agents; obtaining a data processing task, determining a target agent and a dependency mask thereof; performing a bitwise AND operation on the processing state bitmap and the dependency mask, if the operation result is consistent with the dependency mask, calling historical intermediate data as the target agent input; if the operation result is inconsistent with the dependency mask, inserting a front-end dependency task into a task queue according to a dependency sequence for execution; and storing a text data processing result and updating the global state word after any agent completes the task. The application improves the scheduling efficiency, resource utilization rate and execution stability of a multi-agent text data processing process.

Description

Technical Field

[0001] This application relates to the field of text data processing technology, and in particular to a multi-agent text data processing method, device, medium and program product based on bitmap state tracking. Background Technology

[0002] In processing large-scale unstructured text data (such as online news, online novels, and long text documents), data cleaning, segmentation, and processing are crucial steps in building high-quality datasets. Currently, to improve processing efficiency, a pipeline collaborative model consisting of multiple independent agents (such as cleaning agents, segmentation agents, and summarization agents) is commonly used for automated data processing. In this model, different agents undertake different data processing tasks and sequentially perform cleaning, segmentation, summarization, or extraction operations on the same dataset according to a preset processing order. Subsequent processing tasks typically require intermediate products generated by previous processing tasks as input.

[0003] Existing text data processing systems typically rely on key-value stores, relational database records, or task logs to maintain the processing status of each dataset and invoke different agents to execute processing tasks based on task orchestration processes. However, in scenarios involving massive datasets and concurrent execution by multiple agents, traditional state recording methods require frequent state queries, updates, and correlation checks, easily leading to high storage overhead and access latency. Furthermore, due to the pre-dependencies between tasks, when the system receives a target processing task, it usually needs to perform multiple queries or correlation checks to confirm whether pre-tasks have been completed and whether corresponding intermediate products have been generated, resulting in low efficiency in task dependency determination. When the system cannot quickly confirm the availability of pre-intermediate products, it is prone to repeatedly executing pre-tasks such as cleaning, segmentation, and summarization, thereby increasing computational resource consumption and the cost of calling large models.

[0004] Therefore, there is an urgent need in this field for a multi-agent data processing method for large-scale unstructured text data, so as to achieve low-overhead tracking of the processing status of the dataset and quickly determine task dependencies and reuse historical intermediate results based on the processing status. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application provides a multi-agent text data processing method, device, medium, and program product based on bitmap state tracking, which at least solves the problems in existing technologies where, in the multi-agent pipeline processing of large-scale unstructured text data, the overhead of dataset processing state management is high, making it difficult to quickly determine task dependencies and resulting in low efficiency in reusing historical intermediate results.

[0006] To achieve the above objectives and other advantages, some embodiments of this application provide the following aspects:

[0007] In a first aspect, some embodiments of this application provide a multi-agent text data processing method based on bitmap state tracking, including:

[0008] Obtain the text dataset to be processed and determine the dataset identifier corresponding to the text dataset to be processed;

[0009] Construct a global state word, which includes a first segment and a second segment. The first segment is used to store the dataset identifier, and the second segment is used to store a bitmap of the processing state of multiple agents on the text dataset to be processed. Each state bit in the processing state bitmap corresponds to a preset multiple agent identifiers.

[0010] Obtain the data processing task for the text dataset to be processed, and determine the target agent and its corresponding dependency mask for the data processing task. The state bits set in the dependency mask indicate the pre-processing state that the target agent needs to satisfy before executing the data processing task.

[0011] Perform a bitwise AND operation between the processing state bitmap and the dependency mask. If the result matches the dependency mask, retrieve the historical intermediate data generated by the preceding dependency task and use it as the input data for the target agent. If the result does not match the dependency mask, identify the missing preceding dependency task and insert it into the task queue according to the dependency order to trigger execution.

[0012] After any agent completes the corresponding data processing task and generates the corresponding text data processing result, the text data processing result is stored as input data for subsequent dependent tasks, and the corresponding state bit in the second bit field of the global state word is updated based on the agent identifier corresponding to the agent that completed the data processing task.

[0013] Secondly, some embodiments of this application also provide an electronic device, the electronic device comprising:

[0014] One or more processors; and a memory storing computer program instructions, which, when executed, cause the processors to perform the multi-agent text data processing method based on bitmap state tracking as described above.

[0015] Thirdly, some embodiments of this application also provide a computer-readable storage medium having a computer program and / or instructions stored thereon, wherein the computer program and / or instructions, when executed by a processor, implement the multi-agent text data processing method based on bitmap state tracking as described above.

[0016] Fourthly, some embodiments of this application also provide a computer program product, including a computer program and / or instructions, which, when executed by a processor, implement the multi-agent text data processing method based on bitmap state tracking as described above.

[0017] Compared with existing technologies, the solution provided in this application constructs a global state word for the text dataset to be processed, including a dataset identifier and a processing state bitmap. This unifies the processing completion status of multiple agents on the same text dataset using state bits, eliminating the need to query multiple task records or state tables separately. The global state word allows for rapid determination of the current processing status of the text dataset in the multi-agent processing chain. Furthermore, by performing a bitwise AND operation between the processing state bitmap and a dependency mask, it is possible to directly determine whether the required pre-processing states for the target agent to execute the data processing task have been met. When pre-dependencies are met, historical intermediate data can be directly retrieved as input data for the target agent, avoiding repeated execution of completed pre-processing tasks. When pre-dependencies are not met, the corresponding pre-dependency tasks can be determined based on the missing pre-processing states and inserted into the task queue according to the dependency order to trigger execution, thereby ensuring that the multi-agent task chain is completed and executed according to the correct dependencies. Therefore, this application can achieve low-overhead state tracking, rapid dependency determination, reuse of historical intermediate results, and automatic scheduling of missing pre-tasks in large-scale text data processing scenarios, reducing the cost of repetitive calculations and large model calls, and improving the scheduling efficiency, resource utilization, and execution stability of multi-agent text data processing. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other implementation methods can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating a multi-agent text data processing method based on bitmap state tracking provided in this application embodiment;

[0020] Figure 2 This is a second schematic flowchart of a multi-agent text data processing method based on bitmap state tracking provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The following terms are used in this document.

[0024] An intelligent agent, also known as an intelligent proxy, is an autonomous software entity implemented using artificial intelligence technology. It can perceive its environment by receiving input information and, based on this information, perform semantic understanding, decision-making, planning, and behavioral execution to accomplish pre-defined task objectives. Intelligent agents can be implemented based on rule-based models, machine learning models, large language models, or combinations thereof, and can complete corresponding target tasks under the control of task scheduling mechanisms.

[0025] Some embodiments of this application relate to a multi-agent text data processing method based on bitmap state tracking. This method can be applied to scenarios such as text dataset construction, corpus preprocessing, long text knowledge extraction, web content analysis, and novel or document structuring. In the above scenarios, the text dataset to be processed typically needs to go through multiple processing stages such as cleaning, segmentation, summarization, and extraction, and different stages can be executed by different agents. Due to the dependencies between agents, if the processing state of the same text dataset at each stage cannot be accurately recorded, it is easy to cause repeated execution of preliminary tasks, difficulty in reusing intermediate results, and reduced task scheduling efficiency. Therefore, this embodiment uses a bitmap to record the processing state of multiple agents on the same text dataset, and combines a dependency mask to determine whether the preliminary processing state required for the current data processing task is met. (Refer to...) Figure 1 , Figure 2 As shown, the method may include the following steps:

[0026] Step S1: Obtain the text dataset to be processed and determine the dataset identifier corresponding to the text dataset to be processed.

[0027] In one specific embodiment, the text dataset to be processed can be obtained through a file upload interface, a data acquisition interface, an object storage read interface, or a task scheduling interface. The text dataset to be processed may include unstructured text data such as collections of online news texts, online novel texts, long text documents, question-and-answer corpora, or business log texts. The text dataset to be processed can be a single text file, or a batch of data consisting of multiple text files, multiple text fragments, or multiple text records.

[0028] After determining the text dataset to be processed, a corresponding dataset identifier is assigned to it. The dataset identifier is used to uniquely identify the text dataset during subsequent processing. The dataset identifier can be generated based on the dataset access order, data batch number, text source identifier, hash value, timestamp, or a combination thereof; alternatively, an unused dataset identifier can be assigned to the text dataset from a pre-defined dataset number pool. To ensure consistency in state tracking for the same text dataset in the multi-agent processing chain, the dataset identifier can be bound to the data storage address, metadata information, and task creation information of the text dataset. When constructing the global state word corresponding to the text dataset, the dataset identifier is written into the first segment of the global state word, establishing a correspondence between the global state word and the text dataset.

[0029] Step S2: Construct a global state word. The global state word includes a first segment and a second segment. The first segment is used to store the dataset identifier, and the second segment is used to store the processing state bitmap of multiple agents for the text dataset to be processed. Each state bit in the processing state bitmap corresponds to a preset multiple agent identifiers.

[0030] In a preferred embodiment, the global state word is an unsigned integer variable with the same bit width as the processor register; the dataset identifier in the first segment is written or read through the mask corresponding to the first segment and bit shift operations; the processing state bitmap in the second segment is read through the mask corresponding to the second segment, and is matched and judged by bitwise AND operation with the dependency mask, and the state bit of the corresponding agent identifier is updated by bitwise OR operation; wherein, the bit shift operation, the mask corresponding to the first segment, the mask corresponding to the second segment, the bitwise AND operation, and the bitwise OR operation are all executed based on the bit operation instructions supported by the processor.

[0031] A corresponding global state word can be constructed for each text dataset to be processed. The global state word is used to simultaneously record the dataset identifier and the processing completion status of multiple agents on the same text dataset within the same data structure. Specifically, the global state word can be an unsigned integer variable with the same bit width as the processor register. For example, in one implementation, a 64-bit unsigned integer variable can be used. A portion of the consecutive bits in the unsigned integer variable is used as the first segment to write the dataset identifier, and another portion of consecutive bits is used as the second segment to write the processing status bitmaps corresponding to multiple agents. Since bitwise operations typically operate on fixed-width unsigned integers or bit sequences, using unsigned integer variables that match the processor register bit width facilitates the use of bitwise AND, bitwise OR, and bit shift instructions supported by the processor to complete state reading, dependency judgment, and state updates.

[0032] For example, in a 64-bit processor environment, the global state word can use a 64-bit unsigned integer variable, allowing the first and second segments to be encoded within the same integer variable. This facilitates the execution of dataset identifier reading, dependency matching, and state bit updates through bitwise operations such as bit shifting, masking, bitwise AND, and bitwise OR operations supported by the processor. For instance, the high 16 bits of this 64-bit unsigned integer variable can be used as the first segment to store the dataset identifier, and the low 48 bits as the second segment to store the processing state bitmap corresponding to multiple agents. Since the high 16 bits can represent 2^32 bits, this method allows for the global state word to be encoded using a 64-bit unsigned integer variable. 16 The first segment can take many different values, so in this example, it can support up to 65,536 unique dataset numbers. Each bit in the lower 48 bits can correspond to the processing state of an agent, so the second segment can represent the processing completion status of up to 48 agents on the same text dataset. It should be noted that the above 64 bits, high 16 bits, and low 48 bits are only exemplary configurations. In other implementations, the bit width of the global state word and the lengths of the first and second segments can be adjusted according to the processor architecture, the number of datasets, and the number of agents. For example, when more agent states need to be supported, the number of bits in the second segment can be increased, or a higher-width unsigned integer variable can be used.

[0033] When constructing the global state word, the dataset identifier can be written into the first segment of the global state word using a mask corresponding to the first segment and a shift operation. Specifically, the original bit value of the first segment in the global state word can be cleared first based on the mask corresponding to the first segment, and then the dataset identifier can be shifted according to the position of the first segment in the global state word and written into the first segment. Correspondingly, when it is necessary to read the dataset identifier, the bit value of the first segment can be extracted from the global state word using the mask corresponding to the first segment, and the corresponding dataset identifier can be obtained by reverse shifting. In the above way, the dataset identifier allocated in step S1 is encoded into the global state word, so that the global state word establishes a definite binding relationship with the corresponding text dataset to be processed.

[0034] For the second bit segment, the mapping relationship between the agent identifier and the status bit can be pre-maintained. Each agent identifier corresponds to a status bit in the processing status bitmap and can correspond to a unique bitmask. For example, the cleaning agent corresponds to the 0th status bit, the splitting agent corresponds to the 1st status bit, the summarization agent corresponds to the 2nd status bit, and the event extraction agent corresponds to the 3rd status bit; correspondingly, the ith agent can correspond to the bitmask "1<<i" (that is, shift the binary number 1 to the left by i bits to form a bitmask with only the ith status bit being 1 and the remaining status bits being 0). When the value of a certain status bit is 0, it indicates that the corresponding agent has not completed the processing of the text data set to be processed; when the value of this status bit is 1, it indicates that the corresponding agent has completed the processing of the text data set to be processed. Thus, the processing completion status of multiple agents can be compressed and recorded in the same processing status bitmap.

[0035] When initializing the global status word, the first bit segment can be written with the data set identifier, and the status bits in the second bit segment can be initialized to the uncompleted state. For example, for the text data set to be processed with the data set identifier of 15, the system can write the value 15 into the first bit segment of the global status word and initialize the processing status bitmap of the second bit segment to all 0s to indicate that each agent has not completed the processing of this text data set to be processed. As subsequent agents gradually execute the corresponding data processing tasks, the second bit segment is updated according to the status bits corresponding to each agent identifier.

[0036] When reading the processing status bitmap, the bit value of the second bit segment can be extracted from the global status word through the mask corresponding to the second bit segment, so as to obtain the processing status bitmap corresponding to the current text data set to be processed. This processing status bitmap can be used for subsequent bitwise AND operations with the dependency mask to determine whether the prerequisite processing status required before the target agent executes the data processing task has been met. For example, if both the cleaning agent and the splitting agent need to be completed before the target agent executes the task, the status bits corresponding to the cleaning agent and the splitting agent in the dependency mask are set; perform a bitwise AND operation on the processing status bitmap and the dependency mask, and when the operation result is the same as the dependency mask, it is determined that the prerequisite processing status has been met.

[0037] When updating the processing status bitmap, the target status bit can be determined based on the agent identifier corresponding to the agent that has completed the data processing task, and the target status bit can be set to the completed state through a bitwise OR operation. For example, when the splitting agent has completed the splitting task of the text data set to be processed, the bit mask corresponding to it is determined according to the splitting agent identifier, and the processing status bitmap in the second field of the global status word is bitwise ORed with the bit mask, so that the status bit corresponding to the splitting agent is updated from 0 to 1. In this way, the processing status of different agents can be continuously updated in the same global status word without maintaining independent status records for each agent separately.

[0038] In some embodiments, in order to avoid conflicting updates to the same global status word when multiple agents execute concurrently, the concurrency control permission corresponding to the data set identifier can also be obtained before updating the status bit in the second field, and the concurrency control permission is released after the bitwise OR update is completed. Specifically, taking the global status word corresponding to the text data set A to be processed as gsw x and the second field is located in the low field of the global status word as an example, the bit mask corresponding to the i-th agent can be 1 << i. When determining whether the data set A has been processed by the i-th agent, the concurrency control lock lock a corresponding to the data set A can be obtained first, and then the bitwise AND operation is performed on gsw x and 1 << i; if the operation result of the expression (gsw x & (1 << i)) is greater than 0, it means that the status bit corresponding to the i-th agent in the global status word has been set, and it is determined that the data set A has been processed by the i-th agent; if the operation result of the expression (gsw x & (1 << i)) is equal to 0, it means that the i-th agent has not completed the processing of the data set A; after the judgment, the concurrency control lock lock a .

[0039] Furthermore, after the i-th agent has completed the data processing task of the data set A, the concurrency control lock lock a corresponding to the data set A can be obtained again, and the bitwise OR operation is performed on gsw x and 1 << i, that is, the expression (gsw x |= (a << i)) is executed to update the status bit corresponding to the i-th agent to the completed state. Since the bitwise OR operation only sets the target status bit corresponding to the bit mask to 1 without changing the status bits corresponding to other agents, the update of the processing status of the i-th agent can be completed while keeping other processing statuses unchanged. After the update is completed, the system releases the concurrency control lock lock aBy employing the above method, the consistency of the reading, judgment, and updating process of the global state word can be guaranteed when multiple agents or multiple task threads are processing the same text dataset concurrently, thus avoiding state bit overwriting, state loss, or duplicate scheduling caused by concurrent access.

[0040] In this embodiment, the dataset identifier of the text dataset to be processed and the processing completion status of multiple agents are encoded into the same global state word. The dataset identifier reading, processing status query, dependency matching judgment, and state bit update are completed using bitwise operations, bitwise AND, and bitwise OR operations supported by the processor. Compared to methods that determine the processing status of each agent by querying multiple database records, multi-field state tables, or item-by-item task logs, this embodiment transforms the querying and updating of the processing status of multiple agents into bitwise operations on the global state word. This results in constant-time complexity for state querying and updating, thereby reducing the state management overhead and system resource consumption in large-scale text data processing scenarios.

[0041] Step S3: Obtain the data processing task for the text dataset to be processed, and determine the target agent and its corresponding dependency mask for the data processing task. The state bits set in the dependency mask represent the pre-processing states that the target agent needs to satisfy before executing the data processing task.

[0042] In one specific embodiment, after constructing the global state word of the text dataset to be processed, a data processing task for that dataset can be received. This data processing task can be a text cleaning task, text segmentation task, text summarization task, keyword extraction task, event extraction task, etc., or it can be a task with a clearly defined target processing result submitted by the user or the upper-level task scheduling system. For example, a user can submit a data processing task to "extract events from dataset A." The system can identify the task type corresponding to this data processing task based on the task description, task type field, or task interface parameters, and determine the target agent to be invoked to execute the task, such as an event extraction agent.

[0043] After determining the task type of the data processing task, the configuration information corresponding to that task type can be retrieved from the preset task configuration table. The preset task configuration table records the correspondence between different types of data processing tasks and agents, and records the pre-processing states that must be met before each task can be executed. Specifically, the preset task configuration table may include fields such as task type, target agent identifier, the state bit corresponding to the target agent, dependency mask, list of pre-dependent agent identifiers, input data type, and intermediate result storage path type. The target agent identifier is used to determine the agent executing the current data processing task; the dependency mask indicates the pre-processing states that the target agent must complete before executing the current data processing task.

[0044] For example, the task configuration table can be pre-configured as follows: A text cleaning task corresponds to a cleaning agent, which corresponds to the 0th state bit in the processing state bitmap. Since the cleaning task does not depend on other preceding tasks, its dependency mask can be 0. A text segmentation task corresponds to a segmentation agent, which corresponds to the 1st state bit. The text segmentation task depends on the cleaning task for completion, so the 0th state bit in its dependency mask is set. A text summarization task corresponds to a summarization agent, which corresponds to the 2nd state bit. The text summarization task can depend on the text segmentation task for completion, so the 1st state bit in its dependency mask is set. An event extraction task corresponds to an event extraction agent, which corresponds to the 3rd state bit. The event extraction task can depend on the text summarization task for completion, so the 2nd state bit in its dependency mask is set. It should be noted that the above state bit allocation is only an example. In other embodiments, the mapping relationship between each agent identifier and the state bit can be adjusted according to the number of agents, task type, and task chain relationship.

[0045] In one example, if the current data processing task is an "event extraction task," the target agent can be retrieved from the preset task configuration table based on this task type, and the dependency mask corresponding to the event extraction agent can also be retrieved. If the task configuration table specifies that the event extraction task needs to be executed after the summary task is completed, the state bit corresponding to the summary agent in the dependency mask is set. Therefore, the dependency mask is not used to indicate whether the target agent itself has completed execution, but rather to indicate whether the required preprocessing states before the target agent executes the current data processing task have been completed.

[0046] In some embodiments, the dependency mask and the processing state bitmap use the same agent state bit mapping relationship. Specifically, the state bits in the processing state bitmap are used to represent the processing completion state of the corresponding agent, and the state bits in the dependency mask are used to represent the prerequisite processing states that the current data processing task needs to satisfy. When a certain state bit in the dependency mask is set, it indicates that the current data processing task uses the processing completion state of the agent corresponding to that state bit as a prerequisite execution condition. Based on the above consistent state bit mapping relationship, a bitwise AND operation can be performed on the processing state bitmap and the dependency mask to determine whether all the prerequisite processing states required by the current data processing task have been satisfied. The bitwise AND operation can be used to detect whether a specific bit is set, that is, the bit in the result is 1 only when both corresponding bits involved in the operation are 1.

[0047] For example, suppose the cleaning agent, segmentation agent, summarizing agent, and event extraction agent correspond to the 0th, 1st, 2nd, and 3rd state bits in the processing state bitmap, respectively. If the current data processing task is a text summarizing task, and this text summarizing task needs to be executed after the text segmentation task is completed, then the 1st state bit in the dependency mask corresponding to the text summarizing task is set, for example, it can be represented as 0010. This dependency mask does not indicate that the summarizing agent has completed processing, but rather that the text summarizing task requires the segmentation agent corresponding to the 1st state bit to have completed processing before execution. If the current processing state bitmap is 0011, it means that the cleaning agent and the segmentation agent have completed processing. At this time, the bitwise AND operation result of the processing state bitmap and the dependency mask is 0010, which is consistent with the dependency mask, indicating that the pre-processing state required for the text summarization task has been met. If the current processing state bitmap is 0001, it means that only the cleaning agent has completed processing. At this time, the bitwise AND operation result of the processing state bitmap and the dependency mask is 0000, which is inconsistent with the dependency mask, indicating that the segmentation agent has not yet completed processing. The system needs to insert the text segmentation task as a missing pre-dependent task into the task queue to trigger execution.

[0048] The above method can convert the pre-processing dependencies between tasks into a dependency mask that can be judged by bitwise operations. This allows the system to quickly determine whether the pre-processing states required by the current data processing task have been met based on the bitwise AND operation between the processing state bitmap and the dependency mask. This reduces the computational overhead of task dependency judgment in multi-agent text data processing and provides a basis for judgment for subsequent reuse of historical intermediate data and completion of missing pre-processing dependency tasks.

[0049] Step S4: Perform a bitwise AND operation between the processing state bitmap and the dependency mask. If the result matches the dependency mask, retrieve the historical intermediate data generated by the preceding dependency tasks and use it as the input data for the target agent. If the result does not match the dependency mask, identify the missing preceding dependency tasks and insert them into the task queue according to the dependency order to trigger execution.

[0050] In one specific embodiment, the processing state bitmap corresponding to the current text dataset to be processed can be obtained according to the second bit field of the global state word, and a bitwise AND operation can be performed between the processing state bitmap and the dependency mask determined in step S3. The bitwise AND operation can be used to detect whether a specified bit is set. Only when both corresponding state bits involved in the operation are 1 will the state bit in the operation result be 1. Therefore, when the processing state bitmap and the dependency mask adopt the same agent state bit mapping relationship, if the bitwise AND operation result is consistent with the dependency mask, it means that all the preprocessing states required by the dependency mask have been satisfied.

[0051] After confirming that the prerequisite dependencies have been satisfied, the identifier of the preceding agent that needs to retrieve historical intermediate data can be determined based on the dataset identifier and the set state bits in the dependency mask. For example, if the state bit corresponding to the segmentation agent in the dependency mask is set, the text segmentation result corresponding to the text dataset to be processed can be queried in the intermediate data storage area, result cache, object storage, or database based on the dataset identifier and the segmentation agent identifier; if the state bit corresponding to the summarization agent in the dependency mask is set, the summary result or structured summary data corresponding to the text dataset to be processed can be queried based on the dataset identifier and the agent identifier.

[0052] After retrieving historical intermediate data, the data can be formatted or its fields transformed according to the input requirements of the target agent. For example, when the target agent is a summarizing agent, the text block sequence, text block identifiers, and text block order information generated by the segmenting agent can be encapsulated as input data for the summarizing agent; when the target agent is an event extraction agent, the summary results, key entities, time information, or structured summary fields generated by the summarizing agent can be encapsulated as input data for the event extraction agent. In this way, the target agent can directly execute the current data processing task based on the historical intermediate data already generated by the preceding dependent tasks, without having to re-execute the completed preceding dependent tasks.

[0053] In a preferred embodiment, step S4, where the computation result is inconsistent with the dependency mask, involves identifying the missing prerequisite dependency tasks and inserting them into the task queue according to the dependency order to trigger execution. This step specifically includes:

[0054] Step S4A101: Based on the difference state bits between the dependency mask and the operation result, determine the unmet pre-processing state, and determine the missing pre-dependent task based on the agent identifier corresponding to the unmet pre-processing state.

[0055] In one specific embodiment, when the result of a bitwise AND operation between the processing state bitmap and the dependency mask is inconsistent with the dependency mask, it indicates that at least one pre-processing state in the dependency mask that is set is not yet in a completed state in the processing state bitmap. In this case, the unmet pre-processing states can be determined based on the difference state bits between the dependency mask and the operation result. Specifically, the state bits that are set in the dependency mask but not in the operation result can be identified as difference state bits; these difference state bits represent the pre-processing states that the current data processing task has not yet met. Since the processing state bitmap, dependency mask, and agent identifier use the same state bit mapping relationship, the corresponding agent identifier can be queried based on the difference state bits, and the pre-processing dependency tasks that need to be completed can be determined based on this agent identifier.

[0056] For example, suppose the cleaning agent, segmentation agent, summarization agent, and event extraction agent correspond to state bits 0, 1, 2, and 3 respectively. The current data processing task is the event extraction task, which needs to be executed after cleaning, segmentation, and summarization are completed. Then, the dependency mask for this event extraction task can be 0111, where state bit 0 indicates that the cleaning agent has completed processing, state bit 1 indicates that the segmentation agent has completed processing, and state bit 2 indicates that the summarization agent has completed processing. If the current processing state bitmap is 0001, it means the cleaning agent has completed processing. Performing a bitwise AND operation between the processing state bitmap and the dependency mask yields 0001, which is inconsistent with the dependency mask 0111. In this case, the difference state bit between the dependency mask and the operation result is 0110, indicating that the preceding processing states corresponding to the segmentation agent and the summarization agent have not yet been satisfied. Based on the agent identifiers corresponding to the first and second state bits, the missing preceding dependency tasks can be determined to be the text segmentation task and the text summarization task.

[0057] Step S4A102: Generate corresponding task items based on the missing prerequisite tasks, and insert the task items into the task queue according to the dependency order among the missing prerequisite tasks.

[0058] After identifying the missing prerequisite tasks, a corresponding task item can be generated for each missing prerequisite task. The task item serves as the scheduling and execution unit in the task queue, carrying the task context for the prerequisite task during subsequent execution, pause, debugging, resumption, and result reuse. Specifically, a task item may include a task identifier, dataset identifier, agent identifier, task state, input data reference, processing location identifier, user instruction set, and result cache. Among them, the task identifier is used to distinguish different task items; the dataset identifier is used to index the global state word corresponding to the text dataset to be processed, so that it can locate the corresponding processing state bitmap based on the dataset identifier; the agent identifier is used to determine the corresponding state bit in the processing state bitmap, so as to update the processing completion state of the corresponding agent after the task item is executed; the task state is used to indicate that the task item is in the state of pending execution, executing, pending debugging, completed, or execution failure; the input data reference is used to point to the input data required when the task item is executed, which can be the original text data to be processed, historical intermediate data generated by the preceding dependent tasks, or data entities in object storage; the processing position identifier is used to record the processing breakpoint when the corresponding agent processes the text dataset to be processed; the user instruction set is used to record human feedback instructions, debugging instructions, or execution constraints for the task item; and the result cache is used to save reusable intermediate processing results or stage processing results generated during the execution of the task item.

[0059] For example, if the current data processing task is an event extraction task, and the missing prerequisite tasks are identified as text segmentation and text summarization, then segmentation task items and summarization task items can be generated separately. In the segmentation task item, the dataset identifier points to the global state word corresponding to the text dataset to be processed, the agent identifier points to the state bit corresponding to the segmentation agent, the input data reference can point to the cleaned text data or the original text data, and the processing position identifier can record the character position, paragraph position, or text block sequence number currently processed by the segmentation agent. In the summarization task item, the input data reference can point to the text block sequence generated by the segmentation task item, and the result cache can be used to save the summary results generated by the summarization agent for reuse by subsequent event extraction agents. Thus, a task item not only represents a task to be executed but also organizes the dataset, agent, state bit, input data, processing breakpoints, manual instructions, and intermediate results into a schedulable context carrier.

[0060] When inserting tasks into the task queue, the enqueue order can be determined based on the dependency order among missing prerequisite tasks. This dependency order can be determined by a preset task configuration table, an agent topology graph, or a task chain configuration. For example, if a text summarization task depends on the output of a text segmentation task, the segmentation task should be inserted into the task queue first, followed by the summarization task. This ensures that missing prerequisite tasks are completed and executed in the correct task chain order, preventing subsequent agents from being incorrectly scheduled when necessary input data is missing.

[0061] Step S4A103: Set the task status of the task item inserted into the task queue to the pending execution status. The task queue is configured with a position cursor, which is used to locate the pending task item in the task queue.

[0062] In one specific embodiment, after inserting the task item corresponding to the missing prerequisite task into the task queue, the task status of that task item can be set to "pending execution" to indicate that the task item has entered the schedulable stage. The task queue can be configured with a position cursor, which is used to locate the pending task item or the current task item in the task queue. The position cursor can be implemented using a queue index, task pointer, task item identifier mapping, or other data structures that can indicate the current position of the queue. When there are multiple pending task items in the task queue, the task item that needs to be triggered for execution can be determined based on the position cursor; when a task item completes execution, is paused for debugging, or is skipped, the position cursor can move to the next pending task item according to the task status and dependency order.

[0063] Step S4A104: When the position cursor points to a task item, the task item is determined as the current task item, and the task context is loaded based on the current task item. The agent corresponding to the current task item is scheduled to execute the corresponding pre-dependent task.

[0064] In one specific embodiment, when the position cursor locates a task item to be executed in the task queue, this task item can be identified as the current task item, and the task context can be loaded based on the current task item. The task context may include the input data to be input based on the input data reference, the processing start position determined based on the processing position identifier, the agent execution constraints determined based on the user instruction set, and the reusable intermediate processing results determined based on the result cache. Specifically, the corresponding input data to be input can be read first according to the input data reference in the current task item; then, the processing start position when the agent executes the task can be determined according to the processing position identifier, such as continuing processing from a specified character position, a specified paragraph, a specified text block, or a specified processing breakpoint; then, the correction requirements, filtering conditions, segmentation rules, or other execution constraints that the agent needs to follow when executing the task can be determined according to the user instruction set; if there are already reusable intermediate processing results in the result cache, these intermediate processing results can be loaded together to avoid repeatedly processing the already completed parts of the data.

[0065] For example, when the current task corresponds to a segmentation agent, it can load the text to be segmented based on the input data reference, determine whether to continue segmentation from the Mth character or the Nth text block based on the processing position identifier, and determine the correction requirements that the segmentation agent needs to follow during segmentation based on the user instruction set, such as avoiding misclassifying specific semantic phrases as chapter titles. When the current task corresponds to a summary agent, it can load the text block sequence generated by the segmentation agent based on the input data reference or result cache, and use this text block sequence as input data for the summary agent to trigger the summary agent to execute the text summary task. After the current task is completed, the text data processing result generated by the agent can be written to the result cache or intermediate data storage area, and the corresponding state bit in the second field of the global state word can be updated based on the agent identifier in subsequent state update steps.

[0066] In this way, task items in the task queue not only represent tasks to be executed, but also serve as context-bearing structures in the task scheduling process. Global state words can be located using dataset identifiers, state bits in the processing state bitmap can be located using agent identifiers, breakpoint continuation can be achieved using processing position identifiers, re-execution under manual feedback constraints can be achieved using user instruction sets, and intermediate processing results can be reused using result caching. This ensures that missing pre-dependent tasks can be accurately triggered and executed continuously in the order of dependency.

[0067] In the solution provided in this embodiment, through steps S4A101 to S4A104, when the computation results of the processing state bitmap and the dependency mask are inconsistent, the unmet preceding processing states can be determined based on the difference state bits, and the missing preceding dependent tasks can be mapped. By generating task items, enqueuing them in dependency order, setting the pending execution state, and using a position cursor to locate the current task item, the automatic completion and orderly scheduling of missing preceding dependent tasks are achieved. Simultaneously, by loading the task context based on the current task item, the corresponding agent can obtain the necessary input data, processing breakpoints, user instructions, and reusable intermediate results during execution. Therefore, it is possible to avoid the target agent being incorrectly scheduled when preceding data is missing, reduce redundant calculations and manual intervention, and improve the continuity and execution stability of the multi-agent text data processing chain.

[0068] In a preferred embodiment, when the task status of the current task item is in a pending debugging state, a debugging instruction for the current task item is received, and at least one of the following processes is performed according to the debugging instruction:

[0069] When the debugging command is a task backtracking command, adjust the position cursor to the task item before the current task item, and reload the task context based on the adjusted task item;

[0070] When the debugging command is a task forward command, adjust the position cursor to the task item after the current task item to skip the current task item or enter the subsequent task item;

[0071] When the debugging instruction is a user feedback instruction, the user correction information is written into the user instruction set of the current task item, and the agent corresponding to the current task item is rescheduled to perform the data processing task based on the updated user instruction set.

[0072] When the debugging instruction is a rule-based instruction, the user correction information that meets the preset debugging success conditions is converted into the corresponding agent's processing rules and written into the rule set.

[0073] Specifically, to support human intervention, problem localization, and rule correction during multi-agent text data processing, a human-machine loopback debugging mechanism can be provided based on task queues and position cursors. When the agent corresponding to the current task detects a processing anomaly or receives a manual pause command during data processing, the execution of the current task can be paused, and the task status of the current task can be set to a pending debugging state. Processing anomalies can include text segmentation errors, incomplete summary results, abnormal keyword extraction results, missing event extraction results, or events whose format does not meet preset requirements. After the task status is set to a pending debugging state, it can receive debugging commands for the current task and update the position cursor, task context, user command set, or agent processing rules accordingly based on the debugging commands.

[0074] A console interface can be provided to receive debugging commands for the current task. Debugging commands can include at least one of the following: task backtracking commands, task forwarding commands, user feedback commands, and rule persistence commands. As an example, a task backtracking command can be represented by the ' / back' command, a task forwarding command by the ' / forward' command, a user feedback command by the ' / feedback' command, and a rule persistence command by the ' / append' command. It should be noted that the above command names are merely exemplary interface forms; in other embodiments, the same debugging control functionality can be achieved using button triggers, interface calls, message queue events, or other command identifiers.

[0075] When the debugging command is a task backtracking command, the position cursor can be adjusted to the task item preceding the current task item, and the task context can be reloaded based on the adjusted task item. Specifically, the execution environment of the previous task item can be restored based on the dataset identifier, agent identifier, input data reference, processing location identifier, user command set, and result cache in the adjusted task item, so that developers or the system can re-examine the input data, execution results, or intermediate processing of the previous task item. For example, when the event extraction result is abnormal, the position cursor can be rolled back to the text summarization task item or the text segmentation task item using the task backtracking command to check whether the abnormality stems from incomplete summary results or incorrect text segmentation boundaries.

[0076] When the debugging command is a task advance command, the position cursor can be adjusted to the task item after the current task item to skip the currently suspended task item or proceed to the next task item. Specifically, when an exception in the current task item does not affect the continued execution of subsequent tasks, or when the developer confirms that the current task item can be temporarily skipped, the current task item can be kept in a suspended state or marked as skipped, and the position cursor can be moved to the next task item, allowing the task queue to continue advancing. In this way, a single task item can be avoided from blocking the entire multi-agent processing chain for an extended period of time.

[0077] When the debugging instruction is a user feedback instruction, user correction information can be written into the user instruction set of the current task item, and the agent corresponding to the current task item can be rescheduled to execute the data processing task based on the updated user instruction set. Specifically, the user correction information may include suggestions for correcting the processing results, requirements for adjusting the processing boundaries, exclusion conditions for error identification results, requirements for limiting the output format, or other execution constraints for the current task item. After writing the user correction information into the current task item, the input data to be input can be reloaded based on the input data reference in the current task item, the starting position for re-execution can be determined based on the processing position identifier, and the updated user instruction set can be input as an agent execution constraint to the corresponding agent, so that the agent re-executes the current task item according to the corrected context. For example, when the summary result generated by the summarizing agent is missing preset fields or key facts, the developer can input user correction information to supplement the summary elements or limit the summary format, write the user correction information into the user instruction set of the current summarizing task item, and reschedule the summarizing agent to execute the summary generation task based on the updated user instruction set.

[0078] When the debugging command is a rule-based command, user correction information that meets the preset debugging success conditions can be converted into processing rules for the corresponding agent and written into the rule set. Specifically, after re-executing the current task based on the user correction information, the re-execution result can be verified. If the re-execution result meets the preset debugging success conditions, such as the summary result meeting the format requirements, the event extraction result containing necessary fields, the keyword extraction result being manually confirmed, or the task output passing the preset verification rules, then the user correction information can be extracted into reusable processing rules. Processing rules can be natural language rules, prompt word rules, regular expression rules, filtering conditions, or other rule forms that can be loaded and executed by the corresponding agent. For example, in cases where the summary result lacks key fields, the user correction information can be converted into prompt word rules to limit the summary fields, summary granularity, or summary output format, and these prompt word rules can be written into the rule set corresponding to the summarizing agent for use in subsequent summarizing tasks.

[0079] Through the above methods, this embodiment forms an interactive task debugging chain based on task queues, position cursors, and console interfaces. This allows developers to locate the current task item, backtrack to previous tasks, inject user correction information, and solidify successful debugging processing rules during agent execution. Compared to treating the agent merely as an uninterruptible black-box execution unit, this embodiment can dynamically correct abnormal or ambiguous results during agent execution without restarting the service or modifying the agent's program code. It also solidifies effective corrections into reusable rules, thereby improving the maintainability, debuggability, and development iteration efficiency of the multi-agent text data processing system.

[0080] In a preferred embodiment, the agent corresponding to the current task item is a segmentation agent; during the process of the segmentation agent performing text segmentation processing on the text dataset to be processed based on the current segmentation rules...

[0081] If a splitting discrepancy or splitting anomaly is detected, the execution of the current task item is paused, and the task status of the current task item is set to the pending debugging state.

[0082] After receiving user feedback instructions, the user correction information is written into the user instruction set of the current task item, and the segmentation points of the segmentation agent are corrected based on the user correction information.

[0083] Upon receiving the rule solidification instruction, the user correction information that meets the preset debugging success conditions is converted into natural language segmentation instructions or regular expression rules, and the natural language segmentation instructions or regular expression rules are written into the rule set corresponding to the segmentation agent, so that the segmentation agent can load the updated rule set and perform text segmentation processing in subsequent text segmentation tasks.

[0084] Specifically, based on the aforementioned human-machine loopback debugging mechanism, the agent corresponding to the current task can be a segmentation agent. The segmentation agent performs text segmentation processing on the text dataset to be processed based on the current segmentation rules. These rules may include preset rules for chapter title recognition, paragraph boundary recognition, text block length control, semantic boundary judgment, prompt word rules, or regular expression rules. During the text segmentation process, the segmentation agent can perform anomaly detection on the segmentation results to determine if there are any segmentation ambiguities or anomalies. Segmentation ambiguities or anomalies may include: misidentifying main text as chapter titles, omitting chapter titles, incorrectly splitting semantically continuous text, merging text blocks that should be split, or the segmentation results not meeting preset requirements for text block length, semantic integrity, or structural boundaries.

[0085] For example, when processing text from online novels, the segmentation agent might misidentify "first round" in "first round of the competition" as a chapter title, thus incorrectly generating a chapter segmentation point. In this case, the current segmentation result can be determined to have segmentation ambiguity, and the execution of the current task item can be paused, setting its task status to "pending debugging." After setting the task status to "pending debugging," the task context, including input data references, processing location identifiers, user instruction sets, and result caches, can be retained for subsequent reprocessing of text fragments near the segmentation location based on human feedback.

[0086] Upon receiving user feedback instructions, user correction information can be written into the user instruction set for the current task item. This correction information can be natural language suggestions from developers regarding segmentation errors, such as "Do not identify 'First Round' containing competition semantics as a chapter segmentation point," or structured correction information regarding segmentation point locations, segmentation boundaries, or exclusion conditions. After writing the user correction information, the text to be segmented can be reloaded based on the input data references in the current task item. The location of the text where the segmentation discrepancy occurred, or its context window, can be located based on the processing location identifier. The updated user instruction set is then used as the execution constraint for the segmentation agent, causing the agent to re-execute the segmentation process at the corresponding text location according to the corrected segmentation requirements. In this way, user feedback information is not a separately saved note, but rather a context constraint participating in the re-execution of the current task item.

[0087] Upon receiving the rule solidification instruction, it can be determined whether the segmentation result after re-execution based on user correction information meets the preset debugging success conditions. These preset debugging success conditions may include: the regenerated segmentation result is manually confirmed, the segmentation point positions meet the user correction requirements, the text block boundaries meet preset rules, the segmented text blocks maintain semantic integrity, or the segmentation result passes the system's preset format validation rules. If the preset debugging success conditions are met, the user correction information can be converted into processing rules that the segmentation agent can load and written into the rule set corresponding to the segmentation agent.

[0088] User corrections can be converted into natural language segmentation instructions, prompt word rules, regular expression rules, or filtering conditions. For example, in the case where "the first round of the competition" is misclassified as a chapter title, the user correction information can be converted into a natural language segmentation instruction: "When 'the first round' is followed by event-related words such as 'competition,' 'event,' or 'activity,' do not treat it as a chapter title." Alternatively, it can be converted into regular expression rules or filtering conditions to exclude corresponding semantic combinations. After this natural language segmentation instruction or regular expression rule is written into the rule set corresponding to the segmentation agent, the segmentation agent can load the updated rule set in subsequent text segmentation tasks, thereby avoiding the recurrence of the same segmentation errors in the same or similar semantic scenarios.

[0089] In this way, when performing text segmentation tasks, the segmentation agent can pause the current task upon detecting segmentation ambiguities or anomalies, and dynamically correct the segmentation points based on user feedback. Once the correction results meet preset debugging success conditions, the corresponding correction logic is further converted into natural language segmentation instructions, prompt word rules, or regular expression rules and written into the rule set. This allows the segmentation agent to load the updated rule set in subsequent text segmentation tasks. Therefore, hot updates of segmentation rules can be achieved without restarting the service or modifying the segmentation agent's program code, reducing the recurrence of similar segmentation errors and improving the adaptability and maintenance efficiency of the text segmentation process.

[0090] Step S5: After any agent completes the corresponding data processing task and generates the corresponding text data processing result, the text data processing result is stored as the input data for subsequent dependent tasks, and the corresponding state bit in the second segment of the global state word is updated based on the agent identifier corresponding to the agent that completed the data processing task.

[0091] In one specific embodiment, when the current task item in the task queue is scheduled for execution, the agent corresponding to the current task item can execute the corresponding data processing task based on the loaded task context and generate the corresponding text data processing result. The text data processing result can have different data forms depending on the type of agent. For example, the text data processing result generated by the cleaning agent can include cleaned text after removing HTML tags, advertising text, garbled characters, or duplicate content; the text data processing result generated by the segmentation agent can include text block sequences, text block identifiers, text block order information, and segmentation boundary information; the text data processing result generated by the summarizing agent or the abstracting agent can include full-text abstracts, segmented abstracts, or structured abstract fields; the text data processing result generated by the event extraction agent can include structured extraction results such as event subject, event time, event location, event type, or event description.

[0092] After any agent completes its corresponding data processing task, the text data processing result generated by that agent can be associated and stored with the dataset identifier, agent identifier, and task identifier. Specifically, the text data processing result can be written to an intermediate data storage area, result cache, object storage, database, or other storage medium accessible to subsequent tasks, and corresponding result reference information can be generated. The result reference information can serve as input data references for subsequent dependent tasks, enabling subsequent agents to retrieve the text data processing result based on the dataset identifier and the identifier of the preceding agent when executing the corresponding data processing task, without having to re-execute the completed preceding task.

[0093] In some embodiments, different data transfer methods can be selected based on the data size of the text data processing results. For text data processing results with small data volumes, they can be directly written to the result cache, and the cache reference can be used as the input data reference for subsequent dependent tasks. For text data processing results with file sizes within a preset range, such as between 5MB and 20MB, data can be transferred between different agents using streaming or object storage references. Specifically, the segmentation results, summarization results, or other intermediate processing results can be written to object storage, and the corresponding object storage address, file identifier, or data fragment index can be recorded in the task item. When subsequent dependent tasks are scheduled for execution, the target agent obtains the corresponding data in a streaming manner based on the input data reference, without having to load the complete text data into memory all at once. This method can reduce the memory usage of large files or long text data during multi-agent transfer, avoid memory overflow caused by loading large intermediate data at once, and ensure that subsequent dependent tasks can continue to execute based on historical intermediate data generated by previous tasks.

[0094] Furthermore, after the text data processing results are stored, the state bit corresponding to the agent that completed the data processing task can be determined in the processing state bitmap based on the agent identifier, and the state bit in the second bit field of the global state word can be updated. Specifically, the bitmask corresponding to the agent that completed the task can be determined according to the preset mapping relationship between agent identifiers and state bits; then, a bitwise OR operation is performed between the second bit field of the global state word and the bitmask to update the corresponding state bit to the completed state. Since the bitwise OR operation only sets the target state bit to 1 without changing the state bits corresponding to other agents, the current agent's processing state can be marked while keeping other processing states unchanged.

[0095] For example, suppose the segmentation agent corresponds to the first state bit in the processing state bitmap, and its corresponding bitmask is 1 << 1. After the segmentation agent completes the text segmentation task and generates the segmentation result, the first state bit can be determined based on the segmentation agent identifier, and a bitwise OR update can be performed on the second bit segment of the global state word to update the first state bit from an incomplete state to a completed state. Subsequently, when the data processing task corresponding to the summarization agent needs to determine whether the segmentation task has been completed, a bitwise AND operation can be performed between the processing state bitmap and the dependency mask corresponding to the summarization task to quickly confirm that the previous processing state corresponding to the segmentation agent has been satisfied.

[0096] In some embodiments, to ensure consistency between state updates and result storage, the corresponding state bit in the global state word can be updated only after confirming that the text data processing result has been successfully written to the intermediate data storage area or result cache. In other words, the state bit corresponding to the agent is only set to the completed state when the text data processing result corresponding to the agent can be invoked by subsequent dependent tasks. This approach avoids situations where the state bit is set but the corresponding historical intermediate data has not yet been successfully stored, thus ensuring that subsequent dependent tasks can normally retrieve the corresponding input data after determining that the preceding dependencies are satisfied based on the state bit.

[0097] In other embodiments, when multiple agents or task threads may concurrently update the global state word corresponding to the same text dataset to be processed, concurrency control permissions corresponding to the dataset identifier can be acquired before updating the state bit, and the concurrency control permissions can be released after the state bit update is completed. This approach avoids state bit overwriting, state loss, or abnormal state update order caused by concurrent writes, ensuring the consistency of state records for the same text dataset in the multi-agent processing chain.

[0098] In this way, after each agent completes its data processing task, the system can consolidate its output into reusable input data for subsequent dependent tasks and synchronously update the corresponding processing state bit in the global state word. This allows for a continuous data flow loop in the multi-agent processing stages, enabling subsequent agents to directly continue execution based on historical intermediate data generated by preceding agents without repeating completed tasks. This allows for rapid subsequent dependency judgment and task scheduling based on the updated processing state bitmap, thereby improving the efficiency of intermediate result reuse, the accuracy of state tracking, and the stability of task chain execution in multi-agent text data processing.

[0099] In a preferred embodiment, where the data processing task includes multiple subtasks, the method further includes:

[0100] Step A101: Parse the data processing task to identify the target processing result, and determine the set of subtasks corresponding to the data processing task, the dependencies between each subtask, and the target completion status mask from the preset task configuration table based on the target processing result. The status bits set in the target completion status mask correspond to the multiple subtask completion statuses that need to be satisfied to generate the target processing result.

[0101] In one specific embodiment, when the received data processing task is a complex task, the task description, task type field, or target output parameters of the data processing task can be parsed first to identify the target processing result required by the data processing task. The target processing result may include summary results, keyword results, event extraction results, structured text results, or a comprehensive result formed by combining multiple processing results. For example, if the user submits a task of "summarizing and extracting events from the text dataset to be processed," the target processing result can be identified as "summarizing results and event extraction results"; as another example, if the user submits a task of "segmenting, summarizing, and generating an event table from a long document," the target processing result can be identified as including at least a text block sequence, summary results, and structured event results.

[0102] After identifying the target processing result, the corresponding subtask set, dependencies between subtasks, and target completion state mask can be retrieved from the preset task configuration table based on the target processing result. The subtask set represents multiple subtasks that need to be executed or meet completion states to generate the target processing result, such as text cleaning subtasks, text segmentation subtasks, text summarization subtasks, and event extraction subtasks. Dependencies represent the execution constraints between subtasks; for example, the text segmentation subtask depends on the text cleaning subtask, the text summarization subtask depends on the text segmentation subtask, and the event extraction subtask depends on the text summarization subtask. The target completion state mask represents the completion states of multiple subtasks required to generate the target processing result in a bitmap format. The set state bits in the target completion state mask and the processing state bitmap use the same agent state bit mapping relationship.

[0103] For example, suppose the text cleaning agent, text segmentation agent, text summarization agent, and event extraction agent correspond to state bits 0, 1, 2, and 3, respectively. If the target processing result is an event extraction result, and the task configuration table specifies that generating this event extraction result requires sequentially completing text cleaning, text segmentation, text summarization, and event extraction, then the target completion state mask can set all three state bits (0 to 3). This target completion state mask does not represent the currently completed state, but rather the completion states of each subtask that must be satisfied to generate the target processing result.

[0104] Step A102: Perform a bitwise AND operation on the processing state bitmap based on the target completion state mask to obtain the completed state bitmap.

[0105] In one specific embodiment, the processing state bitmap corresponding to the current text dataset to be processed can be read from the second bit field of the global state word. This processing state bitmap represents the actual processing completion status of multiple agents on the current text dataset. Since the processing state bitmap may include the completion states of other agents unrelated to the current target processing result, a bitwise AND operation can be performed on the processing state bitmap based on the target completion state mask to filter out the completed state bits related to the current target processing result, thus obtaining the completed state bitmap.

[0106] For example, the target completion state mask corresponds to four state bits: cleaning, segmentation, summarization, and event extraction. The current processing state bitmap, besides recording the completion of cleaning and segmentation, also records the completion of the keyword extraction agent. Since the keyword extraction state is not necessary for generating the current event extraction result, the system performs a bitwise AND operation between the target completion state mask and the processing state bitmap to mask out keyword extraction states irrelevant to the current target processing result, retaining only the completed state bits in the target processing chain. Therefore, the subsequent judgment of missing state bits will not be interfered with by irrelevant completed state bits.

[0107] Step A103: Perform a bitwise XOR operation between the completed state bitmap and the target completed state mask to determine the missing state bits to be filled, and determine the subtasks to be executed from the subtask set based on the missing state bits.

[0108] In one specific embodiment, a bitwise XOR operation can be performed between the completed state bitmap and the target completed state mask. Since the completed state bitmap has already been filtered by the target completed state mask, its state bit range is limited to the set of states required to generate the target processing result. Under this premise, the bitwise XOR operation can be used to determine the state bits in the target completed state mask that have not yet been satisfied by the current processing state bitmap, i.e., the missing state bits to be filled.

[0109] For example, if the target completion state mask is 1111, it means that generating the target processing result requires completing four sub-tasks: cleaning, segmentation, summarization, and event extraction. The current processing state bitmap, after being filtered by the target completion state mask, results in a completed state bitmap of 0011, indicating that cleaning and segmentation are complete. At this point, performing a bitwise XOR operation between 0011 and 1111 yields 1100, indicating that the state bits corresponding to summarization and event extraction are not yet complete. Based on the set state bits in the missing state bit 1100, the mapping relationship between state bits and agent identifiers can be queried, and the sub-tasks to be executed from the sub-task set can be determined as the text summarization sub-task and the event extraction sub-task.

[0110] In this way, instead of simply re-executing all subtasks according to a fixed process, only the subtasks that have not yet been completed and are related to the target processing result are determined based on the actual processing status of the current text dataset, thereby realizing the dynamic reorganization of complex tasks in the current state.

[0111] Step A104: Based on the agent identifier corresponding to the subtask to be executed, extract at least one candidate execution path from the preset agent topology graph that covers the missing state bits and satisfies the dependency relationship.

[0112] In one specific embodiment, an agent topology graph can be maintained in advance. The agent topology graph is used to represent the executable sequential relationships, data input-output relationships, or dependencies between different agent tasks. Nodes in the topology graph can represent agents or agent tasks, and edges can indicate that the output of one agent task can be used as input data for another agent task, or that there is a prerequisite dependency between two agent tasks.

[0113] After identifying the subtasks to be executed, at least one candidate execution path can be extracted from the agent topology graph based on the agent identifiers corresponding to the subtasks. During the extraction process, the subtasks corresponding to missing state bits are used as path coverage objects, and the dependencies between subtasks are used as path ordering constraints. This ensures that the candidate execution paths include agent tasks used to fill in the missing state bits, and that the execution order of the agent tasks conforms to the dependencies. For example, if the missing state bits correspond to a text summary subtask and an event extraction subtask, and the event extraction subtask depends on the text summary subtask, then the candidate execution paths must include at least a summary agent and an event extraction agent, with the summary agent preceding the event extraction agent.

[0114] Step A105: Based on at least one of the following factors, such as the number of agents to be executed corresponding to each candidate execution path, the reuse of historical intermediate data, and the dependency level, evaluate the path cost of each candidate execution path, and determine the candidate execution path whose path cost meets the preset conditions as the target execution path.

[0115] In one specific embodiment, after obtaining at least one candidate execution path, path cost evaluation can be performed on each candidate execution path. Path cost evaluation can be determined based on at least one of the following: the number of agents to be executed, the reuse of historical intermediate data, and the dependency level. Specifically, the number of agents to be executed can represent the number of agent tasks that need to be actually triggered in the candidate execution path; the reuse of historical intermediate data can represent the amount of generated intermediate data that can be reused in the candidate execution path, the data integrity, or the reuse ratio; and the dependency level can represent the depth of the task chain or the number of preceding and following dependency layers in the candidate execution path.

[0116] For example, if candidate execution path one requires three agent tasks and can only reuse the cleaning results, while candidate execution path two only requires two agent tasks and can reuse the already generated segmentation and summarization results, then candidate execution path two is considered to have a lower path cost. Furthermore, when the number of agents to be executed is the same for two candidate execution paths, the candidate execution path with higher historical intermediate data reuse, shorter dependency levels, or lower expected execution overhead can be preferentially selected as the target execution path.

[0117] In some embodiments, path cost can be calculated using a preset evaluation function. For example, the path cost can be calculated based on factors such as the number of agents to be executed, the amount of unreused intermediate data, the dependency level, the expected number of model calls, or the expected data read overhead. A path cost meeting preset conditions can include having the minimum path cost, a path cost below a preset threshold, or ranking highest among multiple candidate execution paths. In this way, the selection of the optimal execution path can be transformed into a computable path cost evaluation process, thereby avoiding the need to determine the execution path solely based on fixed task templates or human experience.

[0118] Step A106: According to the target execution path and dependencies, insert the tasks of each agent corresponding to the target execution path into the task queue, so as to trigger the sequential execution of each agent task based on the task queue.

[0119] After determining the target execution path, the agent tasks corresponding to the target execution path can be inserted into the task queue according to the target execution path and the dependencies between its subtasks. Specifically, task items can be generated for each agent task in the target execution path, and these task items can be written into the task queue in the execution order determined by the target execution path. Each task item can contain information such as dataset identifier, agent identifier, task state, input data reference, processing location identifier, user instruction set, and result cache, so that subsequent execution scheduling can be based on the task queue.

[0120] For example, if the target execution path is "summarize agent → event extraction agent", then a summary task item can be generated first and inserted into the task queue, followed by an event extraction task item and inserted into the task queue. Once the summary task item completes execution and generates a summary result, this summary result can be used as input data for the event extraction task item, and the event extraction agent can then perform event extraction based on this input data. If the target execution path is "segmentation agent → summarize agent → event extraction agent", then the segmentation task item, summary task item, and event extraction task item can be inserted into the task queue sequentially, and the corresponding agents can be triggered to execute them sequentially through the task queue.

[0121] In the solution provided in this embodiment, through steps A101 to A106, complex data processing tasks can be decomposed into multiple subtasks with dependencies according to the target processing result. Completed subtasks and those to be completed are identified based on the current processing state bitmap, thereby dynamically reorganizing the task chain. Simultaneously, candidate execution paths are generated through the agent topology graph, and path cost is evaluated by combining the number of agents to be executed, historical intermediate data reuse, and dependency levels, allowing for the selection of a target execution path more suitable for the current data processing state. Therefore, this embodiment supports the dynamic decomposition and reorganization of complex tasks. While ensuring the correctness of subtask dependencies, it prioritizes the reuse of generated historical intermediate data, reducing unnecessary repetitive execution of subtasks and large model calls, thereby improving the flexibility of complex task orchestration, the reuse rate of historical intermediate results, and the efficiency of multi-agent collaborative processing.

[0122] In summary, this embodiment provides a multi-agent text data processing method based on bitmap state tracking. By constructing a global state word for the text dataset to be processed, including a dataset identifier and a processing state bitmap, the processing completion status of multiple agents on the same text dataset is uniformly encoded in the form of state bits. This eliminates the need to query multiple task records or multiple state tables separately, allowing the current processing status of the text dataset in the multi-agent processing chain to be quickly determined through the global state word. Furthermore, by performing a bitwise AND operation between the processing state bitmap and the dependency mask, it is possible to directly determine whether the prerequisite processing states required for the target agent to execute the data processing task have been met. When the prerequisite dependencies are met, historical intermediate data can be directly retrieved as input data for the target agent, avoiding the repeated execution of completed prerequisite tasks. When the prerequisite dependencies are not met, the corresponding prerequisite dependency tasks can be determined based on the missing prerequisite processing states and inserted into the task queue according to the dependency order to trigger execution, thereby ensuring that the multi-agent task chain is completed and executed according to the correct dependency relationship. Therefore, this application can achieve low-overhead state tracking, rapid dependency determination, reuse of historical intermediate results, and automatic scheduling of missing pre-tasks in large-scale text data processing scenarios, reducing the cost of repetitive calculations and large model calls, and improving the scheduling efficiency, resource utilization, and execution stability of multi-agent text data processing.

[0123] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0124] Furthermore, some embodiments of this application also provide an electronic device. The electronic device includes: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform a multi-agent text data processing method based on bitmap state tracking as provided in any one or more of the foregoing embodiments. Figure 3An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0125] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103, and output device 1104 may be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0126] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0127] In this embodiment, a computer-readable medium stores a computer program / instruction, which, when executed by a processor, implements the multi-agent text data processing method based on bitmap state tracking provided in any one or more of the above embodiments. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more computer-readable instructions.

[0128] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0129] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0130] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0131] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0132] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. Terms such as "first," "second," etc., are used only to distinguish descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0133] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A multi-agent text data processing method based on bitmap state tracking, characterized in that, include: Obtain the text dataset to be processed and determine the dataset identifier corresponding to the text dataset to be processed; Construct a global state word, which includes a first segment and a second segment. The first segment is used to store the dataset identifier, and the second segment is used to store a bitmap of the processing state of multiple agents on the text dataset to be processed. Each state bit in the processing state bitmap corresponds to a preset multiple agent identifiers. Obtain the data processing task for the text dataset to be processed, and determine the target agent and its corresponding dependency mask for the data processing task. The state bits set in the dependency mask indicate the pre-processing state that the target agent needs to satisfy before executing the data processing task. Perform a bitwise AND operation between the processing state bitmap and the dependency mask. If the result matches the dependency mask, retrieve the historical intermediate data generated by the preceding dependency task and use it as the input data for the target agent. If the result does not match the dependency mask, identify the missing preceding dependency task and insert it into the task queue according to the dependency order to trigger execution. After any agent completes the corresponding data processing task and generates the corresponding text data processing result, the text data processing result is stored as input data for subsequent dependent tasks, and the corresponding state bit in the second bit field of the global state word is updated based on the agent identifier corresponding to the agent that completed the data processing task.

2. The multi-agent text data processing method based on bitmap state tracking according to claim 1, characterized in that, The global state word is an unsigned integer variable with the same bit width as the processor register; the dataset identifier in the first bit segment is written or read through the mask corresponding to the first bit segment and the bit shift operation; the processing state bitmap in the second bit segment is read through the mask corresponding to the second bit segment, and is matched and judged by the dependency mask through bitwise AND operation, and the state bit of the corresponding agent identifier is updated through bitwise OR operation; wherein, the bit shift operation, the mask corresponding to the first bit segment, the mask corresponding to the second bit segment, the bitwise AND operation and the bitwise OR operation are all executed based on the bit operation instructions supported by the processor.

3. The multi-agent text data processing method based on bitmap state tracking according to claim 1, characterized in that, The step of determining the missing prerequisite tasks and inserting them into the task queue to trigger execution in order of dependency order when the calculation result is inconsistent with the dependency mask includes: Based on the difference state bits between the dependency mask and the operation result, the unmet pre-processing state is determined, and the missing pre-dependent task is determined based on the agent identifier corresponding to the unmet pre-processing state. Based on the missing prerequisite tasks, corresponding task items are generated, and the task items are inserted into the task queue according to the dependency order among the missing prerequisite tasks. The task status of the task item inserted into the task queue is set to the pending execution state, wherein the task queue is configured with a position cursor, and the position cursor is used to locate the pending task item in the task queue; When the cursor points to the task item, the task item is determined as the current task item, and the task context is loaded based on the current task item. The agent corresponding to the current task item is then scheduled to execute the corresponding pre-dependent task.

4. The multi-agent text data processing method based on bitmap state tracking according to claim 3, characterized in that, The task item includes a task identifier, a dataset identifier, an agent identifier, a task state, an input data reference, a processing position identifier, a set of user instructions, and a result cache. The dataset identifier is used to index the global state word corresponding to the text dataset to be processed. The agent identifier is used to determine the corresponding state bit in the processing state bitmap. The processing position identifier is used to record the processing breakpoint when the corresponding agent processes the text dataset to be processed. The task context includes the input data to be input based on the input data reference, the processing start position determined based on the processing position identifier, the agent execution constraints determined based on the set of user instructions, and the reusable intermediate processing results determined based on the result cache.

5. The multi-agent text data processing method based on bitmap state tracking according to claim 3, characterized in that, include: When the task status of the current task item is in a pending debugging state, a debugging instruction for the current task item is received, and at least one of the following processes is performed according to the debugging instruction: When the debugging instruction is a task backtracking instruction, the position cursor is adjusted to the task item before the current task item, and the task context is reloaded based on the adjusted task item; When the debugging instruction is a task forward instruction, the position cursor is adjusted to the task item after the current task item in order to skip the current task item or enter the subsequent task item; When the debugging instruction is a user feedback instruction, the user correction information is written into the user instruction set of the current task item, and the agent corresponding to the current task item is rescheduled to perform the data processing task based on the updated user instruction set. When the debugging instruction is a rule solidification instruction, the user correction information that meets the preset debugging success conditions is converted into the corresponding intelligent agent's processing rules and written into the rule set.

6. The multi-agent text data processing method based on bitmap state tracking according to claim 5, characterized in that, include: The agent corresponding to the current task item is a segmentation agent; During the process of the segmentation agent performing text segmentation processing on the text dataset to be processed based on the current segmentation rules, If a splitting discrepancy or splitting anomaly is detected, the execution of the current task item is paused, and the task status of the current task item is set to a pending debugging state. Upon receiving the user feedback instruction, the user correction information is written into the user instruction set of the current task item, and the segmentation point of the segmentation agent is corrected based on the user correction information. Upon receiving the rule solidification instruction, the user correction information that meets the preset debugging success conditions is converted into a natural language segmentation instruction or a regular expression rule, and the natural language segmentation instruction or regular expression rule is written into the rule set corresponding to the segmentation agent, so that the segmentation agent can load the updated rule set and perform text segmentation processing in subsequent text segmentation tasks.

7. The multi-agent text data processing method based on bitmap state tracking according to claim 1, characterized in that, In cases where the data processing task comprises multiple subtasks, it further includes: The data processing task is parsed to identify the target processing result, and the set of subtasks corresponding to the data processing task, the dependencies between the subtasks, and the target completion status mask are determined from the preset task configuration table based on the target processing result. The status bits set in the target completion status mask correspond to the multiple subtask completion statuses that need to be satisfied to generate the target processing result. Based on the target completion state mask, perform a bitwise AND operation on the processing state bitmap to obtain the completed state bitmap; Perform a bitwise XOR operation between the completed state bitmap and the target completed state mask to determine the missing state bits to be filled, and determine the subtasks to be executed from the subtask set based on the missing state bits. Based on the agent identifier corresponding to the subtask to be executed, at least one candidate execution path that covers the missing state bit and satisfies the dependency relationship is extracted from the preset agent topology graph; Based on at least one of the following factors, namely the number of agents to be executed corresponding to each candidate execution path, the reuse of historical intermediate data, and the dependency level, the path cost of each candidate execution path is evaluated, and the candidate execution path whose path cost meets the preset conditions is determined as the target execution path. According to the target execution path and the dependency relationship, each agent task corresponding to the target execution path is inserted into the task queue, so as to trigger the sequential execution of each agent task based on the task queue.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; and a memory storing computer program instructions, which, when executed, cause the processors to perform the multi-agent text data processing method based on bitmap state tracking as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program and / or instructions stored thereon, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the multi-agent text data processing method based on bitmap state tracking as described in any one of claims 1-7.

10. A computer program product comprising a computer program and / or instructions, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the multi-agent text data processing method based on bitmap state tracking as described in any one of claims 1-7.

Images