A document task processing method and device based on multi-level cache

Abstract

This invention discloses a document task processing method and apparatus based on multi-level caching, comprising: receiving a document task to be processed and obtaining task association information and task input data of the document task to be processed; performing normalization preprocessing on the task input data, and determining at least one task processing unit and unit input data and unit association information of the task processing unit based on the preprocessing result and task association information; generating a cache query key based on the unit input data and unit association information; performing cache matching in multi-level cache based on the cache query key; if cache matching fails, performing service scheduling based on the unit association information and unit input data, and calling a target external service to execute the document task to be processed after service circuit breaker detection, and returning the execution result to the requester of the document task to be processed. This invention can avoid the cache failure problem caused by different document formats and can be widely applied in the field of computer document processing technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer document processing, and in particular to a document task processing method and device based on multi-level caching. BACKGROUND

[0002] An artificial intelligence (AI) document processing system relies on external services such as optical character recognition (OCR), vision language model (VLM), large language model (LLM), embedding, reranker, vector database, graph database, and file conversion service to work together to complete processing tasks such as document parsing, content understanding, and structured output.

[0003] The prior art usually uses a single cache layer with simple key-value matching to perform cache queries. The task input data or its hash value is directly used as a cache key to search for existing processing results in the cache.

[0004] However, the prior art has the following defects: generating a cache key directly based on the user-submitted document data to be processed makes it impossible to hit the existing cache due to different cache keys generated by the same document content due to format differences, resulting in repeated calls to high-cost external services. SUMMARY

[0005] The present application provides a document task processing method and device based on multi-level caching, which solves the technical problem of cache invalidation caused by inconsistent cache keys due to document format differences.

[0006] In a first aspect, the present application provides a document task processing method based on multi-level caching, comprising:

[0007] Receiving a document task to be processed and obtaining task association information and task input data of the document task to be processed;

[0008] Normalizing the task input data and determining at least one task processing unit, unit input data, and unit association information of the task processing unit based on the pre-processing result and the task association information;

[0009] Generating a cache query key based on the unit input data and the unit association information;

[0010] Performing cache matching in the multi-level cache based on the cache query key, the unit association information, and the unit input data.

[0011] If the cache matching fails, service scheduling is performed based on the unit association information and the unit input data. After service circuit breaker detection, the target external service is called to execute the document task to be processed.

[0012] Receive the execution result returned by the target external service and return it to the requester of the document task to be processed.

[0013] In some embodiments, the step of performing normalization preprocessing on the task input data and determining at least one task processing unit and the unit input data and unit association information of the task processing unit based on the preprocessing result and the task association information includes:

[0014] The task input data is processed for format standardization and redundant information filtering to generate standardized data;

[0015] Extract the process template corresponding to the target ontology class from the task association information;

[0016] According to the preset task decomposition strategy, the process template is instantiated and parsed to generate multiple task processing units and obtain the unit association information of each task processing unit.

[0017] Extract the data selection range and task type from the unit association information;

[0018] Based on the data selection range, the standardized data is split and allocated to obtain the unit input data for each task processing unit.

[0019] In some embodiments, generating a cache query key based on the unit input data and the unit association information includes:

[0020] Extract the task context information and task type from the unit association information;

[0021] Content hash digests are extracted from the non-text content in the unit input data to generate content digest data for each task processing unit.

[0022] According to the field selection strategy associated with the task type, target fields are extracted from the unit input data and the content summary data and concatenated in order to generate the content feature identifier of each task processing unit;

[0023] The target cache namespace that matches the task type;

[0024] The target cache namespace, the content feature identifier, and the task context information are encoded and combined to generate cache query keys for each task processing unit.

[0025] In some embodiments, the step of performing cache matching in a multi-level cache based on the cache query key, combined with the unit association information and the unit input data, includes:

[0026] Based on the cache query key, perform key matching on each pre-stored cache query key in the multi-level cache;

[0027] If no pre-stored cache query key is matched, the cache match is considered unsuccessful.

[0028] If a pre-stored cache query key is matched, the target cache record data bound to the pre-stored cache query key is retrieved from the multi-level cache.

[0029] Extract the cache validity termination time and data validity flag status from the target cache record data;

[0030] Compare the effective termination time of the cache with the current time;

[0031] According to the preset cache validity verification rules, the unit association information and the target data items in the unit input data are extracted respectively, and each target data item is compared with the corresponding item in the target cache record data;

[0032] If the data validity flag status is invalid, the current time exceeds the cache validity termination time, or any of the target data items do not match, then the cache matching is determined to be unsuccessful.

[0033] In some embodiments, it also includes:

[0034] If the data validity flag is set to "data valid", the current time has not exceeded the cache validity termination time, and all target data items match, then the cache matching is deemed successful.

[0035] If the cache match is successful, the target execution result is extracted from the target cache record data and returned to the requester of the document task to be processed.

[0036] In some embodiments, the step of performing service scheduling based on the unit association information and the unit input data, and invoking a target external service to execute the document task after service circuit breaker detection, includes:

[0037] Extract the task type from the unit association information;

[0038] Match the target external service of the task type and obtain the call status of the target external service; the call status includes the number of consecutive call failures and the call timeout ratio;

[0039] If the number of consecutive failed calls does not exceed a preset threshold and the call timeout ratio does not exceed a preset ratio threshold, then the circuit breaker detection of the target external service is determined to be passed, and the circuit breaker of the target external service is set to the closed state.

[0040] If the fuse is in the closed state, a call request is generated based on the unit association information and the unit input data;

[0041] The call request is sent to the target external service according to the scheduling priority.

[0042] In some embodiments, it also includes:

[0043] If the number of consecutive failed calls exceeds a preset threshold, or the call timeout ratio exceeds a preset ratio threshold, then the circuit breaker detection of the target external service is determined to have failed, the circuit breaker of the target external service is set to the open state and a cooling-off period is initiated.

[0044] If the fuse is in the open state, the call to the target external service is refused and local degradation processing is performed based on the unit association information and the cache query key until the cooling time ends;

[0045] If the cooling time ends, the fuse is switched to a half-open state and a preset number of probe requests are sent to the target external service, and the probe feedback status corresponding to the probe requests is returned by the target external service.

[0046] If the detection feedback status meets the preset service availability conditions, the fuse is switched to the closed state and the process jumps to the step of generating a call request based on the unit association information and the unit input data;

[0047] If the detection feedback status does not meet the preset service availability conditions, the fuse will be switched to the open state and the cooling time will be restarted.

[0048] In some embodiments, it also includes:

[0049] Based on the execution result and the unit association information, cache record data is generated;

[0050] The cached record data is bound to the cache query key and then written into the multi-level cache.

[0051] In some embodiments, generating cache record data based on the execution result and the unit association information includes:

[0052] The execution result is used as a cached value;

[0053] According to the preset metadata extraction rules, the cache verification field data and task type are extracted from the unit association information;

[0054] The target cache validity period matches the task type;

[0055] Add the current time to the target cache validity period to generate the target cache termination time;

[0056] Based on the cached value, the cache verification field data, and the target cache termination time, cache record data is generated.

[0057] Secondly, the present invention provides a document task processing device based on multi-level caching, comprising:

[0058] The task receiving module is used to receive the document task to be processed and to obtain the task association information and task input data of the document task to be processed.

[0059] The unit splitting module is used to perform normalized preprocessing on the task input data, and determine at least one task processing unit and the unit input data and unit association information of the task processing unit based on the preprocessing results and the task association information.

[0060] A key generation module is used to generate a cache query key based on the unit input data and the unit association information;

[0061] The cache matching module is used to perform cache matching in a multi-level cache based on the cache query key, combined with the unit association information and the unit input data;

[0062] The service scheduling module is used to perform service scheduling based on the unit association information and the unit input data if the cache matching fails, and to call the target external service to execute the document task to be processed after service circuit breaker detection.

[0063] The result return module is used to receive the execution result returned by the target external service and return it to the requester of the document task to be processed.

[0064] As can be seen from the above technical solutions, the present invention has the following advantages: The document task processing method and apparatus based on multi-level caching provided by the present invention can completely capture the task context and original input materials by receiving the document task to be processed and obtaining task association information and task input data, providing basic data for subsequent caching and scheduling; it performs standardized preprocessing on the task input data, and determines at least one task processing unit and corresponding unit input data and unit association information by combining the preprocessing results and task association information, decomposing the overall document task into independent task processing units, separating the input and association configuration required by each unit, and realizing granular processing of complex document tasks; based on unit input... The system generates cache query keys by combining input data with unit association information, eliminating identification discrepancies caused by document format differences. This ensures consistent cache query keys for requests with the same substantive business, improving the stability of cache key generation. Based on the cache query keys, multi-level cache matching is performed using unit association information and unit input data to increase the probability of effective cache hits and reduce unnecessary external requests. When a cache miss occurs, service scheduling is performed based on unit association information and unit input data. After service circuit breaker detection, the target external service is called, providing protection and control for external service calls in cache failure scenarios. The system receives the execution result returned by the target external service and returns it to the requester, completing the business output of the document task and ensuring a closed-loop business process. This invention avoids cache failure caused by different document formats and reduces repeated calls to high-cost external services through standardized preprocessing combined with a task processing unit-level cache construction method. Simultaneously, the accompanying scheduling and circuit breaker detection mechanisms enhance the overall operational stability of the document task processing process while improving cache reuse efficiency. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A flowchart illustrating a document task processing method based on multi-level caching, provided as an embodiment of the present invention;

[0067] Figure 2 A schematic diagram of the structure of a document task processing system based on multi-level caching provided in an embodiment of the present invention;

[0068] Figure 3 Another flowchart of a document task processing method based on multi-level caching provided in an embodiment of the present invention;

[0069] Figure 4Another flowchart of a document task processing method based on multi-level caching provided in an embodiment of the present invention;

[0070] Figure 5 This is a schematic diagram of the evidence constraint verification and traceable output processing flow in an embodiment of the present invention;

[0071] Figure 6 A schematic diagram of a document task processing device based on multi-level caching provided in an embodiment of the present invention;

[0072] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0073] This invention provides a document task processing method and apparatus based on multi-level caching to solve the technical problem of cache invalidation caused by inconsistent cache keys due to differences in document formats.

[0074] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0075] Please see Figure 1 , Figure 1 An optional flowchart of a document task processing method based on multi-level caching provided in an embodiment of the present invention includes steps 101 to 106.

[0076] Step 101: Receive the document task to be processed and obtain the task association information and task input data of the document task to be processed;

[0077] Unprocessed document tasks refer to computational tasks that perform parsing, retrieval, generation, or export operations on document resources, covering business scenarios such as document parsing, knowledge retrieval, intelligent writing, AI completion, and export preview.

[0078] Task association information refers to the set of attributes and context parameters bound to a task, which are used for subsequent service routing, cache matching, and priority scheduling. This includes task type, task body, priority identifier, tenant / project scope, model service version, etc.

[0079] Task input data refers to the original business data that supports task execution, such as documents to be parsed, query text, template configuration files, etc., which are the basic data sources for generating content feature identifiers.

[0080] The task ontology refers to the task attributes (such as target ontology class, slot type, evidence type, etc.) defined from the dimension of semantic constraints, which are used to limit the semantic boundaries of task output and the scope of cache reuse.

[0081] A slot refers to a structured output unit (such as field identifier, output format, and confidence threshold) generated by the template structure and ontology constraints, used to control the structure and compliance of the output content.

[0082] Foreground interactive tasks refer to blocking tasks that users wait in real time (such as AI completion in the editor, word rewriting, and single knowledge retrieval), and have a higher scheduling priority than background batch tasks.

[0083] Background batch tasks refer to asynchronous, non-blocking tasks (such as batch document parsing, batch vectorization, knowledge storage, etc.), which have a lower scheduling priority than foreground interactive tasks.

[0084] A task identifier is a globally unique identification code assigned to a single document task to be processed. It is used to link task association information, task input data and execution status throughout the entire process.

[0085] The task context storage component refers to the storage medium used to persistently store all task attributes and input data, allowing for reading and reuse at each stage of the process, and ensuring the consistency of task data.

[0086] Optionally, step 101 is used for task access verification and context initialization, completing legality verification, attribute parsing, input loading, and data persistence, providing a unified data foundation for subsequent end-to-end processing. The specific execution flow is as follows:

[0087] First, perform task access verification and identifier allocation: receive document processing task requests through open interfaces and front-end interaction endpoints, and perform legality verification of user identity and permissions and the integrity of request parameters; after the verification is passed, assign a globally unique task identifier to this task as a unique index for the entire data association;

[0088] Secondly, the task association information extraction and priority determination are completed: All attribute parameters are parsed from the request message, preset task template configuration, and tenant permission configuration to extract task type, task body parameters, model service version constraints, tenant / project scope, business tags, etc., and integrated to form task association information. Simultaneously, based on the request source page, interface endpoint, session identifier, user behavior identifier, interface blocking status, and front-end polling status, combined with the task priority field and waiting timeout configuration, it is determined whether the task belongs to the front-end interactive type or the back-end batch type, mapping the corresponding scheduling priority, and storing it in the task association information.

[0089] Next, load and validate the task input data: retrieve the original business data corresponding to this task, including the document file to be processed, query text, slot template configuration file, etc., as the task input data; simultaneously validate the format, legality and completeness of the input data, and exclude invalid inputs with abnormal format or missing content;

[0090] Finally, the task context is bound and persisted. The parsed task association information and the loaded task input data are uniformly associated with the globally unique task identifier of this task and stored in the task context storage component for subsequent input standardization preprocessing, cache query, and service scheduling to read and reuse throughout the entire process.

[0091] Step 102: Perform normalization preprocessing on the task input data, and determine at least one task processing unit and the unit input data and unit association information of the task processing unit based on the preprocessing results and task association information;

[0092] Standardized preprocessing refers to the noise cleaning and format unification operations performed on the task input data. It includes two core steps: redundant information filtering and format standardization processing. It is used to eliminate the format differences of heterogeneous data and ensure that the same substantive content has a consistent data representation.

[0093] Redundant information filtering refers to a sub-step of normalized preprocessing, used to remove noise data from the task input data that is irrelevant to the core business content (examples include document headers and footers, invalid watermarks, redundant whitespace characters, and irrelevant decorative elements).

[0094] Format standardization refers to the sub-step of standardization preprocessing, which is used to unify the format rules of multi-source heterogeneous data (examples include unifying whitespace characters, character case and full-width / half-width characters, unifying units of measurement, and sorting of structured data fields) to eliminate representational biases caused by format differences.

[0095] Standardized data refers to the standard format data output after the task input data has undergone standardized preprocessing. It serves as a unified data source for task processing unit splitting, feature generation, and business calculation.

[0096] A process template refers to a static rule configuration file bound to the target ontology class. It defines the splitting logic, attribute configuration, and execution order of task processing units in the corresponding business scenario. It only carries general rules and does not contain the actual business data of a single task.

[0097] A task processing unit refers to the smallest independent execution granularity after the document task to be processed is broken down (examples include semantic parsing unit, text generation unit, and compliance verification unit), which can independently complete the entire process of cache matching, service scheduling, and external calls.

[0098] Unit association information refers to the set of attribute metadata corresponding to a single task processing unit. It inherits the common attributes of task association information and supplements the configuration rules specific to the unit (examples include data selection range, unit task type, and processing priority).

[0099] The data selection range refers to the data positioning rules recorded in the unit association information, which are used to delineate the data boundaries and data types required for the corresponding task processing unit from the normalized data.

[0100] Unit input data refers to the target data fragments that are split and allocated from normalized data and are specifically used by a single task processing unit. They serve as the direct input carrier for the unit's business execution.

[0101] SHA-256 (Secure Hash Algorithm 256-bit) is a cryptographic hash function that can map input data of any length to a 256-bit fixed-length hash value. It has strong collision resistance and is used to generate stable content feature identifiers.

[0102] Embedding refers to the process of converting text content into high-dimensional semantic vectors, which are used in semantic retrieval and similarity calculation scenarios.

[0103] The reranker task refers to the process of performing semantically precise ranking on the initially screened candidate set to improve the matching accuracy of the search results.

[0104] The Vision-Large Language Model (VLM) refers to a multimodal large model that integrates image understanding and language capabilities, used for parsing document images and layout content.

[0105] A PDF conversion task (Portable Document Format conversion task) refers to calling a document conversion service to complete processes such as converting source files to PDF format, parsing PDF files, and extracting PDF content.

[0106] The LLM generation task (Large Language Model) refers to calling the large language model service to output structured or unstructured text content based on prompt templates, input text fragments, ontology constraints, and slot configurations.

[0107] Optionally, step 102 includes the following sub-steps:

[0108] The task input data is processed for format standardization and redundant information filtering to generate standardized data;

[0109] Extract the process template corresponding to the target ontology class from the task association information;

[0110] According to the preset task decomposition strategy, the process template is instantiated and parsed to generate multiple task processing units and obtain the unit association information of each task processing unit.

[0111] Extract the data selection range and task type from the unit association information;

[0112] Based on the data selection range, the normalized data is split and allocated to obtain the unit input data for each task processing unit.

[0113] Optionally, step 102 first completes the standardization and cleaning of the task input data, then combines the task association information to complete the task granularity decomposition, and outputs independently executable task processing units and corresponding supporting data. Here, the document task to be processed is the upper-level overall task, and the task processing unit is a subtask decomposed from this overall task. Each task processing unit's task type is any one of the following: Embedding (vector embedding) task, Reranker (re-ranking) task, PDF conversion task, LLM (Large Language Model) generation task, or VLM (Vision Large Model) task. Each task processing unit can independently complete the stable fingerprint generation, cache key construction, cache matching, and external service call process. The specific execution process is as follows:

[0114] The task input data undergoes format standardization and redundant information filtering to generate standardized data: For text and structured data in the task input data, redundant information filtering and format standardization are performed sequentially. In the redundant information filtering stage, noise unrelated to core business content, such as document headers and footers, invalid watermarks, and meaningless blank paragraphs, is identified and removed, and the volume of invalid data is compressed. In the format standardization stage, for text data, blank characters are standardized, character case and full / half-width forms are standardized, and units of measurement are standardized. For JSON (JavaScript Object Notation) structured data, field sorting is performed, and for tabular data, cells are serialized by row and column. By unifying rules, the representational differences between data from different sources and in different formats are eliminated, ultimately outputting standardized data with a unified format and concise content.

[0115] Matching the corresponding process template: Extract the process template corresponding to the target ontology class from the task association information. The target ontology class is the core field in the task association information that identifies the business semantic scenario. Different business scenarios correspond to different task decomposition logic and processing rules. Read the target ontology class field from the task association information, and use this field as an index to match the corresponding process template from the preset template library. The process template and the target ontology class have a one-to-one correspondence, and the configuration and verification are completed in advance. The process template predefines the task type corresponding to each task processing unit after decomposition, namely, one of the following: Embedding task, Reranker task, PDF conversion task, LLM generation task, and VLM task.

[0116] According to the preset task decomposition strategy, the process template is instantiated and parsed to generate multiple task processing units and obtain the unit association information of each task processing unit. The preset task decomposition strategy consists of predefined template instantiation rules used to combine the general process template with the attribute parameters of the overall task of the upper-level document to be processed, generating directly executable task processing units. During the instantiation and parsing process, the corresponding number of task processing units are obtained according to the unit splitting rules and execution order defined in the process template. Simultaneously, unit association information is generated for each task processing unit. This unit association information inherits common attributes (such as tenant identifier, ontology version, model service version, caching strategy, and tags) from the task association information of the upper-level document task and injects unit-specific attributes configured in the process template, including the task type to which the task processing unit belongs, forming a complete set of unit metadata.

[0117] Extracting the data selection range and task type from the unit association information: After generating the task processing unit, read the core configuration fields stored in the unit association information to extract the data selection range and task type corresponding to each task processing unit. The data selection range is used to define the data boundary and data type to be processed by the unit. The task type identifies which type of task processing unit it is: Embedding task, Reranker task, PDF conversion task, LLM generation task, or VLM task. Both provide a configuration basis for subsequent data splitting, fingerprint_input field concatenation, cache matching, and service scheduling.

[0118] Based on the data selection range, the normalized data is split and allocated to obtain the unit input data for each task processing unit: According to the data selection range of each task processing unit, the corresponding data segment is located and extracted from the normalized data, and this data segment is assigned to the corresponding task processing unit as its unit input data. For simple upper-level overall tasks that do not require splitting, only a single task processing unit is generated, and the entire normalized data is used as the unit input data for that unit. Each task processing unit uses its own unit input data and key fields from the unit association information, concatenating them according to its task type to obtain fingerprint_input, then calculating a stable fingerprint using SHA-256, and finally assembling it to obtain the complete cache_key.

[0119] For example, taking the pending document task (upper-level total task) of single-slot AI completion in an enterprise document platform as an example, the execution process of step 102 is illustrated as follows:

[0120] The task input data for this upper-level document processing task includes user-edited project plan document fragments, JSON format slot template configuration files, and ontology constraint rule files. The target ontology class in the task association information of the overall task is "project implementation steps".

[0121] First, perform normalization preprocessing, including:

[0122] In the redundant information filtering stage, the enterprise platform logo in the header of the project solution document fragment, the page number and internal confidentiality statement in the footer are identified and removed, consecutive blank lines and meaningless spaces in the document are deleted, and comment fields and invalid configuration items in the slot template file are removed.

[0123] In the format standardization process, full-width spaces in the document are uniformly replaced with half-width spaces, the capitalization of English technical terms is standardized, and the interchangeable use of kilowatts (kW) and KW is standardized to kW. The JSON-formatted slot template configuration file is sorted in ascending order by field name. The parameter table embedded in the slot template is serialized in row-major order, converting the two-dimensional table into an ordered text sequence. After these two types of processing, standardized data with uniform format and concise content is obtained.

[0124] Subsequently, the target ontology class "Project Implementation Steps" is extracted from the task association information of the upper-level overall task. Using this field as an index, the pre-configured "Project Implementation Steps Single Slot Completion Process Template" is obtained from the template library. This process template defines that the overall task is decomposed into three task processing units (subtasks). Among them, the task type of the task processing unit corresponding to the completion text generation is LLM generation task.

[0125] The process template is instantiated and parsed according to the preset task decomposition strategy, generating three task processing units: context semantic parsing unit, text completion generation unit, and output compliance verification unit. The unit association information corresponding to each unit is generated simultaneously. The unit association information inherits the tenant identifier, ontology version V1.5, large language model service version V4.2, first-level scheduling priority, and other common attributes from the task association information of the upper-level document to be processed total task, and is injected into the unit attributes configured in the process template.

[0126] Next, the data selection scope and task type are extracted from the associated information of each unit: the data selection scope of the context semantic parsing unit is the entire text of the project plan document; the data selection scope of the text completion generation unit is the semantic context fragment and slot template configuration, and the task type is LLM generation task; the data selection scope of the output compliance verification unit is the generated text and ontology constraint rules.

[0127] Finally, based on the data selection range of each unit, the normalized data is split and allocated: the normalized complete project plan text is allocated as unit input data to the context semantic parsing unit; the parsed semantic context fragments and serialized slot template data are allocated as unit input data to the text completion generation unit; and the generated text cache bits to be verified and the normalized ontology constraint rule file are allocated as unit input data to the output compliance verification unit, thus completing the input data allocation for all task processing units.

[0128] The subsequent text completion generation unit (LLM generation task) will concatenate fingerprint_input based on its own unit input data, the prompt template identifier, prompt version, target ontology class, slot type, and model name in the unit association information, obtain a stable fingerprint through SHA-256 operation, and then assemble a complete cache_key by combining the task type, ontology version, target ontology class, label, and model version for multi-level cache lookup.

[0129] Step 103: Generate a cache query key based on the unit input data and unit association information;

[0130] Content feature identifier (also known as stable fingerprint) refers to a fixed-length unique feature string generated based on normalized input content and unit association information. It is the core basis for cache hit judgment and task deduplication.

[0131] Content hash digest extraction refers to the process of generating a fixed-length digest based on the file content bytes for non-text resources such as images and binary files. It is used to characterize the uniqueness of the content of non-text resources and is equivalent to normalized text data.

[0132] Field selection strategy refers to the field combination rules bound to the task type (such as embedding task selection normalized text, model version field), which is used to construct fingerprint input plaintext. Different task types correspond to different field combinations.

[0133] Optionally, step 103 includes the following sub-steps:

[0134] Extract task context information and task type from the unit association information;

[0135] Content hash digests are extracted from the non-text content in the unit input data to generate content digest data for each task processing unit.

[0136] Based on the field selection strategy associated with task type, target fields are extracted from unit association information and content summary data and concatenated in order to generate content feature identifiers for each task processing unit.

[0137] The target cache namespace matches the task type;

[0138] The target cache namespace, content feature identifier, and task context information are encoded and combined to generate cache lookup keys for each task processing unit.

[0139] The cache lookup key (also known as the ontology-aware cache key) is a unique index identifier used to locate cache record data in a multi-level cache system. It integrates content feature identifiers and semantic constraint class task attributes, which can avoid the incorrect reuse of cache results in different semantic task scenarios with the same input content.

[0140] A cache namespace refers to a logical partition of the cache based on task type (such as embedded computing or large language model generation). It is used to isolate cached data of different business types, prevent cache key name conflicts across scenarios, and improve cache retrieval efficiency.

[0141] Least Recently Used (LRU) cache is a memory caching mechanism deployed locally within a process. It sorts cached data according to access time and prioritizes evicting the least recently accessed data to carry frequently accessed cached data and reduce response latency.

[0142] Ontology validation shape refers to the structured validation rule template (such as field requirement constraints, numerical value range, output confidence threshold) bound to the ontology version, used to verify whether the cached results meet the constraints of the current ontology.

[0143] Step 103 is used to generate a unified and stable content feature identifier based on the unit input data and unit association information, providing a reliable retrieval benchmark for subsequent cache matching. The specific execution process is as follows:

[0144] First, perform non-text content summary extraction: For non-text resources such as images and binary files in the unit input data, perform content hash summary extraction, and generate fixed-length content summary data based on the complete byte content of the file. This data serves as an equivalent normalized representation of the non-text resources and together with the normalized text data, constitutes the basic material for subsequent feature generation.

[0145] Next, extract the task type of the task processing unit from the unit association information, retrieve the field selection strategy bound to the task type, and determine the target field range for feature generation.

[0146] Then, the fingerprint input plaintext is constructed: according to the field selection strategy, the corresponding target fields are extracted from the content summary data and unit association information, and concatenated in a preset fixed order to generate the fingerprint input plaintext. For example, the Embedding task selects normalized text, embedded model version, ontology version, and business tag as target fields; the re-ranking task selects query summary, candidate set identifier summary, re-ranking model version, and business tag as target fields; the PDF conversion task selects file content hash, conversion configuration parameters, and conversion service version as target fields; the large language model generation task selects prompt word template identifier, prompt word version, input summary, target ontology class, slot type, and model name as target fields; and the visual large model task selects image content hash, recognition region coordinates, and visual prompt word version as target fields.

[0147] Finally, a content feature identifier is generated: the SHA-256 algorithm is used to perform a hash operation on the plaintext fingerprint input, and a 256-bit fixed-length hash string is output, which is the content feature identifier corresponding to the task processing unit.

[0148] For example, using the aforementioned front-end interactive single-slot AI completion task, one of the task processing units is a large language model generation task, and the unit input data includes a fragment of the project solution document to be completed and a slot template JSON configuration file.

[0149] First, perform normalization preprocessing on the text content of the unit input data: filter out header "XX Enterprise Project Management General Template" and footer page number noise carried in project plan document fragments; convert full-width punctuation and full-width numbers in the text to half-width format, unify multiple irregular line breaks and multiple spaces to standard single spaces, and standardize mixed expressions such as kilowatt and KW to kW; reorder the JSON configuration fields of the slot template in lexicographical order to eliminate field order differences and obtain standardized data with uniform format.

[0150] This unit's input data contains no image or binary file resources, so content hashing and digest extraction is unnecessary. The task type is read as "large language model generation" from the unit's association information, and the corresponding field selection strategy is matched accordingly.

[0151] Subsequently, the fingerprint input plaintext is constructed: according to the field selection strategy, each target field is extracted in sequence, including the prompt word template identifier as slot_completion_v2, the prompt word version as V1.3, the normalized input text summary, the target ontology class as "project implementation steps", the slot type as "step description", and the model name as LLM-V4.2. These fields are then concatenated in a preset fixed order to generate the complete fingerprint input plaintext.

[0152] Finally, the plaintext fingerprint input is fed into the SHA-256 algorithm to perform a hash operation, generating a 256-bit fixed-length hash string, which is the content feature identifier corresponding to this task and is used for the construction of the subsequent cache lookup key.

[0153] Optionally, step 103 is also used to construct an ontology-aware cache index that integrates content features and semantic constraints, ensuring the accuracy of subsequent cache matching and avoiding cache misuse across semantic scenarios. The specific execution process is as follows:

[0154] Extracting core task parameters: Parsing the task type identifier from the unit association information, and extracting task context information, including fields such as model / service version, ontology version, target ontology class, business tag, evidence type, slot type, tenant or project scope;

[0155] Matching cache namespaces: Mapping to the target cache namespace based on task type to achieve logical isolation of cache data for different task types. For example, task types include embedding computation, reordering computation, file conversion, vector retrieval, language model generation, and visual parsing tasks. Different task types correspond to independent cache namespaces.

[0156] The cache query key is generated by encoding and concatenating the following fields: Using the target cache namespace identifier as a prefix, the ontology version, target ontology class, business tag, content feature identifier, and model / service version fields are concatenated sequentially according to a preset fixed field order. After unified encoding and formatting, a cache query key is generated. Specifically, the fields involved in the concatenation can be adapted as needed for different task types. For example, the embedding task uses the content feature identifier and the embedded model version as the core concatenated fields; the re-sorting task additionally concatenates the candidate set summary feature field; the document conversion task additionally concatenates the file content summary, conversion parameters, conversion service version, and other fields; the vector query task additionally concatenates the dataset identifier and filtering conditions, and other fields; the large language model generation task additionally concatenates the slot type and evidence type, and other fields; the visual large model parsing task additionally concatenates the recognition region coordinates and visual cue word version, and other fields.

[0157] The generated cache lookup key is associated with the current task context and passed to subsequent multi-level cache matching stages. This cache lookup key also records the unique identifier of the input content and the business semantic constraints, ensuring that only tasks that match both the input content and the business semantic constraints can hit the same cache entry.

[0158] For example, the aforementioned front-end interactive single-slot AI completion task is used, with the task type being large language model generation. Input normalization processing has been completed and corresponding content feature identifiers have been generated.

[0159] First, extract the core parameters from the unit association information: the task type is LLM generation task, the task context fields include ontology version V1.5, the target ontology class is project implementation steps, the business tag is project management, the evidence type is original text support, the slot type is step description, and the large language model service version is LLM-V4.2.

[0160] Subsequently, the target cache namespace is matched: the LLM generation task is mapped to the language model generation cache namespace (such as llm_completion), which is logically isolated from the cache data of other types of tasks such as embedding computation and file conversion.

[0161] Next, construct the cache query key: using the language model generated namespace identifier as a prefix, concatenate the ontology version V1.5, the target ontology class project implementation steps, business tag project management, content feature identifier hash string, and the large language model service version LLM-V4.2 in a preset order. After uniformly encoding each field using preset delimiters (such as "|" or "&"), generate the ontology-aware cache query key and bind it to the current task context. The key structure is as follows:

[0162] llm_completion|V1.5|Project Implementation Steps|Project Management|[Content Feature Identifier Hash Value]|LLM-V4.2.

[0163] Step 104: Based on the cache query key, perform cache matching in the multi-level cache by combining the cell association information and the cell input data;

[0164] Multilevel caching refers to a layered cache storage system, consisting of multiple layers of cache with increasing access speed and decreasing storage capacity, used to balance cache access efficiency and data sharing capabilities. In this embodiment, multilevel caching includes local in-process cache LRU, distributed shared cache (such as Redis), and task-level persistent cache, etc.

[0165] Redis (Remote Dictionary Server) refers to a distributed key-value storage system. As a cross-process shared cache layer, it supports multiple service nodes to share cached data and is used to carry cached record data and task status data that can be reused across instances.

[0166] Cache record data refers to complete data entries stored in the cache system, indexed by the cache query key, and includes multiple fields such as task execution result, version identifier, verification rules, validity period, and status flag.

[0167] The cache validity expiration time (TTL) refers to the expiration time node associated with cached record data. When the current system time exceeds this expiration time node, the cached record data automatically expires and can no longer be reused.

[0168] Dirty data tag (also known as data validity tag status) refers to the status identifier field of cached record data, which is used to mark whether the cached result has become invalid due to the update of ontology rules or changes in business configuration. Cache record data marked as dirty cannot be directly reused.

[0169] O(1) time complexity is an indicator of algorithm performance, meaning that the execution time of cache lookup operations is not affected by the total number of data entries in the cache, and can achieve a fixed time consumption to complete key-value retrieval.

[0170] Optionally, step 104 includes the following sub-steps:

[0171] Based on the cache lookup key, perform key matching on each pre-stored cache lookup key in the multi-level cache;

[0172] If no pre-stored cache query key is matched, the cache match is considered unsuccessful.

[0173] If a pre-stored cache query key is matched, the target cache record data bound to the pre-stored cache query key is retrieved from the multi-level cache.

[0174] Extract the cache validity termination time and data validity flag status from the target cache record data;

[0175] Compare the valid expiration time of the cache with the current time;

[0176] According to the preset cache validity verification rules, the target data items in the unit association information and unit input data are extracted respectively, and each target data item is compared with the corresponding item in the target cache record data;

[0177] If the data validity status is marked as invalid, the current time exceeds the cache validity expiration time, or any target data item does not match, then the cache match is deemed unsuccessful.

[0178] If the data validity flag is set to valid, the current time has not exceeded the cache validity expiration time, and all target data items match, then the cache match is considered successful.

[0179] If the cache match is successful, the target execution result is extracted from the target cache record data and returned to the requester of the pending document task.

[0180] Step 104 performs hierarchical retrieval and multi-dimensional validity verification based on a multi-level caching system, ensuring that the reused cached results match the content, semantic constraints, and validity period of the current task processing unit. The specific execution flow is as follows:

[0181] First, perform hierarchical key matching: according to the order of the cache hierarchy from near to far, perform cache lookup key matching in each level of cache in turn: first perform key lookup in the local LRU cache with O(1) time complexity. If no matching is found for the corresponding pre-stored cache lookup key, continue to query the distributed shared cache formed by Redis. If no matching is found for the pre-stored cache lookup key in all multiple caches, it is directly determined that the cache matching has failed.

[0182] Secondly, retrieve the target cache record: when the corresponding pre-stored cache query key is matched in any level of cache, retrieve the target cache record data bound to the pre-stored cache query key. The target cache record data includes fields such as task execution result, ontology version, model service version, business tag, ontology verification shape, cache valid termination time, input summary, and dirty data marker.

[0183] Next, the validity period verification and business attribute verification are performed sequentially: First, the cache validity expiration time is extracted from the target cache record data and compared with the current time. If the current time exceeds the cache validity expiration time, the validity period verification is deemed to have failed; if the current time does not exceed the cache validity expiration time, the validity period verification is deemed to have passed. Then, according to the preset verification rules, fields such as ontology version, model service version, business tag, and ontology verification shape are extracted from the unit association information, and fields such as input summary are extracted from the normalized processing results of the unit input data. These are compared with the corresponding fields in the target cache record data item by item. At the same time, the dirty data marking status of the target cache record data is verified.

[0184] Finally, the matching result is output: If the current time has not exceeded the cache expiration time, and the ontology version, model version, business tag, ontology validation shape, and input digest all match, and dirty data is marked as not dirty (i.e., the data is valid), then the cache matching is successful; the task execution result is extracted from the target cache record data and returned to the task requester, while the access order of the local cache is updated. If the validity period verification fails, or any business attribute field does not match, or dirty data is marked as dirty (i.e., the data is invalid), then the cache matching is unsuccessful, and the process continues to the next level of cache query or external service call.

[0185] For example, using the aforementioned front-end interactive single-slot AI completion task, a corresponding ontology-aware cache query key has been generated, and a multi-level cache matching process has been initiated.

[0186] First, perform hierarchical key matching: first, perform a key lookup in the local LRU cache, retrieve the pre-stored cache lookup key that matches the current cache lookup key, and then retrieve the target cache record data corresponding to the pre-stored cache lookup key.

[0187] Subsequently, a multi-dimensional validity check is performed: The first step is to check the validity period by extracting the cache expiration time from the target cache record data and comparing it with the current time. If the current time does not exceed the cache expiration time, the validity period check passes. The second step is to perform a business attribute comparison: Extract the ontology version V1.5, the large language model service version LLM-V4.2, the business tag project management, and the ontology verification shape slot_step_v1 from the unit association information. Extract the input summary from the normalized data corresponding to the unit input data and compare it with the corresponding fields in the target cache record data to confirm that all fields are consistent. At the same time, it is verified that dirty data is marked as not dirty, that is, the data is valid. The cache match is determined to be successful. The slot completion text result in the cache record is extracted and returned to the front-end editor requester.

[0188] In addition, if any field is inconsistent or dirty data is marked as dirty during the second step of the verification process, it is determined that the cache match is unsuccessful, and the distributed shared cache is queried again. If the pre-stored cache query key is still not matched, the subsequent service scheduling stage will be entered.

[0189] Step 105: If the cache matching fails, service scheduling is performed based on the task association information. After service circuit breaker detection, the target external service is called to execute the document task to be processed.

[0190] Service scheduling refers to a processing mechanism that prioritizes, controls traffic, and routes external service call requests based on task association information. It is used to ensure the response time of high-priority tasks and prevent background batch tasks from excessively occupying service resources.

[0191] Scheduling priority refers to the resource scheduling weight identifier bound to the document task to be processed (e.g., front-end interactive tasks have high priority, and back-end batch parsing tasks have low priority), which is used to determine the execution order of call requests in the waiting queue.

[0192] A service circuit breaker is a fault protection component for external services. Each external service has an independent circuit breaker, which is used to actively block call requests when an external service is abnormal, so as to prevent the failure from spreading and causing the overall task to collapse.

[0193] The circuit breaker closed state refers to the normal operating state of the circuit breaker. In this state, call requests can be sent to external services normally, and the system synchronously records measurement data such as call time, success and failure status.

[0194] The fuse open state refers to the fault-blocking state of the fuse. In this state, the system refuses to send new call requests to the corresponding external services, reserving a recovery window for the faulty services and avoiding continuous pressure that will exacerbate the fault.

[0195] The fuse half-open state refers to the fuse recovery probe state. It enters this state after the cooling period ends, allowing only a small number of probe requests to pass through, which is used to verify whether the external service has been restored to an available state.

[0196] Cooling-off time refers to the duration of call blocking set after the fuse enters the open state. During this duration, new business call requests are prohibited from being sent, reserving a fault recovery period for external services.

[0197] An adaptive concurrency controller is a management component that dynamically adjusts the upper limit of concurrent calls based on the running status of external services. It adjusts the number of concurrent permissions based on recent call latency, error rate, and queue length to avoid overloading the service connection pool.

[0198] Local degradation processing (service degradation) refers to the fault tolerance mechanism when external services are unavailable, which does not send business call requests to the failed target external service. It includes querying multi-level cache to obtain historical cache results. If the cache is not hit, a degradation output result is generated based on the semantic constraint rules corresponding to the target ontology class, or the task processing unit is marked as abnormal, so as to ensure that the main business process of the document task to be processed can continue to be executed.

[0199] Service call metrics data refers to the operational statistics collected during external service calls, including the call initiation time, call return time, call duration, call result identifier (including success, failure, timeout, etc.), task type, target ontology class, and exception information. These data are used for circuit breaker determination, adaptive concurrency adjustment, and operational metric statistics.

[0200] Semantic constraint rules refer to a set of preset business rules bound to the target ontology class, which specify the fields, data structures and value ranges of the corresponding business entity output. In the case of external service circuit breaking and degradation, simplified degradation results that meet the business format requirements are generated based on these rules to ensure the business compliance of the task output.

[0201] The task processing unit marker refers to the status identifier bound to the task processing unit. It is obtained by parsing the process template corresponding to the target ontology class and is used to record the business attributes of the task processing unit, including key / non-key task identifiers, whether skipping is allowed, whether degradation is supported, and abnormal status markers, for logical judgment by circuit breakers and fault-tolerant branches.

[0202] Optionally, step 105 includes the following sub-steps:

[0203] Extract the task type from the unit association information;

[0204] Match the target external service to the task type and obtain the call status of the target external service; the call status includes the number of consecutive call failures and the call timeout rate;

[0205] If the number of consecutive failed calls does not exceed the preset threshold and the call timeout ratio does not exceed the preset ratio threshold, then the circuit breaker detection of the target external service is deemed to have passed, and the circuit breaker of the target external service is set to the closed state.

[0206] If the fuse is in the closed state, a call request is generated based on the unit association information and unit input data;

[0207] The call request is sent to the target external service according to the scheduling priority.

[0208] If the number of consecutive failed calls exceeds the preset threshold, or the call timeout ratio exceeds the preset ratio threshold, the circuit breaker detection of the target external service is determined to have failed, the circuit breaker of the target external service is set to the open state and the cooling time is started.

[0209] If the circuit breaker is in the open state, calls to the target external service will be refused and local degradation processing will be performed based on the unit association information and cache query key until the cooldown period ends;

[0210] If the cooldown period ends, the fuse will be switched to a half-open state and a preset number of probe requests will be sent to the target external service. The target external service will then return a probe feedback status corresponding to the probe requests.

[0211] If the detection feedback status meets the preset service availability conditions, the fuse will be switched to the closed state and the process will jump to the step of generating a call request based on the unit association information and unit input data.

[0212] If the detection feedback status does not meet the preset service availability conditions, the fuse will be switched to the open state and the cooldown time will be restarted.

[0213] Step 105 is used to perform priority scheduling, circuit breaker protection, and concurrency control on external service calls in cache miss scenarios to ensure the stability of service calls and the rationality of resource allocation. The specific execution process is as follows:

[0214] First, the request is enqueued and prioritized: if the cache match fails, the scheduling priority and task type are extracted from the unit association information, and the target external service corresponding to the task type is matched; according to the scheduling priority, the call request of the task processing unit is placed into the corresponding waiting queue, and the task with the higher scheduling priority is placed at the front of the waiting queue;

[0215] Secondly, perform circuit breaker status detection: retrieve the current status of the circuit breaker corresponding to the target external service, collect the number of consecutive call failures, call timeout ratio, and average latency index within a preset time window, and determine whether the circuit breaker trigger threshold is met.

[0216] Subsequently, processing is performed according to the fuse status and the specific scenario:

[0217] If the circuit breaker is in the closed state, and the number of consecutive failed calls and the call timeout ratio do not exceed the preset thresholds, the service circuit breaker detection is considered to have passed. After the call request is verified by the adaptive concurrency controller to meet the concurrency allowance, it is sent to the target external service. The target external service executes the current task processing unit and synchronously records the service call metric data of this call. This data is used to update the circuit breaker statistics and operation and maintenance observation metrics. Specifically, if the service call is successful, the service success statistics and average latency metrics are updated; if the service call fails, the number of failures and timeout counts are accumulated. When the accumulated number of failures exceeds the preset failure threshold, the circuit breaker is switched to the open state and a cooldown period is initiated.

[0218] If the circuit breaker is in the open state, the service circuit breaker detection is deemed to have failed, new call requests to the target external service are rejected, and local degradation processing is performed based on the unit association information and cache lookup key. Specifically, local degradation processing includes:

[0219] Use the cache lookup key corresponding to the current task processing unit to re-query the multi-level cache and determine whether there are any available historical cache records.

[0220] If a valid historical cache record is found, the task execution result within that historical cache record will be returned directly.

[0221] If no valid historical cache record is found in the query, and the current document task supports fault tolerance and degradation, then the target ontology class carried by the unit association information of the current task processing unit is read, and the degradation output result is returned based on the semantic constraint rules corresponding to the target ontology class, or the task processing unit tag obtained by parsing the process template corresponding to the target ontology class is read.

[0222] Based on the read task processing unit flag, determine whether the task processing unit bound to the currently faulty external service is a non-preset critical task;

[0223] If a task processing unit is marked as a non-preset critical task, then the task processing unit is marked as an abnormal processing unit and skipped. If the remaining task processing units correspond to other non-faulty target external services, the remaining task processing units of the current pending document task continue to be executed to maintain the overall business link of the task without interruption.

[0224] If a task processing unit is marked as a preset critical task and cannot be downgraded, the call request will be moved into the delayed retry queue and wait for the cooldown time to end. For example, if the upper-level document task to be processed is a front-end interactive task and the current task processing unit cannot be downgraded, the downgrade output result returned will be a prompt that the external service of the task processing unit is temporarily unavailable.

[0225] If the cooling-off period ends, the fuse switches to a half-open state. In the half-open state, only a preset number of low-overhead, retryable probe requests are allowed, while other service requests are still processed according to the rules when the fuse is in the open state. A preset number of probe requests are sent to the target external service, and the probe feedback status corresponding to the probe request is returned by the target external service.

[0226] If the probe feedback status meets the preset service availability conditions, that is, the probe request is successfully called and the service indicators are restored to the threshold range, the circuit breaker is switched to the closed state, the concurrent permission of external services is gradually released, and the process jumps to the step of generating a call request based on the unit association information and unit input data.

[0227] If the probe feedback status does not meet the preset service availability conditions, i.e. the probe request call fails, the circuit breaker will be switched to the open state, the cooldown time will be extended, and the cooldown timer will be restarted.

[0228] Step 106: Receive the execution result returned by the target external service and return it to the requester of the document task to be processed.

[0229] The requester refers to the terminal entity that initiates the document processing task, including the front-end interactive terminal (user front-end operation interface) and the back-end batch task terminal (system automatic batch processing program), and is the final response object of the entire document processing task.

[0230] Optionally, step 106 is the final response stage for the document task to be processed, used to integrate the valid task execution results and provide compliant feedback to the requesting party. The specific execution steps are as follows:

[0231] Result reception: In the scenario where the circuit breaker is closed and the target external service is available, after completing the call request issuance and external service operation process, the standard task execution results returned by the target external service corresponding to each task processing unit are uniformly received. At the same time, the legality and integrity of the results are verified, and abnormal, incomplete and invalid return data are removed.

[0232] Result Integration: Traverse the execution output of all task processing units under the current document task to be processed, and distinguish between two types of valid results: one is the standard task execution result returned normally by the target external service, and the other is the degraded output result or unit skip status record generated by local degradation processing in the case of service circuit breaker failure. All valid outputs of all units are integrated and summarized to form a complete document task processing result set.

[0233] Result adaptation: Adapt the output format according to the type and attributes of the task to be processed (front-end interactive task, back-end batch task). Optimize the result display format for front-end interactive tasks to ensure that the front-end can render normally; retain complete structured data for back-end batch tasks to meet the needs of batch archiving and secondary processing.

[0234] Result return: The integrated and matched complete task processing result is encapsulated into a compliance response message and uniformly returned to the requester who initiated the document task to be processed, thus completing the closed loop of the entire process of a single document processing;

[0235] Post-caching update: For standard task execution results returned normally by external services, based on the cache query key of the corresponding task processing unit, update multi-level cache data, overwrite expired cache content, and provide a basis for cache reuse for subsequent similar document tasks; the degraded output results generated locally do not participate in cache updates.

[0236] In some embodiments, a task checkpoint refers to a snapshot of the task state that is persistently stored after each step of the parsing pipeline and document generation task has been completed. The core of the checkpoint includes the identifier of the completed step, the key intermediate calculation results, the task error set, and the task running statistics, which are used to enable long task breakpoint resume, state backtracking, and data consistency assurance.

[0237] Unified observable data refers to pipeline operation metrics, service status metrics, and fault tolerance mechanism metrics collected throughout the entire lifecycle of a task. It covers multi-dimensional data such as step time consumption, task success rate, cache hit rate, external service latency, circuit breaker status, and rate limiting counts. It serves as the core basis for task operation and maintenance, fault location, and adaptive parameter tuning.

[0238] Optionally, after the single task pipeline in step 106 is completed, the current task checkpoint is automatically generated and saved, recording all completed pipeline steps, key intermediate results of each task processing unit, error sets generated by circuit breaking / degradation, and full-process statistics. If a task cancellation instruction is received during task execution, subsequent external service calls will be terminated at the step boundary of the current task processing unit, and the database task state will be synchronously fixed to ensure system data consistency. When a task needs to recover from an anomaly, historical checkpoint data is read, steps that have been completed normally are automatically skipped, and only failed, interrupted, or incomplete task processing units are re-executed. The results of subsequent tasks with data dependencies are also recalculated to avoid a full rerun and improve the efficiency of long task recovery.

[0239] Unified observable data acquisition and adaptive optimization: Using the complete task execution chain in step 106 as the data acquisition carrier, all steps of this pipeline are recorded, including single-step time, total task time, overall success rate, number of errors, multi-level cache hit rate, latency of external AI service calls, circuit breaker state switching records, and the number of rate limiting triggers. Simultaneously, the operational status of the database, Redis cache, vector library, graph database, and various external AI services is collected in real time through a health check interface. This data is then aggregated through an indicator interface to form visualized trend data and bottleneck location data. Based on the collected observable data, operational and fault-tolerance parameters are adaptively adjusted in reverse, dynamically optimizing cache expiration time, service concurrency thresholds, circuit breaker trigger thresholds, and model call priorities to achieve dynamic iterative optimization of system performance and fault tolerance.

[0240] In some embodiments, the above-described document task processing method based on multi-level caching further includes:

[0241] Based on the execution results and unit association information, cache record data is generated;

[0242] The cached record data is bound to the cache query key and then written to the multi-level cache.

[0243] Specifically, the cache record data generated above based on execution results, service call metric data, and task association information includes:

[0244] Use the execution result as a cached value;

[0245] According to the preset metadata extraction rules, extract the cache verification field data and task type from the unit association information;

[0246] Target cache validity period matching task type;

[0247] Add the current time to the target cache validity period to generate the target cache termination time;

[0248] Cache record data is generated based on the cache value, cache validation field data, and the target cache termination time.

[0249] Cache record data refers to the complete storage entry written to a multi-level cache. It consists of the cache value, cache verification field data, and the target cache expiration time, realizing the binding storage of cache content with verification metadata and expiration time.

[0250] The cached value refers to the business entity data stored in the multi-level cache, which is the standard task execution result returned normally by the target external service; the degradation output result generated by the local degradation processing does not generate cache record data and is not written to the multi-level cache.

[0251] The cache validation field data refers to a set of validation metadata extracted from the unit association information, including information such as model / service version, ontology version, and tags. It is used to perform consistency checks when the cache is hit, preventing the reuse of historical cache results after model upgrades or ontology rule changes.

[0252] The target cache expiration time refers to the expiration time of the cache record. It is calculated by adding the current time to the target cache validity period. Once the current time reaches this time, the corresponding cache record is determined to be expired.

[0253] Optionally, the above steps are a post-cache write process after a cache hit, executed only when the task processing unit calls the target external service to obtain the standard task execution result; if local degradation processing is performed to obtain a degradation output result, this cache write process is not executed to avoid storing emergency degradation data in the cache. This process is linked with the aforementioned cache query, circuit breaker degradation, and observable indicator collection processes, and the specific execution steps are as follows:

[0254] Get basic data: Get the execution result returned by the target external service as the cache value; read the unit association information of the corresponding task processing unit, and extract the cache verification field data and task type according to the preset metadata extraction rules; the cache verification field data should at least include the model version, service version, ontology version, and tag, which are used to perform version consistency verification when the cache is hit in the future, and avoid reusing old cache data after the model is upgraded;

[0255] Determine the cache validity period: Based on the extracted task type, match the target cache validity period corresponding to that object type; read the current time, add the current time to the target cache validity period, and calculate the target cache expiration time;

[0256] Assemble cache record data: Use the standard task execution result as the cache value, and combine it with cache verification field data and target cache termination time to assemble cache record data; the cache record data can also include cache creation time, source service, and hit statistics fields for operation and maintenance statistics.

[0257] Bind and write to multi-level cache: Bind cache record data with the generated cache query key, and write the bound entries to the multi-level cache; prioritize writing to the local LRU cache, and simultaneously write to the Redis distributed cache to achieve cross-process sharing of cache entries;

[0258] Linked metrics statistics: After the cache write is completed, update the cache-related statistical metrics in the unified observable system, update the cache hit rate statistics and cache write count statistics, and provide raw metric data for subsequent system adaptive parameter tuning;

[0259] If the current task processing unit performs local degradation processing and obtains a degradation output result, then all cache record generation and writing steps are skipped, and the degradation output result is not stored in the multi-level cache.

[0260] In some embodiments, refer to Figure 2 , Figure 2 This is an optional structural diagram of a document task processing system for implementing the above-mentioned document task processing method based on multi-level caching, provided by an embodiment of the present invention. The system is used to execute all processing steps of the aforementioned method. The system as a whole consists of a multi-level caching module, a circuit breaker and degradation module, a concurrency scheduling module, a task consistency module, and an observability module. At the hardware level, data storage is completed collaboratively through business service processes, Redis distributed caching, and relational databases.

[0261] Among them, the local process LRU cache implements millisecond-level cache hit lookup and is used for the cache matching lookup step in the execution method; the Redis distributed cache is responsible for sharing cache data across processes and saving task running status, and is used to realize multi-level cache sharing across instances and task status persistence; the relational database completes the persistent storage of long-term task records; the observability module provides program interfaces (APIs) to provide data support for the front-end operation and maintenance dashboard.

[0262] The modules work together to perform cache read / write operations, circuit breaker and degradation handling, concurrent traffic control, long task checkpoint maintenance, and full-process operation indicator collection, respectively. The functions, processing content, and corresponding technical effects of each module in the overall system are shown in Table 1 below.

[0263] Table 1. System Functional Modules and Corresponding Processing Content and Technical Effects

[0264]

[0265] The modules in Table 1 work together in the following ways: the concurrency scheduling module regulates task traffic; when a task calls an external service, the circuit breaker and degradation module monitors the service health status and performs local degradation processing in case of failure; after a successful external service call, the multi-level caching module completes the result cache writing; throughout the entire process of a long task, the task consistency module continuously maintains checkpoints and task status, and persists task records to a relational database; all process operation data is uniformly summarized in the observability module, which outputs data to the front-end dashboard and provides indicator data to the circuit breaker and degradation module and the concurrency scheduling module for threshold adaptive optimization.

[0266] In some embodiments, refer to Figure 3 , Figure 3 This is another optional flowchart of a document task processing method based on multi-level caching provided in an embodiment of the present invention. Figure 3 The complete execution flow of the document task processing method based on multi-level caching of the present invention is shown. It is used to realize document parsing, knowledge retrieval, intelligent writing, AI completion, cache reuse of exported document tasks, priority scheduling, circuit breaking and degradation, adaptive concurrency control, and result output and task recovery processing.

[0267] Combination Figure 3 The overall method flow of this invention embodiment is described below. This flow is used to execute the aforementioned document task processing method based on multi-level caching, as detailed below:

[0268] S1. Receive the document task to be processed, identify the task type and user identity information; the document task to be processed includes document parsing task, knowledge retrieval task, intelligent writing task, AI completion task, and export task;

[0269] S2. Perform normalization preprocessing on the task input data, including input files, text blocks, query statements, prompt words, and model parameters, to generate a stable fingerprint. This stable fingerprint is used for cache hit determination in subsequent multi-level caching.

[0270] S3. Before calling at least one of the target external services such as vector embedding, reranker, PDF conversion, vector query, or large language model (LLM), generate a cache query key based on stable fingerprint and unit association information, and perform multi-level cache matching verification.

[0271] S4. Determine the cache matching result; if the cache is not hit, the call request is sent to the rate limiter and priority scheduler; assign higher priority to interactive completion, user front-end query, and breakpoint recovery type front-end interactive tasks, and schedule background tasks of batch parsing and batch vectorization in queue order.

[0272] S5. The scheduled request is sent to the service circuit breaker. When the number of consecutive failed calls or the timeout ratio of the corresponding target external service exceeds the preset threshold, the circuit breaker switches to the open state and executes the circuit breaking logic. After the cooling period, it enters the half-open state to execute the probe request. When the service is normal, the circuit breaker remains in the closed state and allows the call request to pass.

[0273] S6. The request passes through the adaptive concurrency controller; the adaptive concurrency controller collects the service response latency, call error rate and task queue length in the most recent time window, and dynamically adjusts the number of concurrent permissions to prevent the target external service and database connection pool from being overwhelmed by requests.

[0274] S7. Perform subsequent processing, including verifying the results returned by external services, writing cached records to multi-level caches, returning task results to the requester, persisting long task checkpoints, and handling task interruption recovery, to complete the entire process of a pending document task.

[0275] If the cache match is determined to be a cache hit, the cache record data in the multi-level cache is read directly, skipping the scheduling, circuit breaking, and external service call process of S4-S6, and directly entering the subsequent output steps of S7.

[0276] In some embodiments, refer to Figure 4 , Figure 4 Another optional flowchart of a document task processing method based on multi-level caching provided in an embodiment of the present invention is as follows:

[0277] Generate an ontology-aware cache key: Obtain the stable fingerprint corresponding to the task processing unit, and combine it with the task type, ontology version, target ontology class, model / service version, tag, tenant or project scope to generate an ontology-aware cache key; use this ontology-aware cache key to perform multi-level cache matching verification; if the cache is hit, directly obtain the cache result; if the cache is not hit, proceed to the next step. This step corresponds to the cache query key generation and multi-level cache matching verification process;

[0278] Service routing and circuit breaker degradation based on ontology constraints: Based on the target ontology class corresponding to the task processing unit, the corresponding target external service is matched to complete service routing; the request enters the circuit breaker to perform fault detection; when the external service fails and triggers the circuit breaker to open, the semantic constraint rules of the target ontology class are read, and the degradation result that conforms to the business semantic boundary is output; if the service is allowed to proceed normally, the target external service is called to obtain the task execution result. This step corresponds to the service circuit breaker detection and degradation processing flow;

[0279] Resilience metrics are linked with semantic quality: collect resilience metrics such as cache hit rate, circuit breaker count, rate limiting count, and error distribution; combine the semantic constraints of the target ontology class to evaluate the semantic compliance quality of the task output content; output the resilience metrics and semantic quality evaluation results to the observability module to perform adaptive tuning of task scheduling, circuit breaker threshold, and cache expiration time. This step corresponds to the result processing, metric collection, and parameter tuning process.

[0280] In some embodiments, refer to Figure 5 , Figure 5 This is an optional schematic diagram of the evidence constraint verification and traceable output processing flow in an embodiment of the present invention. The evidence constraint verification and traceable output processing flow is an embedded sub-flow of the above-mentioned document task processing method based on multi-level caching, as detailed below:

[0281] Obtain the input object of the task processing unit: Read the task processing unit to be processed, obtain the unit input data and unit association information corresponding to the task processing unit, and use them as the input object of this process;

[0282] Ontology-tag-evidence constraint processing: Load the constraints corresponding to the target ontology class, tags, and evidence types from the unit association information of the task processing unit to determine the semantic boundaries, tag filtering range, and evidence acceptance rules for this processing;

[0283] Service and rule joint processing: Based on constraints, the target external service is called; at the same time, preset business rules are loaded to impose pre-constraints on the input and make rule corrections on the intermediate output of the service to obtain the preliminary task processing results;

[0284] Result verification and traceable output: Verify the initial task processing results and generate verification metadata; bind the verification metadata and evidence tracing information to the task processing results to form task processing results carrying tracing information, which can be written to the cache and returned by the requester.

[0285] It should be noted that the ontology, tags, and evidence constraints are all bound to the task processing unit; under the same top-level document task, different task processing units can have independent constraints, and each task processing unit independently executes the above evidence constraint verification and traceable output processing flow.

[0286] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0287] The document task processing apparatus based on multi-level caching provided in the embodiments of this application will be described below. The document task processing apparatus based on multi-level caching described below can be referred to in correspondence with the document task processing method based on multi-level caching described above.

[0288] Reference Figure 6 , Figure 6 This is an optional structural diagram of a document task processing device based on multi-level caching provided in an embodiment of the present invention. The device is used to execute the aforementioned document task processing method based on multi-level caching, including:

[0289] The task receiving module 201 is used to receive the document task to be processed and to obtain the task association information and task input data of the document task to be processed.

[0290] The unit splitting module 202 is used to perform normalization preprocessing on the task input data, and determine at least one task processing unit and the unit input data and unit association information of the task processing unit based on the preprocessing results and task association information.

[0291] The key generation module 203 is used to generate cache query keys based on the cell input data and cell association information;

[0292] The cache matching module 204 is used to perform cache matching in a multi-level cache based on the cache query key, combined with cell association information and cell input data;

[0293] The service scheduling module 205 is used to perform service scheduling based on the unit association information and unit input data if the cache matching fails. After service circuit breaker detection, it calls the target external service to execute the document task to be processed.

[0294] The result return module 206 is used to receive the execution result returned by the target external service and return it to the requester of the document task to be processed.

[0295] In some embodiments, the unit splitting module 202 is configured as follows:

[0296] The task input data is processed for format standardization and redundant information filtering to generate standardized data;

[0297] Extract the process template corresponding to the target ontology class from the task association information;

[0298] According to the preset task decomposition strategy, the process template is instantiated and parsed to generate multiple task processing units and obtain the unit association information of each task processing unit.

[0299] Extract the data selection range and task type from the unit association information;

[0300] Based on the data selection range, the normalized data is split and allocated to obtain the unit input data for each task processing unit.

[0301] In some embodiments, the key generation module 203 is configured as follows:

[0302] Extract task context information and task type from the unit association information;

[0303] Content hash digests are extracted from the non-text content in the unit input data to generate content digest data for each task processing unit.

[0304] Based on the field selection strategy associated with task type, target fields are extracted from unit association information and content summary data and concatenated in order to generate content feature identifiers for each task processing unit.

[0305] The target cache namespace matches the task type;

[0306] The target cache namespace, content feature identifier, and task context information are encoded and combined to generate cache lookup keys for each task processing unit.

[0307] In some embodiments, the cache matching module 204 is configured as follows:

[0308] Based on the cache lookup key, perform key matching on each pre-stored cache lookup key in the multi-level cache;

[0309] If no pre-stored cache query key is matched, the cache match is considered unsuccessful.

[0310] If a pre-stored cache query key is matched, the target cache record data bound to the pre-stored cache query key is retrieved from the multi-level cache.

[0311] Extract the cache validity termination time and data validity flag status from the target cache record data;

[0312] Compare the valid expiration time of the cache with the current time;

[0313] According to the preset cache validity verification rules, the target data items in the unit association information and unit input data are extracted respectively, and each target data item is compared with the corresponding item in the target cache record data;

[0314] If the data validity status is marked as invalid, the current time exceeds the cache validity expiration time, or any target data item does not match, then the cache match is deemed unsuccessful.

[0315] If the data validity flag is set to valid, the current time has not exceeded the cache validity expiration time, and all target data items match, then the cache match is considered successful.

[0316] If the cache match is successful, the target execution result is extracted from the target cache record data and returned to the requester of the pending document task.

[0317] In some embodiments, the service scheduling module 205 is configured as follows:

[0318] Extract the task type from the unit association information;

[0319] Match the target external service to the task type and obtain the call status of the target external service; the call status includes the number of consecutive call failures and the call timeout rate;

[0320] If the number of consecutive failed calls does not exceed the preset threshold and the call timeout ratio does not exceed the preset ratio threshold, then the circuit breaker detection of the target external service is deemed to have passed, and the circuit breaker of the target external service is set to the closed state.

[0321] If the fuse is in the closed state, a call request is generated based on the unit association information and unit input data;

[0322] The call request is sent to the target external service according to the scheduling priority.

[0323] If the number of consecutive failed calls exceeds the preset threshold, or the call timeout ratio exceeds the preset ratio threshold, the circuit breaker detection of the target external service is determined to have failed, the circuit breaker of the target external service is set to the open state and the cooling time is started.

[0324] If the circuit breaker is in the open state, calls to the target external service will be refused and local degradation processing will be performed based on the unit association information and cache query key until the cooldown period ends;

[0325] If the cooldown period ends, the fuse will be switched to a half-open state and a preset number of probe requests will be sent to the target external service. The target external service will then return a probe feedback status corresponding to the probe requests.

[0326] If the detection feedback status meets the preset service availability conditions, the fuse will be switched to the closed state and the process will jump to the step of generating a call request based on the unit association information and unit input data.

[0327] If the detection feedback status does not meet the preset service availability conditions, the fuse will be switched to the open state and the cooldown time will be restarted.

[0328] In some embodiments, the above-described document task processing apparatus based on multi-level caching further includes a cache writing module, which is configured to:

[0329] Based on the execution results and unit association information, cache record data is generated;

[0330] The cached record data is bound to the cache query key and then written to the multi-level cache.

[0331] In some embodiments, the cache writing module includes a cache generation unit, which is configured to generate cache record data based on the execution result and unit association information, specifically configured as follows:

[0332] Use the execution result as a cached value;

[0333] According to the preset metadata extraction rules, extract the cache verification field data and task type from the unit association information;

[0334] Target cache validity period matching task type;

[0335] Add the current time to the target cache validity period to generate the target cache termination time;

[0336] Cache record data is generated based on the cache value, cache validation field data, and the target cache termination time.

[0337] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0338] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0339] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0340] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0341] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0342] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned document task processing method based on multi-level caching. This electronic device can be any smart terminal, including tablet computers.

[0343] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0344] Please see Figure 7 , Figure 7 The hardware structure of an electronic device according to another embodiment is illustrated, including:

[0345] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0346] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the document task processing method based on multi-level caching according to the embodiments of this invention.

[0347] The input / output interface 903 is used to implement information input and output;

[0348] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0349] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0350] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0351] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described document task processing method based on multi-level caching.

[0352] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0353] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0354] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A document task processing method based on multi-level caching, characterized in that, include: Receive the document task to be processed, and obtain the task association information and task input data of the document task to be processed; The task input data is preprocessed to be normalized, and at least one task processing unit, the unit input data and unit association information of the task processing unit are determined based on the preprocessing results and the task association information. Based on the unit input data and the unit association information, a cache query key is generated; Based on the cache query key, cache matching is performed in a multi-level cache by combining the unit association information and the unit input data; If the cache matching fails, service scheduling is performed based on the unit association information and the unit input data. After service circuit breaker detection, the target external service is called to execute the document task to be processed. Receive the execution result returned by the target external service and return it to the requester of the document task to be processed.

2. The document task processing method based on multi-level caching according to claim 1, characterized in that, The step of performing normalization preprocessing on the task input data, and determining at least one task processing unit and its unit input data and unit association information based on the preprocessing results and the task association information, includes: The task input data is processed for format standardization and redundant information filtering to generate standardized data; Extract the process template corresponding to the target ontology class from the task association information; According to the preset task decomposition strategy, the process template is instantiated and parsed to generate multiple task processing units and obtain the unit association information of each task processing unit. Extract the data selection range and task type from the unit association information; Based on the data selection range, the standardized data is split and allocated to obtain the unit input data for each task processing unit.

3. The document task processing method based on multi-level caching according to claim 1, characterized in that, The step of generating a cache query key based on the unit input data and the unit association information includes: Extract the task context information and task type from the unit association information; Content hash digests are extracted from the non-text content in the unit input data to generate content digest data for each task processing unit. According to the field selection strategy associated with the task type, target fields are extracted from the unit input data and the content summary data and concatenated in order to generate the content feature identifier of each task processing unit; The target cache namespace that matches the task type; The target cache namespace, the content feature identifier, and the task context information are encoded and combined to generate cache query keys for each task processing unit.

4. The document task processing method based on multi-level caching according to claim 1, characterized in that, The step of performing cache matching in a multi-level cache based on the cache query key, combined with the unit association information and the unit input data, includes: Based on the cache query key, perform key matching on each pre-stored cache query key in the multi-level cache; If no pre-stored cache query key is matched, the cache match is considered unsuccessful. If a pre-stored cache query key is matched, the target cache record data bound to the pre-stored cache query key is retrieved from the multi-level cache. Extract the cache validity termination time and data validity flag status from the target cache record data; Compare the effective termination time of the cache with the current time; According to the preset cache validity verification rules, the unit association information and the target data items in the unit input data are extracted respectively, and each target data item is compared with the corresponding item in the target cache record data; If the data validity flag status is invalid, the current time exceeds the cache validity termination time, or any of the target data items do not match, then the cache matching is determined to be unsuccessful.

5. The document task processing method based on multi-level caching according to claim 4, characterized in that, Also includes: If the data validity flag is set to "data valid", the current time has not exceeded the cache validity termination time, and all target data items match, then the cache matching is deemed successful. If the cache match is successful, the target execution result is extracted from the target cache record data and returned to the requester of the document task to be processed.

6. The document task processing method based on multi-level caching according to claim 1, characterized in that, The process of performing service scheduling based on the unit association information and the unit input data, and invoking a target external service to execute the document task after service circuit breaker detection, includes: Extract the task type from the unit association information; Match the target external service of the task type and obtain the call status of the target external service; the call status includes the number of consecutive call failures and the call timeout ratio; If the number of consecutive failed calls does not exceed a preset threshold and the call timeout ratio does not exceed a preset ratio threshold, then the circuit breaker detection of the target external service is determined to be passed, and the circuit breaker of the target external service is set to the closed state. If the fuse is in the closed state, a call request is generated based on the unit association information and the unit input data; The call request is sent to the target external service according to the scheduling priority.

7. The document task processing method based on multi-level caching according to claim 6, characterized in that, Also includes: If the number of consecutive failed calls exceeds a preset threshold, or the call timeout ratio exceeds a preset ratio threshold, then the circuit breaker detection of the target external service is determined to have failed, the circuit breaker of the target external service is set to the open state and a cooling-off period is initiated. If the fuse is in the open state, the call to the target external service is refused and local degradation processing is performed based on the unit association information and the cache query key until the cooling time ends; If the cooling time ends, the fuse is switched to a half-open state and a preset number of probe requests are sent to the target external service, and the probe feedback status corresponding to the probe requests is returned by the target external service. If the detection feedback status meets the preset service availability conditions, the fuse is switched to the closed state and the process jumps to the step of generating a call request based on the unit association information and the unit input data; If the detection feedback status does not meet the preset service availability conditions, the fuse will be switched to the open state and the cooling time will be restarted.

8. The document task processing method based on multi-level caching according to claim 1, characterized in that, Also includes: Based on the execution result and the unit association information, cache record data is generated; The cached record data is bound to the cache query key and then written into the multi-level cache.

9. The document task processing method based on multi-level caching according to claim 8, characterized in that, The step of generating cache record data based on the execution result and the unit association information includes: The execution result is used as a cached value; According to the preset metadata extraction rules, the cache verification field data and task type are extracted from the unit association information; The target cache validity period matches the task type; Add the current time to the effective duration of the target cache to generate the target cache termination time; Based on the cached value, the cache verification field data, and the target cache termination time, cache record data is generated.

10. A document task processing device based on multi-level caching, characterized in that, include: The task receiving module is used to receive the document task to be processed and to obtain the task association information and task input data of the document task to be processed. The unit splitting module is used to perform normalized preprocessing on the task input data, and determine at least one task processing unit and the unit input data and unit association information of the task processing unit based on the preprocessing results and the task association information. A key generation module is used to generate a cache query key based on the unit input data and the unit association information; The cache matching module is used to perform cache matching in a multi-level cache based on the cache query key, combined with the unit association information and the unit input data; The service scheduling module is used to perform service scheduling based on the unit association information and the unit input data if the cache matching fails, and to call the target external service to execute the document task to be processed after service circuit breaker detection. The result return module is used to receive the execution result returned by the target external service and return it to the requester of the document task to be processed.

Images