Intelligent retrieval enhancement instruction generation system for automated document processing

Abstract

This invention relates to the field of electronic digital data processing technology, specifically to an intelligent retrieval enhancement instruction generation system for automated file processing. The system includes: receiving a file processing request for a file to be processed; responding to the file processing request by acquiring historical feedback information; determining the granularity adjustment requirements of relevant knowledge fragments in the file processing request, and adjusting the granularity of the knowledge fragments to obtain adjusted knowledge fragments; determining the irrelevance indicators between the adjusted knowledge fragments and the file processing request, and, based on the contextual relevance between the adjusted knowledge fragments and the file to be processed, determining the reference value indicators of the adjusted knowledge fragments; generating retrieval enhancement instructions corresponding to the file processing request; and performing automated file processing on the file to be processed. This invention achieves dual adaptive optimization of knowledge fragment granularity and reference weight, enabling the generation of retrieval enhancement instructions to dynamically adapt to the differentiated needs of different file processing scenarios.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and more specifically to an intelligent retrieval enhancement instruction generation system for automated document processing. Background Technology

[0002] Retrieval-Augmented Generation (RAG) technology is widely used in the field of automated document processing. This technology divides reference materials from external knowledge bases into knowledge fragments, retrieves relevant fragments based on user-input processing requests, and generates targeted processing instructions using a large language model. This enables automated cleaning, information extraction, and format conversion of document content. Regarding knowledge fragment segmentation strategies, existing technologies typically employ block-based methods with fixed character lengths or a fixed number of paragraphs, dividing long texts into fragments with uniform granularity to facilitate subsequent vector encoding and similarity retrieval.

[0003] However, the aforementioned fixed-granularity segmentation strategy fails to consider the differentiated requirements of different file processing requests for knowledge fragment granularity. This results in the acquired knowledge fragments often containing a mixture of valid information highly relevant to the user's current processing goal and irrelevant or even misleading invalid information. Because existing technologies lack a feedback optimization mechanism based on actual user modification behavior, they cannot infer the required granularity of knowledge fragments from the user's corrections of historical processing results. This leads to a mixture of valid and invalid feedback in the generated search enhancement instructions, requiring users to frequently manually correct processing results, increasing manual intervention costs, and reducing the accuracy and user experience of automated file processing. Summary of the Invention

[0004] This invention provides an intelligent retrieval enhancement instruction generation system for automated document processing to solve existing problems.

[0005] The intelligent retrieval enhancement instruction generation system for automated document processing of the present invention adopts the following technical solution: One embodiment of the present invention provides an intelligent retrieval enhancement instruction generation system for automated file processing, comprising: receiving a file processing request for a file to be processed; responding to the file processing request, obtaining historical feedback information; wherein the historical feedback information includes user modification records of historical automated file processing results; based on the historical feedback information, determining the granularity adjustment requirement index of relevant knowledge fragments of the file processing request, and adjusting the granularity of the knowledge fragments to obtain adjusted knowledge fragments; based on the historical feedback information, determining the irrelevance index between the adjusted knowledge fragments and the file processing request, and determining the reference value index of the adjusted knowledge fragments in combination with the contextual relevance degree between the adjusted knowledge fragments and the file to be processed; based on the reference value index, generating a retrieval enhancement instruction corresponding to the file processing request; and performing automated file processing on the file to be processed based on the retrieval enhancement instruction.

[0006] Furthermore, determining the granularity adjustment requirement index of the relevant knowledge fragments of the file processing request based on the historical feedback information includes: determining the user modification frequency and user modification magnitude of the relevant knowledge fragments of the file processing request in the generation of historical enhancement instructions based on the historical feedback information; and determining the granularity adjustment requirement index of the relevant knowledge fragments based on the user modification frequency and the user modification magnitude.

[0007] Further, the step of granularly adjusting the knowledge fragment to obtain the adjusted knowledge fragment includes: responding to the granularity adjustment requirement index meeting the preset adjustment conditions, performing fine-grained segmentation based on semantic boundaries on the relevant knowledge fragment to obtain intermediate knowledge fragments; determining the granularity adjustment requirement index of the intermediate knowledge fragment based on the historical feedback information; iteratively executing the fine-grained segmentation step and the granularity adjustment requirement index determination step until the granularity adjustment requirement index no longer meets the preset adjustment conditions, thereby obtaining the adjusted knowledge fragment.

[0008] Further, determining the irrelevance indicators between the adjusted knowledge fragment and the file processing request based on the historical feedback information includes: obtaining historical processing records matching the file processing request; determining the user modification frequency and cumulative percentage of user modifications in the corresponding historical enhancement instruction generation based on the historical processing records; and determining the irrelevance indicators between the adjusted knowledge fragment and the file processing request based on the user modification frequency and the cumulative percentage of user modifications.

[0009] Furthermore, determining the reference value index of the adjusted knowledge fragment by combining the degree of contextual relevance between the adjusted knowledge fragment and the file to be processed includes: determining the degree of contextual relevance between the adjusted knowledge fragment and the file to be processed; determining the reference value index of the adjusted knowledge fragment based on the degree of contextual relevance and the irrelevant index; wherein the reference value index increases with the increase of the degree of contextual relevance and decreases with the increase of the irrelevant index.

[0010] Furthermore, the acquisition of historical feedback information includes: recording the source identifiers of each knowledge fragment referenced when generating the historical search enhancement instruction; locating the modified knowledge fragments through the source identifiers based on the user's modifications to the historical search enhancement instruction; generating feedback tags for the modified knowledge fragments; and generating historical feedback information based on the feedback tags.

[0011] Furthermore, it also includes: after performing automated document processing, recording the source identifiers of each knowledge fragment referenced by the retrieval enhancement instruction generated this time; obtaining the user's modification record of the automated document processing result; based on the modification record and the source identifier, determining the knowledge fragment that was modified this time, and updating the historical feedback information.

[0012] Furthermore, the step of generating a search enhancement instruction corresponding to the file processing request based on the reference value index includes: filtering and sorting the adjusted knowledge fragments based on the reference value index; assembling the filtered and sorted adjusted knowledge fragments with the context of the file processing request and the file to be processed into an enhanced prompt; and generating a search enhancement instruction corresponding to the file processing request based on the enhanced prompt.

[0013] Furthermore, before determining the granularity adjustment requirement index of the knowledge fragment related to the file processing request based on the historical feedback information, the method further includes: retrieving candidate knowledge fragments from the knowledge base based on the processing requirements of the file processing request; filtering out knowledge fragments with a negative feedback tag ratio exceeding a first preset ratio threshold according to the feedback tags of the candidate knowledge fragments recorded in the historical feedback information, and retaining the knowledge fragments as the knowledge fragments related to the file processing request.

[0014] Furthermore, after generating the search enhancement instruction corresponding to the file processing request based on the reference value index, the method further includes: performing a weight self-verification on the search enhancement instruction, and regenerating the search enhancement instruction in response to the knowledge fragment citation ratio of the low reference value index exceeding a second preset ratio threshold.

[0015] The beneficial effects of the technical solution of the present invention are: In this embodiment of the invention, a file processing request for a file to be processed is received; in response to the file processing request, historical feedback information is obtained; wherein, the historical feedback information includes the user's modification records for historical automated file processing results; based on the historical feedback information, the granularity adjustment requirement index of the relevant knowledge fragments of the file processing request is determined, and the granularity of the knowledge fragments is adjusted to obtain the adjusted knowledge fragments; based on the historical feedback information, the irrelevance index between the adjusted knowledge fragments and the file processing request is determined, and the reference value index of the adjusted knowledge fragments is determined by combining the contextual correlation between the adjusted knowledge fragments and the file to be processed; based on the reference value index, a search enhancement instruction corresponding to the file processing request is generated; based on the search enhancement instruction, automated file processing is performed on the file to be processed.

[0016] This invention achieves dual adaptive optimization of knowledge fragment granularity and reference weight by establishing a mapping relationship between user modification behavior and the granularity and weight of knowledge fragments. This enables the generation of retrieval enhancement instructions to dynamically adapt to the differentiated needs of different file processing scenarios. On the other hand, it determines the granularity adjustment requirements based on the inverse correlation between user modification frequency and modification magnitude, and iteratively executes fine-grained segmentation until preset conditions are met, effectively eliminating the mixed state of effective and invalid information in knowledge fragments and reducing the manual correction cost of subsequent instruction generation. Furthermore, by integrating the user modification frequency and cumulative proportion of modification content of knowledge fragments in historical enhancement instruction generation with the correlation degree of the current file context, a reference value evaluation system with two dimensions of task relevance and scenario adaptability is constructed, improving the context awareness capability of retrieval enhancement instructions. Moreover, by using source identification to establish a reference mapping relationship between instruction generation and knowledge fragments, and continuously updating historical feedback information based on real-time user modification records, a self-evolving knowledge base optimization mechanism is formed, enabling automated file processing capabilities to continuously improve with increasing usage frequency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the intelligent retrieval enhancement instruction generation system for automated document processing provided in this application embodiment; Figure 2 This is a flowchart illustrating an intelligent retrieval enhancement instruction generation system for automated file processing in an application scenario provided in this application embodiment. Detailed Implementation

[0019] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following detailed description, in conjunction with the accompanying drawings and preferred embodiments, provides a detailed account of the specific implementation methods, structures, features, and effects of the method proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] like Figure 1 As shown, it illustrates an intelligent retrieval enhancement instruction generation system for automated document processing provided by an embodiment of the present invention, comprising: Step S110: Receive a file processing request for the file to be processed.

[0022] The aforementioned files to be processed refer to electronic documents containing content to be automated. This content can be in the form of structured text, semi-structured data, or unstructured paragraphs, specifically including contract texts, order emails, business reports, or log records. These files can be obtained through methods such as uploading from the user's local terminal, calling the enterprise content management system interface, automatically capturing email attachments, or batch exporting from a database. Metadata information including the filename, source system, file format, and creation time will be obtained to support file identification and contextual analysis in subsequent processing flows.

[0023] The aforementioned file processing request refers to a specific operation instruction issued by a user for a file to be processed. Its content defines the desired processing objective, such as anonymizing personal privacy information, extracting specific fields, standardizing format conversion, or classifying and labeling content. This request can be obtained through natural language descriptions entered by the user on the front-end interface, or automatically inferred from historical operation records using an intent recognition model, or received as a structured call instruction from an upstream business system via an application programming interface (API). The request content typically includes parameters such as processing type identifier, target file index, and expected output format.

[0024] Step S120: In response to the file processing request, obtain historical feedback information; wherein, the historical feedback information includes the user's modification records for the historical automated file processing results.

[0025] Historical feedback information is a data set recording the corrective operations and related behavioral characteristics performed by users on the generated processing results during past automated file processing. Its content includes records of user additions, deletions, and modifications to the processing results, specific text information of the modified content, timestamps of the modification operations, and identifiers of the original processing result range involved in the modification. It is used to quantitatively reflect the degree to which user needs are met and the degree of deviation of the generated results in a specific processing scenario. Historical feedback information can be obtained in real time by monitoring user interaction behavior on the file processing terminal interface, including but not limited to recording the user's text editing trajectory on the generated results in the visual editor, capturing the addition and deletion marks submitted by the user through revision mode, extracting version comparison difference data when the user uploads the corrected version file, or parsing user confirmation, rejection, and retry behavior records for specific processing tasks from the business system's operation logs.

[0026] Optionally, obtaining historical feedback information includes: recording the source identifiers of each knowledge fragment referenced when generating historical search enhancement instructions; locating the modified knowledge fragments through the source identifiers based on the user's modifications to the historical search enhancement instructions; generating feedback tags for the modified knowledge fragments; and generating historical feedback information based on the feedback tags.

[0027] The aforementioned source identification is a unique identifier assigned to each knowledge fragment in the knowledge base, used to establish a precise mapping relationship between the content of the retrieval enhancement command and the source of the knowledge. This identifier can take the form of the knowledge fragment's storage index number in the vector database, the hash digest value of the rule template, the regular expression library entry code, or the globally unique identifier (UUID) of the paragraph-level semantic unit, etc. It is persistently stored synchronously with the command content when generating historical retrieval enhancement commands, forming a reverse reference link from the command text to the knowledge fragment.

[0028] Feedback tags are classification labels used to determine the validity of specific knowledge fragments in historical processing instances, automatically labeled based on user corrections to the generated results. When a user modifies the correctly generated content, the corresponding knowledge fragment is marked as negative feedback, indicating that the fragment has produced misleading enhancements in the current processing scenario. When a user adopts the generated content without modification, it can be marked as positive feedback, indicating that the fragment highly matches the processing requirements. The tagging system can also be expanded to include dimensions such as modification magnitude level, modification type classification (e.g., replacement, deletion, appending), and confidence score to support multi-dimensional statistical analysis.

[0029] In terms of the technical implementation of locating the modified knowledge fragment, text difference comparison algorithms (such as the Longest Common Subsequence (LCS) algorithm, Myers difference algorithm, or similarity calculation based on edit distance) can be used to analyze the character-level differences between the user's modified content and the original generated instruction. Combined with the reference range information of the source identifier, the modified text fragment can be accurately associated with the original knowledge fragment identifier it depends on through coordinate mapping or semantic alignment techniques. It is understood that the above-mentioned text difference comparison algorithms are mature existing technologies, and their specific implementation methods and working principles can be found in the existing technologies, which will not be repeated in the embodiments of this application.

[0030] In the above scheme, the mechanism for generating historical feedback information is based on the aggregation and statistics of feedback tag data accumulated over a long period of time. By calculating the positive and negative feedback distribution frequency, cumulative modification frequency, and proportion of modified content for specific knowledge fragments under different processing request types, a quantitative indicator reflecting the granularity adaptability and task relevance of knowledge fragments is constructed, providing data support for the subsequent dynamic optimization and weight adjustment of knowledge fragments.

[0031] Step S130: Based on historical feedback information, determine the granularity adjustment requirements of relevant knowledge fragments for file processing requests, and adjust the granularity of the knowledge fragments to obtain the adjusted knowledge fragments.

[0032] The aforementioned granularity adjustment requirement index is a parameter metric used to quantitatively evaluate the granularity adaptation of a specific knowledge fragment under the current file processing request. Its value reflects the degree of deviation between the internal information composition of the knowledge fragment and the target processing requirement. When a knowledge fragment simultaneously contains effective information that positively promotes instruction generation and invalid information that causes the generated result to deviate from the user's expectations, this index indicates the degree of necessity for further refinement of the fragment.

[0033] Knowledge fragments refer to semantic information units extracted from external knowledge bases or reference materials to enhance the retrieval and instruction generation process. Their initial granularity may be initially defined based on fixed-length blocks, paragraph boundaries, or thematic continuity. Adjusted knowledge fragments are information units that have undergone granular optimization and reorganization. Their information boundaries are redefined according to the differentiated needs of specific processing scenarios to improve the semantic purity and task relevance of the fragment content.

[0034] In this application embodiment, the granularity adjustment requirement index can be determined through one of the following multiple implementation methods: The first implementation method is based on the inverse correlation between the frequency of user modifications and the magnitude of user modifications. Optionally, the above-mentioned determination of the granularity adjustment requirement indicators for relevant knowledge fragments of file processing requests based on historical feedback information includes: determining the user modification frequency and user modification magnitude of relevant knowledge fragments of file processing requests in the generation of historical enhancement instructions based on historical feedback information; and determining the granularity adjustment requirement indicators for relevant knowledge fragments based on user modification frequency and user modification magnitude.

[0035] The aforementioned user modification frequency refers to the percentage of times a user performs a modification operation on the enhanced instruction content generated for a knowledge fragment within a specific historical time window, targeting historical processing instances with the same processing requirements as the current file processing request (such as the same file format conversion target or the same information extraction dimension). User modification frequency can be obtained by statistically analyzing the ratio of the number of times a user triggers modification behavior to the total number of records for similar processing requests, used to quantify the prevalence of user dissatisfaction when processing this type of task. User modification magnitude refers to the percentage of text length, number of fields, or operation steps that a user modifies in a single historical processing instance for the enhanced instruction content corresponding to the knowledge fragment. User modification magnitude can be obtained by calculating the ratio of the length of the modified content to the total length of the original generated content, used to characterize the effectiveness of the information within the knowledge fragment and the degree of interference from other information.

[0036] The formula for calculating the granularity adjustment requirement index is: ; in, This represents the granular adjustment requirements (dimensionless) for the initial requirement instruction a and knowledge fragment b. This indicates the number of times (in units, dimensionless) a user modified the enhancement command containing knowledge fragment b in the history. This represents the total number (in units of times, dimensionless count) of all historical processing records that have the same processing requirements as the initial requirement instruction a. This represents the proportion (dimensionless, range of values) of the modified content of the enhancement instructions related to knowledge fragment b in the k-th modification record relative to all the content of the enhancement instructions related to knowledge fragment b. In the above formula, It represents the frequency of user dissatisfaction, and the unit is the dimensionless frequency ratio. This represents the cumulative percentage of content modifications across all the above modification records (dimensionless). This represents the inverse of the cumulative modification magnitude, where the summation term is a dimensionless proportional sum, hence the overall formula output is... As a dimensionless index, its physical meaning characterizes the degree of mismatch between the granularity of knowledge fragment b and the current file processing request. The larger the value, the more likely the fragment is to contain both effective information that positively promotes instruction enhancement and invalid information that causes the generated result to deviate from the expectation. In other words, the higher the degree of mixing of positive and negative feedback, the more granular refinement is needed to separate heterogeneous information.

[0037] In the above scheme, by monitoring the user's intervention patterns in the generated content of specific knowledge fragments in historical processing, the adaptation deviation between the fragment granularity and the current processing requirements is identified. When a high-frequency, small-amplitude modification pattern is detected, the knowledge fragment is determined to be in an overly coarse-grained state. The multiple semantic information contained within it interferes with each other in the processing scenario, requiring a segmentation operation based on semantic boundaries to separate heterogeneous information into different sub-fragments. This eliminates the interference of invalid information on instruction generation, improves the accuracy of subsequent automated processing, and enhances user satisfaction.

[0038] The second implementation method: expert rule determination based on structural features; In this implementation, the level of granularity adjustment requirement can be directly determined by using manually preset knowledge fragment quality scoring standards, combined with static structural features such as fragment length, information density, topic span, and document structure hierarchy. For example, when a knowledge fragment spans multiple semantic paragraphs and contains multiple data types, a high-granularity adjustment requirement index is automatically assigned.

[0039] The third implementation method: determination based on semantic vector analysis; In this embodiment, knowledge fragments can be divided into several sub-units and the semantic similarity distribution between sub-units can be calculated. When obvious semantic clustering is detected and the similarity between clusters is lower than a preset threshold, a granularity adjustment requirement index is automatically generated, indicating that the information units of different topics need to be segmented along the semantic boundary.

[0040] The aforementioned granularity adjustment is a technical process of structurally reorganizing knowledge fragments based on granularity adjustment requirement indicators. It aims to separate valid reference information from interfering information within the fragment by reducing or reconstructing the granularity of information units, thereby precisely aligning the semantic boundaries of the adjusted knowledge fragment with the specific file processing requirements. In the embodiments of this application, granularity adjustment can be performed through one of the following multiple implementation methods: The first implementation method: an iterative fine-grained segmentation mechanism based on semantic boundaries; Optionally, the above-mentioned granular adjustment of knowledge fragments to obtain adjusted knowledge fragments includes: responding to the granular adjustment requirement index meeting the preset adjustment conditions, performing fine-grained segmentation based on semantic boundaries on the relevant knowledge fragments to obtain intermediate knowledge fragments; determining the granular adjustment requirement index of the intermediate knowledge fragments based on historical feedback information; iteratively executing the fine-grained segmentation step and the granular adjustment requirement index determination step until the granular adjustment requirement index no longer meets the preset adjustment conditions, thereby obtaining the adjusted knowledge fragments.

[0041] The aforementioned semantic boundaries refer to the points where significant topic shifts or logical breaks occur within the semantic content of a knowledge fragment. These are typically manifested as locations where the semantic similarity between adjacent text units falls below a preset threshold, root node switching occurs in the dependency syntax structure, or hierarchical jumps occur in document format tags. Intermediate knowledge fragments refer to transitional information units generated during a single iteration of segmentation. These fragments have a smaller granularity than the original knowledge fragments but have not yet undergone final optimization and verification. They still require re-evaluation of the granularity adjustment requirement indicators to determine whether they meet the output conditions. Preset adjustment conditions refer to the criteria used to trigger or terminate granularity refinement operations. They are typically defined as granularity adjustment requirement indicators after normalization (the minimum-maximum normalization method can be used to normalize the granularity adjustment requirement indicators). Perform normalization to obtain If the value exceeds a certain threshold (such as 0.7), the threshold indicates that the heterogeneity of information within the knowledge fragment has reached a critical state where physical isolation is required.

[0042] The above scheme can use one of the following strategies to determine whether the preset adjustment conditions are met: The first strategy can use a fixed threshold method, directly applying the normalized granularity adjustment requirement index. The value is compared with a preset constant threshold (e.g., 0.7). The first strategy is to trigger the segmentation operation on a timed basis; the second strategy is to use a dynamic threshold determination method, which is based on the distribution of granularity adjustment requirements for all knowledge fragments in the current processing batch, and dynamically generates an adaptive threshold by calculating quantiles (such as taking the upper third) or by adding the standard deviation of historical statistics; the third strategy is to use a multi-condition joint determination method, which, in addition to considering the granularity adjustment requirements, also introduces the original length of the knowledge fragment, information density, or number of topics as auxiliary constraints, and only triggers the segmentation when multiple indicators simultaneously meet the preset range.

[0043] The above scheme can be implemented in one of the following ways to achieve fine-grained segmentation: (1) First implementation method: adopt a segmentation strategy based on embedding vector similarity mutation detection, use pre-trained language models (such as BERT, SimCSE) to encode knowledge fragments into dense vector sequences, calculate the cosine similarity between adjacent text windows, and determine the semantic boundary and perform segmentation when a local minimum point with similarity below the preset threshold is detected; (2) Second implementation method: adopt a structural boundary recognition method based on dependency parsing, and identify the position where there is a lack of grammatical dependency relationship or cross-sentence reference break between different sentences as the segmentation boundary by constructing the dependency parsing tree of the sentence; Third implementation method: adopt a hierarchical parsing method based on document structure tags, and directly use the inherent structural metadata of the document (such as title level tags, paragraph separators, list item tags or table cell boundaries) as the segmentation basis to ensure that the intermediate knowledge fragments after segmentation retain the structural integrity of the original document.

[0044] After obtaining intermediate knowledge fragments, their granularity adjustment requirements can be redefined, and several implementation methods can be adopted: The first method is the full historical data reuse method, which treats the intermediate knowledge fragments as new independent knowledge units and completely recalculates their granularity adjustment requirements based on user modification frequency and magnitude. The second method is the local feedback inheritance method, which, based on the historical modification records of the original knowledge fragments, inherits through content similarity matching or source identification, only selecting historical feedback data that overlaps with the content range of the intermediate knowledge fragments for local statistics to reduce computational overhead. The third method is the prediction and estimation method, which uses a trained regression model to directly predict its granularity adjustment requirements based on the semantic features of the intermediate knowledge fragments (such as topic consistency and information density), without performing a complete historical data traversal.

[0045] In the above solutions, a full historical data reuse method can be adopted, directly inheriting the historical feedback information calculation logic of the original knowledge fragment, treating the intermediate knowledge fragment as a new independent knowledge unit, and re-executing statistical calculations based on user modification frequency and magnitude. Alternatively, a partial feedback inheritance method can be used, inheriting from the historical modification records of the original knowledge fragment through content similarity matching or source identification, only selecting historical feedback data overlapping with the content range of the intermediate knowledge fragment for partial statistics to reduce computational overhead. Of course, a predictive estimation method can also be used, utilizing a trained regression model or classifier to directly predict the granularity adjustment requirement indicators based on the semantic features of the intermediate knowledge fragment (such as topic consistency, information density, and entity distribution), without performing a complete historical data traversal.

[0046] In the above scheme, by continuously monitoring the granularity adjustment requirements of intermediate knowledge fragments, high index values ​​are used as segmentation trigger signals to drive the semantic boundary recognition module to perform physical segmentation, generating smaller sub-units. Subsequently, by re-evaluating the index values ​​of the sub-units, an iterative closed loop of "evaluation-segmentation-re-evaluation" is formed. This iterative closed loop mechanism ensures that the granularity of knowledge fragments adaptively converges to the optimal balance point under specific processing requirements. This avoids the mixing of positive and negative feedback caused by excessive granularity and prevents semantic fragmentation and loss of contextual information caused by excessive segmentation, thereby achieving dynamic adaptation between the knowledge base structure and the actual needs of users in physical terms.

[0047] The second implementation method: a recursive decomposition method based on the document structure hierarchy; In this embodiment, the inherent hierarchical structure markers of the document (such as heading levels, paragraph boundaries, list item separators, or table cell divisions) can be directly used as the basis for segmentation to recursively decompose the original knowledge fragments and obtain sub-fragments with clear structural boundaries as adjusted knowledge fragments.

[0048] The third implementation method: an adaptive reorganization method based on topic models; In this embodiment, topic modeling algorithms such as Latent Dirichlet Allocation or BERTopic are used to detect the semantic topic distribution within knowledge fragments. When a significant multi-topic mixture is identified and the similarity between topics is below a preset threshold, a segmentation operation is performed along the topic boundaries to ensure that each adjusted knowledge fragment carries a single, pure semantic topic, thereby improving its adaptation accuracy to specific processing requests. It is understood that the aforementioned topic modeling algorithms are mature existing technologies; their specific implementation methods and working principles can be found in related technologies, and will not be elaborated upon in this embodiment.

[0049] Optionally, before determining the granularity adjustment requirement index of the knowledge fragments related to the file processing request based on historical feedback information, the method further includes: retrieving candidate knowledge fragments from the knowledge base based on the processing requirements of the file processing request; filtering out knowledge fragments with a negative feedback tag ratio exceeding a first preset ratio threshold based on the feedback tags of the candidate knowledge fragments recorded in the historical feedback information, and using the retained knowledge fragments as the knowledge fragments related to the file processing request.

[0050] Step S140: Based on historical feedback information, determine the irrelevant indicators between the adjusted knowledge fragment and the file processing request, and combine the contextual relevance between the adjusted knowledge fragment and the file to be processed to determine the reference value indicators of the adjusted knowledge fragment.

[0051] The irrelevance index between the adjusted knowledge fragment and the file processing request is used to characterize the degree of semantic deviation or task mismatch between a specific adjusted knowledge fragment and the processing requirements of the current file processing request. By aggregating user correction behavior patterns recorded in historical feedback information, the irrelevance index reflects the strength of the tendency for the knowledge fragment to trigger user intervention in similar past processing scenarios. Its value is negatively correlated with the actual reference value of the knowledge fragment to the current processing goal. A high irrelevance index indicates that the adjusted knowledge fragment frequently triggers substantial user corrections in historical enhancement instruction generation, and the correction content accounts for a large proportion. This suggests a structural deviation between the information elements contained within it and the processing logic of the file processing request. Using it as a reference for instruction enhancement will lead to results that deviate from user expectations. Conversely, a lower value indicates that the knowledge fragment has high semantic consistency with the processing requirements, and its information content can effectively support the achievement of automated file processing goals. As a key negative factor in subsequent reference value calculation, the value change of the irrelevance index directly affects the dynamic weight allocation of the knowledge fragment in the process of generating retrieval enhancement instructions, thereby achieving a data-driven evaluation of the task relevance of the knowledge fragment. In the embodiments of this application, one of the following multiple implementation methods can be used to determine the irrelevance index between the adjusted knowledge fragment and the file processing request: The first implementation method is determined based on the frequency of user modifications and the cumulative percentage of user-modified content. Optionally, the above-mentioned determination of the irrelevance indicators between the adjusted knowledge fragment and the file processing request based on historical feedback information includes: obtaining historical processing records that match the file processing request; determining the user modification frequency and the cumulative proportion of user modification content in the generation of the corresponding historical enhancement instructions based on the historical processing records; and determining the irrelevance indicators between the adjusted knowledge fragment and the file processing request based on the user modification frequency and the cumulative proportion of user modification content.

[0052] The aforementioned user modification frequency is a statistical measure characterizing the degree of user intervention triggered by a specific adjusted knowledge fragment in similar historical processing scenarios. It is defined as the ratio of the frequency of user modifications to the knowledge fragment in historical processing records with the same processing requirements as the current file processing request to the total number of historical processing records. The cumulative percentage of user-modified content is a cumulative measure characterizing the overall negative intensity of the generated content of the knowledge fragment by users. It is defined as the sum of the proportions of the modified content of the relevant content to the total content of each generation enhancement instruction in all historical records of modifications to the knowledge fragment. The irrelevance index characterizes the degree of irrelevance between the adjusted knowledge fragment and the file processing request. Its value is positively correlated with the user modification frequency and the cumulative percentage of user-modified content; that is, the higher the index value, the greater the deviation of the knowledge fragment from the current processing requirements, and the lower its effectiveness as a reference.

[0053] The above scheme can obtain historical processing records that match the file processing request in the following ways: (1) First method: use exact matching method. This method requires that the processing target description, processing type identifier and input file domain attributes of the historical processing record are completely consistent with the current request. Only historical records that meet the exact matching condition are included in the statistics. (2) Second method: use semantic similarity matching method. This method uses text embedding technology to calculate the vector similarity between the current processing request and the description of the historical processing request. Historical records with similarity exceeding the preset threshold are regarded as the same scenario and included in the statistics to expand the sample size. (3) Third method: use hierarchical classification matching method. This method is based on the preset processing task classification system (such as information extraction, format conversion, content cleaning, etc. first-level classification and sub-type second-level classification). As long as the historical record and the current request are in the same classification node, it is determined to be a match, taking into account both statistical accuracy and recall rate.

[0054] The above methods for obtaining user modification frequency can be: (1) First method: using the simple frequency method, directly calculating the ratio of the absolute number of times the adjusted knowledge fragment was modified in the matching historical processing records to the total number of historical processing records. (2) Second method: using the time-weighted frequency method, assigning higher time decay weight to recent historical records and lower weight to older records, and calculating the corrected modification frequency by weighted average to reflect the performance trend of knowledge fragments in recent processing scenarios. (3) Third method: using the confidence-weighted method, assigning differentiated weights according to the feedback reliability level of historical processing records (e.g., the weight of manually confirmed records is higher than that of automatically captured records), and calculating the weighted modification frequency.

[0055] The above methods for obtaining the proportion of modified content can be: (1) The first method: using the character-level edit distance ratio method, calculating the number of characters that differ between the user's modified text and the original generated text based on the Levenshtein distance, dividing it by the total number of characters of the original content generated by the knowledge fragment in the corresponding history, and then summing the single modification ratio. (2) The second method: using the semantic unit ratio method, dividing the enhanced instructions into semantic units such as words, phrases or sentences, and accumulating the proportion of the number of modified semantic units to the total number of semantic units. (3) The third method: using the difference block volume ratio method, identifying the changed continuous text blocks based on the text difference comparison algorithm, calculating the proportion of the changed block character volume to the total volume of the original content and accumulating it.

[0056] The formula for calculating the above irrelevant indicators is as follows: ; in, This indicates the number of times (in units, dimensionless) a user modified the enhancement command containing knowledge fragment b in the history. This represents the total number (in units of times, dimensionless count) of all historical processing records that have the same processing requirements as the initial requirement instruction a. This represents the proportion of the enhanced instructions related to knowledge fragment b after the adjustment in the k-th modification record, out of the total enhanced instructions in that record. It is a dimensionless proportion with a theoretical range of [value missing]. ; This represents the cumulative percentage of content modifications made to the knowledge fragment across all modification records, expressed as a dimensionless cumulative value. The physical meaning of the above formula lies in establishing a quantitative correlation between user dissatisfaction and information deviation. When the adjusted knowledge fragment is highly relevant to the file processing request, the generated enhancement instructions should require less user modification, manifested as... Smaller value or The value is small, resulting in The value approaches zero; conversely, when a knowledge fragment frequently triggers modifications and each modification involves a significant amount of text, it indicates a structural deviation between the information elements it contains and the processing requirements. The value increases accordingly. This is understandable. As a dimensionless index, the larger the value, the higher the degree of irrelevance to the processing requirements.

[0057] In the above scheme, the degree of user intervention in the generation of specific knowledge fragments during historical processing is monitored to identify the adaptation deviation between fragment information and processing objectives. High-frequency, large-scale modification patterns indicate a misalignment between the processing logic of the knowledge fragment and the current task requirements. Based on this, a higher irrelevant index value is assigned, thereby reducing the weight allocation of that fragment in subsequent reference value calculations. This avoids introducing information that deviates from user needs into the generation process of current search enhancement instructions, improving the output quality and user satisfaction of automated document processing.

[0058] The second implementation method: a weighted evaluation strategy based on time decay is used to determine the evaluation strategy. In this implementation, a time-weighted factor is introduced when calculating the frequency and proportion of user modifications. More recent historical processing records are given a higher weight, while older historical records are given a lower weight after attenuation. This weighted calculation yields time-sensitive modification frequencies and cumulative proportions, thereby identifying irrelevant indicators that reflect the recent performance trend of knowledge fragments and avoiding interference from early historical noise in the current evaluation.

[0059] The third implementation method: Determined based on a classification metric strategy for modifying operation types; In this implementation, in addition to statistically analyzing the frequency and scale of modifications, the type of user modification operations is further differentiated, such as classifying modifications into deletion, replacement, or append operations. By statistically analyzing the frequency distribution and content proportion of different types of operations, a multi-dimensional relevance assessment vector is constructed. For example, when a knowledge fragment is detected to primarily trigger deletion operations rather than replacement operations, a higher irrelevance index value is assigned, indicating that the content of that fragment has a stronger disruptive effect on the current processing requirements rather than being correctable, thereby providing a more refined basis for relevance determination.

[0060] The degree of contextual relevance between the adjusted knowledge fragment and the file to be processed is a quantitative measure of the semantic fit between the granularly optimized knowledge fragment and the current content of the file to be processed. Its value reflects the semantic proximity of the information content carried by the knowledge fragment to the current context of the file to be processed. The degree of contextual relevance is constructed based on the principle of semantic similarity. By measuring the geometric proximity between the semantic feature vector of the knowledge fragment and the context feature vector of the file to be processed, it quantifies the contextual adaptability of a specific knowledge fragment in the current processing scenario. When this measure shows a high value, it indicates that the topic content, entity distribution, or logical structure of the knowledge fragment has a high degree of semantic overlap with the current paragraph of the file to be processed. This fragment, as a reference, can effectively utilize the contextual information of the current file to generate targeted processing instructions. Conversely, a low value indicates that the knowledge fragment deviates semantically from the current processing context, and its reference value should be reduced accordingly. This degree of relevance, as a key positive factor in dynamic weight allocation, together with the irrelevant indicators representing task deviation, constitutes a two-dimensional basis for reference value assessment, ensuring that the generation of enhanced retrieval instructions both aligns with historical experience feedback and fits the specific contextual features of the current file. In the embodiments of this application, one of the following multiple implementation methods can be used to determine the degree of contextual association between the adjusted knowledge fragment and the file to be processed: The first implementation method: Determining the dense vector encoding and cosine similarity calculation strategy based on the pre-trained language model; In this implementation, a bidirectional encoder representation model (such as BERT) or a sentence contrast learning model (such as SimCSE) can be used to encode the context text of the file to be processed and the adjusted knowledge fragments into dense vector representations of fixed dimensions. The degree of association is obtained by calculating the cosine similarity between the two vectors. Cosine similarity reflects the consistency of semantic direction by measuring the cosine value of the angle between two vectors in the vector space. Its value range is [-1, 1]. In practical applications, after normalization, it is mapped to the interval [0, 1]. The closer the value is to 1, the more similar the semantic connotation of the context and the knowledge fragment are, and the higher the degree of association. The value is close to 0, indicating that the two are semantically orthogonal or unrelated. This implementation fully utilizes the ability of pre-trained language models to capture deep semantic associations and can effectively identify content matching with different literal expressions but similar semantics.

[0061] The second implementation method is to determine the statistical similarity based on sparse vector representation. In this implementation, the context and knowledge fragments can be represented as high-dimensional sparse vectors using the Term Frequency-Inverse Document Frequency (TF-IDF) algorithm or the Bag-of-Words model. Each dimension of the vector corresponds to a specific term in the vocabulary, and the dimension value reflects the importance weight of that term in the text. Subsequently, cosine similarity or Jaccard similarity coefficient is used to calculate the degree of overlap between the two sparse vectors, and the consistency of the distribution of co-occurring terms is used to quantify the degree of association.

[0062] The third implementation method: using a probability distribution similarity strategy based on topic models to determine similarity; In this implementation, topic models such as Latent Dirichlet Allocation (LDA) or Probabilistic Latent Semantic Analysis (pLSA) can be used to infer the probability distribution vectors of context and knowledge fragments in a predefined topic space. The degree of difference in topic distributions is measured by calculating the Kullback-Leibler Divergence or Hellinger distance between the two distribution vectors, and then converted into an index of relevance. When two texts have similar probability quality distributions in the same topic dimension, it indicates that their discussion areas or core issues are highly consistent, and the degree of relevance is thus enhanced.

[0063] It is understood that the bidirectional encoder representation model, sentence contrast learning model, word frequency-inverse document frequency algorithm, bag-of-words model, etc. mentioned above are all mature existing technologies. For the specific implementation methods and working principles of each technology, please refer to the relevant technologies. The embodiments of this application will not be repeated here.

[0064] Optionally, the above-mentioned determination of the reference value index of the adjusted knowledge fragment by combining the degree of contextual relevance between the adjusted knowledge fragment and the file to be processed includes: determining the degree of contextual relevance between the adjusted knowledge fragment and the file to be processed; determining the reference value index of the adjusted knowledge fragment based on the degree of contextual relevance and irrelevant indicators; wherein the reference value index increases with the increase of the degree of contextual relevance and decreases with the increase of irrelevant indicators.

[0065] The aforementioned reference value index is a quantitative evaluation value constructed by comprehensively considering the semantic fit between the adjusted knowledge fragment and the context of the file to be processed, as well as the risk of task deviation between the knowledge fragment and the processing requirements of the file processing request. It is used to characterize the comprehensive utility of the knowledge fragment in generating retrieval enhancement instructions under a specific processing scenario. This index integrates the degree of contextual relevance. and unrelated indicators The information in two dimensions dynamically quantifies the contextual adaptability and task relevance of knowledge fragments, and the magnitude of the value directly determines the reference priority and weight allocation ratio of the fragment in the subsequent instruction generation process.

[0066] The above solution can be implemented by fusing the degree of contextual relevance in one of the following ways. and unrelated indicators Information in two dimensions: (1) The first approach: adopting a nonlinear fusion model based on the reciprocal of the product. This model constructs a reference value index by multiplying the product of the complement of the degree of contextual relevance and the reciprocal of the unrelated index. It uses the reciprocal operation to establish a monotonic mapping relationship between the index value and the input parameters, ensuring that the reference value monotonically increases with the improvement of contextual fit and monotonically decreases with the improvement of task deviation risk. (2) The second approach: adopting a fusion strategy based on weighted harmonic average, treating the degree of contextual relevance as a positive contribution factor and the complement of the unrelated index (i.e. (3) The third approach: A fusion model based on logarithmic linear weighting can be used. After taking logarithmic transformations of contextual relevance and unrelated indicators, differential weight coefficients are applied and linearly combined. The reference value indicators are obtained by restoring them through exponential operations, so as to achieve sensitivity adjustment of different risk level segments.

[0067] The calculation formula for the above reference value indicators is as follows: ; in, This indicates the reference value of the adjusted knowledge fragment b to the file processing request a. This indicates the degree of relevance between the adjusted knowledge fragment b and the context of the file to be processed; This represents the relevance index between the adjusted knowledge fragment b and the file processing request a (i.e., the normalized degree of processing irrelevance), with a value closer to 0 indicating higher task relevance. In the above calculation formula, A positive gain factor based on contextual association is formed when a knowledge fragment is highly consistent with the current file context. When the value approaches 1, the denominator approaches zero, and the value of this term increases sharply, significantly amplifying its reference value; when the contextual relationship is weak ( When the value approaches 0, the term approaches 1, and the gain effect is weak. A negative inhibitory factor based on historical relevance is formed when knowledge fragments are highly correlated with processing requirements. When the value approaches 0 (indicating low correlation), this value increases sharply, increasing its reference value weight; when historical data shows high correlation (…), the value of this item increases dramatically, increasing its reference value weight. When the value approaches 1, the term approaches 1, indicating a significant inhibitory effect. The product effect of the two factors ensures that only knowledge fragments with both strong contextual adaptability and strong task relevance can obtain high reference value scores; the absence of either dimension will lead to a decrease in the index value.

[0068] In calculating and obtaining reference value indicators Then, the min-max normalization method can be used to... After normalization, we get Its value range is [0,1]. This is obtained by combining all refined knowledge fragments corresponding to the multiple knowledge fragments initially obtained through expert analysis related to the initial requirement instruction a. Summation is performed to refine the knowledge fragments individually. The ratio of b to s is used as a reference weight for enhancing the user's initial requirement instruction a by refining knowledge fragment b.

[0069] In the above scheme, by using the degree of contextual relevance as a positive amplifier and the irrelevant index as a negative attenuator for nonlinear fusion, the joint constraint of the scenario sensitivity and task accuracy of knowledge fragments is realized. This ensures that the knowledge fragments that finally enter the retrieval enhancement instruction generation process are both consistent with historical experience verification and accurately adapted to the current processing context, thereby improving the output quality and user satisfaction of automated document processing.

[0070] Step S150: Based on the reference value index, generate retrieval enhancement instructions corresponding to the file processing request.

[0071] The aforementioned enhanced retrieval instructions are a structured instruction set generated based on knowledge retrieval results to guide the execution of automated file processing operations. Their content integrates the user's intended file processing request with the processing patterns, rule logic, or operation templates carried by reference knowledge fragments retrieved from the knowledge base. Enhanced retrieval instructions can adopt various forms to adapt to different execution environments and application needs. For example, they can be presented as natural language step descriptions to support manual review and confirmation, as executable programming code to support automatic parsing and execution, or as structured data exchange formats to support cross-platform interface calls and workflow orchestration. The core feature of enhanced retrieval instructions lies in their generation process, which integrates external knowledge retrieval with the semantic generation capabilities of a large language model. This retains the historical processing experience and standardized rules accumulated in the knowledge base while achieving personalized customization for specific file processing scenarios through semantic understanding and context adaptation, thus achieving a balance between accuracy, flexibility, and interpretability in automated file processing.

[0072] Optionally, step S150 includes: filtering and sorting the adjusted knowledge fragments based on reference value indicators; assembling the filtered and sorted adjusted knowledge fragments with the context of the file processing request and the file to be processed into an enhanced prompt; and generating a search enhancement instruction corresponding to the file processing request based on the enhanced prompt.

[0073] Filtering and ranking adjusted knowledge fragments based on reference value indicators is a technical process that optimizes the knowledge input set through quality threshold control and priority orchestration. The filtering operation involves retaining a subset of candidate knowledge fragments that meet specific quality standards based on the numerical distribution characteristics of the reference value indicators, thereby eliminating low-value noise and controlling the computational load of subsequent processing. The ranking operation involves arranging the retained knowledge fragments in an ordered manner according to the numerical value of the reference value indicators to establish their priority order in the subsequent instruction generation process. Enhanced prompts are complete contextual texts used to input into a large language model, formed by structurally integrating the filtered and ranked knowledge fragments with the user's intent and the current context. The content organization of these prompts directly affects the accuracy and relevance of the generated instructions.

[0074] The above scheme can be implemented using one of the following strategies: (1) First strategy: Use a fixed threshold screening method, set a lower threshold for the reference value index, and retain only knowledge fragments whose index values ​​exceed the threshold to ensure that all fragments involved in subsequent generation meet the basic quality requirements. (2) Second strategy: Use a dynamic Top-K truncation method, based on the descending order of the reference value index, and retain only the top preset number (e.g., K=5) of knowledge fragments to achieve fixed-scale knowledge input control. (3) Third strategy: Use a cumulative contribution screening method, calculate the cumulative distribution of the reference value index, and stop screening when the cumulative value reaches the preset coverage rate (e.g., the top 80%) to ensure the complete retention of high-value information while controlling the number of fragments.

[0075] The above scheme can be implemented using one of the following strategies: (1) First strategy: a single indicator descending order method can be used, directly sorting from high to low according to the value of the reference value indicator, so that the highest value segment is placed in the highest priority position. (2) Second strategy: a multi-factor composite sorting method can be used, introducing a time decay factor or diversity constraint on the basis of the reference value indicator, and obtaining a comprehensive sorting score through weighted calculation, avoiding over-reliance on historical high-frequency segments and ignoring emerging patterns. (3) Third strategy: a hierarchical sorting method can be used, first dividing the knowledge segments into high, medium and low levels according to the reference value indicator, and then refining the sorting within each level according to other auxiliary indicators (such as segment length and information density) to form a hierarchical priority structure.

[0076] When assembling enhanced prompts, template filling can be used. This involves using a predefined prompt template and filling the designated slots with the file processing request, the context of the file to be processed, and the filtered and sorted knowledge fragments, creating a uniformly formatted enhanced prompt. Alternatively, tag-based separation can be used, employing specific delimiters or XML tags to explicitly distinguish information from different sources, helping the large language model identify the functional roles of each part. Furthermore, weighted annotation can be used, explicitly annotating the prompt text with the reference value index or relative weight level of each knowledge fragment, guiding the large language model to assign higher attention weights to high-value fragments during the generation process.

[0077] When generating retrieval enhancement instructions based on enhancement prompts, the assembled enhancement prompts can be input into the large language model all at once, requiring it to directly output the complete retrieval enhancement instruction text. Alternatively, specific format constraints or rules can be attached to the prompts (such as the output must be in JSON format or must contain specific fields), requiring the large language model to follow structured constraints during the generation process. Furthermore, a multi-round iterative refinement method can be employed. First, a preliminary instruction draft is generated. Then, defects are identified through self-verification or human feedback, and the model is called again to iteratively optimize based on the corrected prompts until a final instruction that meets quality requirements is generated.

[0078] The above scheme fundamentally prevents low-quality knowledge fragments from interfering with the generation process by using reference value indicators for screening; it ensures that high-value fragments occupy a priority position in the prompts through ranking, and leverages the attention mechanism of the large language model to give them higher decoding weights; through structured prompt assembly, it integrates discrete knowledge units with user intent and the current document context into a coherent semantic input space, enabling the model to perform accurate intent understanding and knowledge fusion with rich contextual support; finally, through the generation capabilities of the large language model, it transforms the structured prompts into executable retrieval enhancement instructions, realizing an end-to-end automated process from knowledge retrieval to instruction output.

[0079] Optionally, after step S150 above, the method further includes: performing a weight self-check on the search enhancement instruction, and regenerating the search enhancement instruction in response to the fact that the proportion of knowledge fragment references with low reference value indicators exceeds a second preset proportion threshold.

[0080] The aforementioned weight self-verification is a compliance verification mechanism executed after the retrieval enhancement instruction is generated. It aims to verify whether the generated result effectively follows the knowledge fragment priority ranking established by the reference value index. This mechanism quantitatively analyzes the content composition of the generated instruction, identifying the distribution of knowledge fragment sources cited within, particularly detecting whether knowledge fragments with low reference value indices occupy an excessively high proportion of the generated result. The proportion of knowledge fragments cited with low reference value indices is defined as the percentage of the semantic content of the complete retrieval enhancement instruction whose semantics can be traced back to adjusted knowledge fragments with reference value indices below a specific threshold. This proportion reflects the degree of dependence of the generated result on low-quality reference information. The second preset proportion threshold is a critical boundary used to trigger the regeneration process. When the proportion of low-value fragments cited exceeds this boundary, it indicates that the initial generation process failed to effectively suppress the interference of low-quality information, requiring the initiation of a correction process.

[0081] When performing weight self-verification, one of the following strategies can be used to calculate the proportion: The first strategy is to use a precise calculation method based on source identification. By parsing the generation log of the enhanced instruction or calling the attribution analysis interface of the large language model, the original knowledge fragment identifiers referenced by each statement in the instruction content can be identified, and the proportion of statements or characters originating from low reference value index fragments can be counted to the total output.

[0082] The second strategy is to use a semantic similarity-based inference calculation method. This method uses vector encoding technology to compare the similarity of the generated instruction content paragraph with each adjusted knowledge segment, assigns the instruction paragraph to the knowledge segment with the highest similarity, and then calculates the equivalent citation ratio of low-value segments.

[0083] The third strategy is to use an attention weight-based estimation method. During the generation process of a large language model using the Transformer architecture, the attention distribution data of the model to each knowledge fragment in the input prompt can be extracted. By normalizing the attention weights, the contribution ratio of each fragment to the generated result can be estimated, thereby determining whether low-value fragments are over-attentive.

[0084] The above solutions can employ a fixed threshold method to determine whether to trigger regeneration. This involves directly comparing the calculated low-value fragment citation ratio with a preset second ratio threshold; regeneration is triggered immediately when the former exceeds the latter. Alternatively, a trend-based method can be used. This involves recording the trend of low-value fragment citation ratio changes across multiple generation attempts, triggering regeneration only when the ratio shows a sustained increase and exceeds a threshold, thus avoiding excessive retries caused by single statistical fluctuations. Furthermore, a multi-level threshold method can be used to determine whether to trigger regeneration. This involves setting two levels of boundaries: a warning threshold and a mandatory threshold. When the ratio exceeds the warning threshold, a partial correction is attempted first; only when the mandatory threshold is exceeded is a full regeneration process executed.

[0085] During regeneration, constraints can be explicitly added to the enhanced prompts to force the large language model to prioritize knowledge fragments with high reference value, or to explicitly prohibit the use of specific low-value fragments. Alternatively, a filtering and truncation method can be used to directly remove knowledge fragments with low reference value from the input prompts, retaining only high-value fragments for regeneration, thus physically blocking the input of low-value information. Of course, a prompt reordering method can also be used to adjust the order of knowledge fragments in the enhanced prompts, placing high-value fragments at the beginning to leverage the positional preference effect of the large language model, while simultaneously increasing the frequency of repetition of high-value fragments to strengthen their influence weight.

[0086] Because the generation process of large language models is probabilistic and emergent, even with clear reference value indicators and ranking information, the model may still over-rely on low-value segments during decoding due to random fluctuations in attention distribution or interference from training priors. The above solution introduces weighted self-verification as a post-generation verification step, which can identify and intercept such priority deviations before instruction output. It converts the conformity of generated quality into a measurable numerical judgment through a quantified proportion indicator and implements automated quality gating based on a preset threshold. When a deviation is detected, regeneration is performed by adjusting the composition or constraints of the input prompt, forming a closed-loop control of generation-verification-correction. This ensures that the final output retrieval enhancement instruction faithfully reflects the priority ranking established by the reference value indicators in terms of content composition, improving the reliability and consistency of automated file processing.

[0087] Step S160: Perform automated file processing on the file to be processed based on the search enhancement instructions.

[0088] The aforementioned automated file processing refers to the technical process of transforming, analyzing, or reorganizing input files without human intervention, based on the operational logic defined by retrieval enhancement instructions. This process encompasses various operation types, including data cleaning, information extraction, format standardization, content classification, and privacy anonymization. Essentially, it transforms high-level semantic instructions into low-level data operations that are applied to the file content. The execution context refers to the technical environment in which automated file processing runs, including infrastructure configurations such as the file parsing engine, memory buffer, temporary storage space, and exception handling mechanisms. These configurations collectively ensure that retrieval enhancement instructions can be accurately parsed and securely executed.

[0089] The above solution can be implemented in various ways to automate document processing, adapting to different security and performance requirements: The first implementation method employs a code interpretation and execution mode, suitable for retrieval enhancement commands presented in scripting languages ​​such as Python. It interprets and executes commands line by line by invoking the language runtime environment, providing dynamic type checking and runtime debugging support, thus adapting to scenarios with complex logic and dynamic computation requirements.

[0090] The second implementation method employs a structured operation sequence execution mode, suitable for natural language steps or operation sequences defined in JSON format. By parsing the operation type identifier and parameter mapping in the instructions, it calls a pre-built operation template library to execute atomic operations sequentially, suitable for batch processing tasks with a high degree of standardization.

[0091] The third implementation method is to adopt a compilation execution mode. For performance-sensitive tasks, the retrieval enhancement instructions are first compiled into machine code or intermediate bytecode, and then the compiled executable file is run in a controlled sandbox environment, sacrificing some flexibility for higher processing throughput.

[0092] The fourth implementation method adopts a workflow orchestration mode, which decomposes the retrieval enhancement instruction into several atomic operation nodes, and schedules them to be executed in the order of dependencies through a predefined workflow engine. It supports node-level retries and state persistence, and is suitable for complex processing pipelines that require multi-stage collaboration.

[0093] Furthermore, when performing automated file processing, transactional processing mechanisms and parallel execution strategies can be introduced. For transactional processing, when a retrieval enhancement command contains multiple related operations, atomic transactions can be used for encapsulation. The processing result is persisted to the file only when all sub-operations complete successfully. If an exception occurs at any stage, a rollback operation is triggered, restoring the file to its pre-processing state and preventing data corruption caused by partial execution. For parallel execution, in batch file processing scenarios, based on the independent characteristics of the files to be processed, multiple files can be allocated to different execution threads or distributed computing nodes for concurrent processing. Horizontal scaling of computing resources improves overall processing throughput, while a result aggregation mechanism ensures that the processing results output by each independent execution unit are correctly summarized in the original request order.

[0094] Retrieval enhancement instructions, as high-level abstract semantic representations, contain semantic information such as processing targets, operational logic, and reference rules, but do not directly possess the ability to physically manipulate file content. The aforementioned scheme transforms semantic instructions into specific execution plans through an instruction parsing layer. These plans define the byte-level or character-level operation sequence for the file to be processed. Subsequently, the file operation engine directly accesses the memory mapping or streaming buffer of the file to be processed according to the execution plan, performing physical data read, write, and conversion operations. Finally, the processed content is output as a new file version or the original file is overwritten, thus completing an end-to-end automated process from semantic intent to physical state change.

[0095] Optionally, it also includes: after performing automated document processing, recording the source identifiers of each knowledge fragment referenced by the generated search enhancement instruction; obtaining the user's modification records of the automated document processing results; and based on the modification records and source identifiers, determining the knowledge fragments that have been modified and updating the historical feedback information.

[0096] The aforementioned source identification is a unique identifier assigned to a knowledge fragment within a specific processing instance. This identifier is used to accurately trace the knowledge source upon which the generation instruction relies after automated document processing is completed. In this processing flow, the identifier is bound to the generation instance of the retrieval enhancement instruction, forming the index basis for subsequent feedback attribution. User modification records refer to the substantive corrections performed by the user on the output content after reviewing the results of this automated document processing, along with the corresponding changed text data. This record carries the user's evaluation information on the current processing quality. Feedback information updates refer to the process of transforming user intervention behaviors in this processing instance into structured historical feedback data and appending it to the cumulative feedback database, thereby enabling the continuous evolution of the knowledge fragment quality assessment model.

[0097] For recording the source identifiers of each knowledge fragment referenced by the generated retrieval enhancement command, a transactional write method similar to that of a relational database can be used to persist the mapping relationship between command instance identifiers and knowledge fragment identifiers to the associated table to ensure data consistency. Alternatively, an append-only write method using distributed logs can be used to record the reference relationship in a structured log format, supporting high-concurrency write throughput. Another option is to use a combination of memory caching and asynchronous disk flushing, first temporarily storing the reference relationship in a high-speed cache, and then asynchronously persisting it to the backend storage in batches to balance real-time performance and overhead.

[0098] To obtain the user's modification record of the automated file processing results, the operation monitoring method of the front-end rich text editor can be adopted, which captures the user's real-time editing action sequence through DOM change events or collaborative editing protocols; alternatively, a file version comparison method can be adopted, which calculates the changes relative to the output version through difference comparison algorithms after the user saves the corrected version on the local terminal; or an explicit feedback submission method can be adopted, which provides a dedicated interactive interface for users to upload the corrected file and mark the modification type label.

[0099] For knowledge fragments modified based on modification records and source identifiers, a text coordinate mapping method can be used. This involves locating the character coordinate range of the changed text within the original generated content using a difference comparison algorithm, and then retrieving the source identifier of the knowledge fragment corresponding to that coordinate range. Alternatively, a semantic similarity backtracking method can be used. This involves calculating the semantic similarity between the user-modified text fragment and each of the original referenced knowledge fragments, attributing modifications with similarity below a threshold to specific knowledge fragments. Another method is operation chain tracing. This involves recording the contribution ratio matrix of each knowledge fragment to the output content during the generation process, and then using matrix operations to infer the dependency weight of the modified content on each knowledge fragment.

[0100] For updating historical feedback information, an incremental appending method can be used, directly appending the feedback tags of the current processing instance to the end of the historical feedback list for that knowledge fragment; a sliding window update method can also be used, retaining only a fixed number of recent feedback records and updating statistical indicators to reflect the timeliness changes in user preferences; or a model retraining trigger method can be used, triggering the background statistical model to recalculate the granularity adjustment requirements and irrelevant indicators for each knowledge fragment when the accumulated amount of new feedback data reaches a preset threshold.

[0101] In the above scheme, a traceable path from output to input knowledge is established by fully recording the knowledge reference chain after each processing step. When a user modifies the output, difference analysis attributes the modifications to specific knowledge fragment sources, thus accurately mapping the user's evaluation of the final result to the judgment of the validity of specific knowledge fragments. This attribution-based feedback label is injected into the historical feedback information database in real-time or near real-time, enabling subsequent processing requests to perform statistical calculations based on an expanded dataset containing this experience when evaluating the same or similar knowledge fragments. This closed-loop mechanism ensures that the above scheme has self-evolution capabilities, continuously optimizing the granular structure and weight allocation strategy of knowledge fragments as processing instances accumulate, achieving a gradual improvement in the accuracy of automated document processing.

[0102] To facilitate understanding of the working principle of the intelligent retrieval enhancement instruction generation system for automated file processing described above, this application also provides specific application examples in a certain application scenario. For example... Figure 2 As shown, in this application scenario, the intelligent retrieval enhancement instruction generation system for automated file processing mainly includes: Step 1: Obtain the file processing request for the file to be processed and retrieve historical feedback information; The system receives file processing requests for documents to be processed (such as contract texts, order emails, or business reports) and performs intelligent block segmentation on the documents, preserving structural information (such as titles, paragraphs, lists, and tables). For long texts, a semantic block segmentation strategy (based on embedding similarity or document structure parsing) is used to divide the file into several text segments, while extracting file-level metadata (filename, source, language, domain). The system parses the processing goal defined in the file processing request (such as "cleaning personal privacy information" or "extracting dates and amounts from contracts"), uses the current text segment and the processing goal as a composite query, and retrieves relevant processing patterns from the knowledge base (such as regular expression libraries, field mapping rules, text manipulation templates, and historical success cases). The retrieval uses a hybrid search (keyword matching + vector similarity) and filters by processing type.

[0103] The retrieved knowledge fragments are fused using a large language model to generate structured search enhancement instructions (which can be natural language steps, Python code, or JSON format operation sequences). During generation, the source identifiers of each referenced knowledge fragment (such as vector library index numbers and rule template hash values) are recorded. The generated instructions undergo static validation (such as regular expression syntax checking and field existence verification), and are tested on text samples in a sandbox environment. If the test results deviate significantly from expectations, a secondary search correction is triggered. After validation, the search enhancement instructions are applied to the target text, performing automated file processing (such as personal information anonymization and key field extraction).

[0104] The system acquires user modification records of automated file processing results, as well as historical modification records of historical automated file processing results, forming historical feedback information. Specifically, a difference comparison algorithm is used to locate the atomic knowledge fragments corresponding to the user's modifications. Based on the traceability identifier of the record, the modified knowledge fragment is determined, and a feedback label is generated for the modified knowledge fragment (e.g., marked as negative when the user modifies the correct content, and marked as positive when the user adopts the content without modification). The feedback labels are associated with the traceability identifier and stored to form a historical feedback information database.

[0105] Step 2: Determine the granularity adjustment requirements and perform granularity adjustment, identifying irrelevant indicators and indicators with reference value; Based on historical feedback information, the granularity adjustment requirements for knowledge fragments related to file processing requests are determined, and the knowledge fragments are adjusted to obtain the adjusted knowledge fragments. Specifically, historical processing records matching the processing requirements of file processing requests are obtained, and the frequency of modification of knowledge fragments in these historical processing records and the cumulative content modification ratio of these knowledge fragments in the corresponding historical enhancement instructions are statistically analyzed. Based on the inverse correlation between user modification frequency and cumulative content modification ratio, granularity adjustment requirements are determined: ; Using the maximum-minimum normalization method After normalization, we get Its range is [0,1]. When When the value exceeds the preset adjustment condition (e.g., 0.7), fine-grained segmentation based on semantic boundaries is performed on the knowledge fragment (e.g., using a pre-trained language model to detect semantic similarity mutation points or segmentation based on document structure tags) to obtain intermediate knowledge fragments; then, the granularity adjustment requirement index of the intermediate knowledge fragment is re-determined based on historical feedback information, and the above segmentation and determination steps are iteratively executed until the granularity adjustment requirement index no longer meets the preset adjustment condition, and the finally obtained intermediate knowledge fragment is taken as the adjusted knowledge fragment.

[0106] Furthermore, based on historical feedback information, irrelevant metrics between the adjusted knowledge fragments and file processing requests are identified. Specifically, historical processing records matching file processing requests are obtained to determine the frequency of user modifications to the adjusted knowledge fragments and the cumulative percentage of user modifications in the corresponding historical enhancement instruction generation. Based on the user modification frequency and the cumulative percentage of user modifications, irrelevant metrics are determined: ; Using the maximum-minimum normalization method After normalization, we get Its range is [0,1].

[0107] Subsequently, the degree of contextual relevance between the adjusted knowledge fragment and the file to be processed is determined. Based on this degree of contextual relevance and irrelevance metrics, a reference value index for the adjusted knowledge fragment is determined. Specifically, a pre-trained language model (such as BERT or SimCSE) is used to encode the context of the file to be processed and the adjusted knowledge fragment into dense vectors, and the cosine similarity is calculated to obtain the degree of contextual relevance. (Value range [0,1], larger values ​​indicate stronger correlation). Based on the degree of contextual correlation and normalized irrelevant indicators. Determine reference value indicators : ; Using the maximum-minimum normalization method After normalization, we get Its value range is [0,1], which serves as the reference weight for the knowledge fragment in the generation of subsequent instructions.

[0108] Step 3: Generate search enhancement instructions and execute automated file processing to achieve closed-loop feedback updates; Based on reference value indicators, retrieval enhancement instructions corresponding to the file processing request are generated. Specifically, the adjusted knowledge fragments are filtered (e.g., retaining knowledge fragments that meet preset conditions) and sorted based on the reference value indicators. The filtered and sorted knowledge fragments are then assembled with the file processing request and the context of the file to be processed into an enhancement prompt (e.g., using XML tags to explicitly distinguish different types of information using a tag-based separation method). Based on this enhancement prompt, a large language model is used to generate retrieval enhancement instructions (which can be natural language steps, Python code, or JSON format operation sequences).

[0109] Optionally, after generating the search enhancement instruction, a weight self-check can be performed on the instruction to verify whether the key information of the knowledge fragment with high reference value index is given priority. If the proportion of the knowledge fragment with low reference value index exceeds the second preset proportion threshold, the search enhancement instruction is regenerated to ensure the generation quality.

[0110] Based on the generated search enhancement instructions, automated file processing is performed on the files to be processed, such as personal information anonymization, key field extraction, or format conversion. After the automated file processing is completed, the source identifiers of each knowledge fragment referenced by the generated search enhancement instructions are recorded, and the user's modification records for the automated file processing results are obtained. Based on the modification records and source identifiers, the modified knowledge fragments are identified, and historical feedback information is updated (such as generating new feedback tags for the modified knowledge fragments and adding them to the historical feedback information database). This forms a self-evolving knowledge base optimization loop, enabling subsequent processing to continuously optimize the granular structure and weight allocation of knowledge fragments based on the updated historical feedback information.

[0111] This invention is now complete.

[0112] In summary, in this embodiment of the invention, a file processing request for a file to be processed is received; in response to the file processing request, historical feedback information is obtained; wherein, the historical feedback information includes the user's modification records for historical automated file processing results; based on the historical feedback information, the granularity adjustment requirement index of the relevant knowledge fragments of the file processing request is determined, and the granularity of the knowledge fragments is adjusted to obtain the adjusted knowledge fragments; based on the historical feedback information, the irrelevance index between the adjusted knowledge fragments and the file processing request is determined, and the reference value index of the adjusted knowledge fragments is determined by combining the contextual correlation between the adjusted knowledge fragments and the file to be processed; based on the reference value index, a search enhancement instruction corresponding to the file processing request is generated; based on the search enhancement instruction, automated file processing is performed on the file to be processed. This invention establishes a mapping relationship between user modification behavior and the granularity and weight of knowledge fragments, achieving dual adaptive optimization of knowledge fragment granularity and reference weight. This enables the generation of retrieval enhancement instructions to dynamically adapt to the differentiated needs of different file processing scenarios. Furthermore, it determines granularity adjustment requirements based on the inverse correlation between user modification frequency and modification magnitude, and iteratively executes fine-grained segmentation until preset conditions are met, effectively eliminating the mixing of effective and invalid information in knowledge fragments and reducing the manual correction cost of subsequent instruction generation. Moreover, by integrating the user modification frequency and cumulative proportion of modified content in historical enhancement instruction generation with the correlation degree between the knowledge fragment and the current file context, a dual-dimensional reference value evaluation system of task relevance and scenario adaptability is constructed, improving the context awareness capability of retrieval enhancement instructions. Finally, by using source identification to establish a reference mapping relationship between instruction generation and knowledge fragments, and continuously updating historical feedback information based on real-time user modification records, a self-evolving knowledge base optimization mechanism is formed, enabling automated file processing capabilities to continuously improve with increasing usage frequency.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart retrieval enhancement instruction generation system for automated document processing, characterized in that, include: Receive file processing requests for files to be processed; In response to the file processing request, historical feedback information is obtained; wherein, the historical feedback information includes the user's modification records of historical automated file processing results; Based on the historical feedback information, the granularity adjustment requirement index of the relevant knowledge fragments of the file processing request is determined, and the granularity of the relevant knowledge fragments is adjusted to obtain the adjusted knowledge fragments. Based on the historical feedback information, the irrelevance indicators between the adjusted knowledge fragment and the file processing request are determined, and the reference value indicators of the adjusted knowledge fragment are determined by combining the contextual relevance between the adjusted knowledge fragment and the file to be processed. Based on the reference value index, generate retrieval enhancement instructions corresponding to the file processing request; Based on the enhanced search instructions, automated file processing is performed on the file to be processed; The step of determining the granularity adjustment requirement index of the relevant knowledge fragments of the file processing request based on the historical feedback information includes: determining the user modification frequency and user modification magnitude of the relevant knowledge fragments of the file processing request in the historical enhancement instruction generation based on the historical feedback information; and determining the granularity adjustment requirement index of the relevant knowledge fragments based on the user modification frequency and the user modification magnitude. The step of granularly adjusting the relevant knowledge fragments to obtain adjusted knowledge fragments includes: responding to the granularity adjustment requirement index meeting the preset adjustment conditions, performing fine-grained segmentation based on semantic boundaries on the relevant knowledge fragments to obtain intermediate knowledge fragments; determining the granularity adjustment requirement index of the intermediate knowledge fragments based on the historical feedback information; iteratively executing the fine-grained segmentation step and the granularity adjustment requirement index determination step until the granularity adjustment requirement index no longer meets the preset adjustment conditions, thereby obtaining the adjusted knowledge fragments; The step of determining the irrelevance indicators between the adjusted knowledge fragment and the file processing request based on the historical feedback information includes: obtaining historical processing records that match the file processing request; determining the user modification frequency and cumulative percentage of user modifications in the corresponding historical enhancement instruction generation based on the historical processing records; and determining the irrelevance indicators between the adjusted knowledge fragment and the file processing request based on the user modification frequency and the cumulative percentage of user modifications.

2. The intelligent retrieval enhancement instruction generation system for automated document processing according to claim 1, characterized in that, The determination of the reference value index of the adjusted knowledge fragment by combining the contextual relevance between the adjusted knowledge fragment and the file to be processed includes: Determine the degree of contextual association between the adjusted knowledge fragment and the file to be processed; Based on the degree of contextual relevance and the irrelevance index, the reference value index of the adjusted knowledge fragment is determined; wherein, the reference value index increases with the increase of the degree of contextual relevance and decreases with the increase of the irrelevance index.

3. The intelligent retrieval enhancement instruction generation system for automated document processing according to claim 1, characterized in that, The acquisition of historical feedback information includes: When generating enhanced historical retrieval instructions, the source identifiers of each knowledge fragment referenced are recorded; Based on the user's modifications to the historical search enhancement instructions, the modified knowledge fragments are located using the source tracing identifier; Generate feedback tags for the modified knowledge fragments; Based on the feedback tags, historical feedback information is generated.

4. The intelligent retrieval enhancement instruction generation system for automated document processing according to claim 3, characterized in that, Also includes: After performing automated document processing, the source identifiers of each knowledge fragment referenced by the generated retrieval enhancement command are recorded; Retrieve the user's modification history for the results of this automated file processing; Based on the modification record and the source identifier, the knowledge fragment that was modified this time is determined, and the historical feedback information is updated.

5. The intelligent retrieval enhancement instruction generation system for automated document processing according to any one of claims 1-4, characterized in that, The step of generating enhanced search instructions corresponding to the file processing request based on the reference value index includes: Based on the aforementioned reference value indicators, the adjusted knowledge fragments are filtered and sorted. The adjusted knowledge fragments, after being filtered and sorted, are assembled with the file processing request and the context of the file to be processed to form an enhanced hint; Based on the enhanced prompts, a search enhancement instruction corresponding to the file processing request is generated.

6. The intelligent retrieval enhancement instruction generation system for automated document processing according to any one of claims 1-4, characterized in that, Before determining the granularity adjustment requirement index for the knowledge fragment related to the file processing request based on the historical feedback information, the method further includes: Based on the processing requirements of the file processing request, candidate knowledge fragments are retrieved from the knowledge base; Based on the feedback tags of the candidate knowledge fragments recorded in the historical feedback information, knowledge fragments with a negative feedback tag ratio exceeding a first preset ratio threshold are filtered out, and the retained knowledge fragments are used as knowledge fragments related to the file processing request.

7. The intelligent retrieval enhancement instruction generation system for automated document processing according to any one of claims 1-4, characterized in that, After generating the enhanced search instruction corresponding to the file processing request based on the reference value index, the method further includes: The search enhancement instruction performs a weight self-check. If the proportion of knowledge fragment references with low reference value exceeds the second preset proportion threshold, the search enhancement instruction is regenerated.

Images