A semantic index-based intelligent retrieval method for file content

Abstract

The application relates to the technical field of intelligent retrieval of file content, and particularly relates to an intelligent retrieval method of file content based on semantic index, which comprises the following steps: S1, performing structural analysis and semantic segmentation on a target file; S2, generating a validity dependency chain; S3, performing propagation determination on the validity state of each text unit; S4, constructing a semantic index library; and S5, filtering based on the final validity state label of the candidate semantic vector association and returning the file segment corresponding to the text unit satisfying the effective return condition. According to the application, the file content is subjected to semantic segmentation, validity dependency recognition, validity state propagation and label semantic index construction, so that the retrieval result can meet the semantic correlation and exclude invalid, replaced and non-limit-condition file segments, thereby improving the accuracy of file content retrieval.

Description

Technical Field

[0001] This invention relates to the field of intelligent document content retrieval technology, and in particular to an intelligent document content retrieval method based on semantic indexing. Background Technology

[0002] With the continuous increase in the number of electronic documents, institutional documents, normative documents, contract texts, and corporate knowledge documents, document content retrieval has become an important function in information management systems. Existing document retrieval methods are usually based on keyword matching, full-text indexing, or semantic vector retrieval. Among them, semantic vector retrieval can map the query text and document fragments to the same semantic space and recall relevant content based on semantic similarity. Compared with traditional keyword retrieval, it has better semantic understanding capabilities and is suitable for content search scenarios involving long texts, multiple chapters, and multiple versions of documents.

[0003] However, in actual document management, document content is not always independently valid; the validity of some clauses, paragraphs, or chapters may change due to the subsequent repeal, replacement, supplementation, restriction, or citation of document content. Existing semantic retrieval typically returns results based solely on text similarity, making it difficult to identify the validity dependencies between document fragments, and also difficult to determine whether the recalled content has become invalid, been replaced, or is only valid within a limited scope. This results in semantically similar but currently inapplicable document fragments being returned. To address these issues, it is necessary to provide an intelligent document content retrieval method based on semantic indexing to improve the accuracy and usability of retrieval results. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides a method for intelligent retrieval of file content based on semantic indexing.

[0005] A method for intelligent file content retrieval based on semantic indexing includes the following steps: S1: Perform structural parsing and semantic segmentation on the target file to obtain several text units with complete semantic boundaries, and assign a unique block identifier to each text unit; S2: Based on the content of each text unit, identify the effectiveness dependency relationship between text units and generate an effectiveness dependency chain; S3: Based on the effectiveness dependency chain, invoke the preset effectiveness state propagation rules to propagate and determine the effectiveness state of each text unit, determine the final effectiveness state label of each text unit, and output the set of text units carrying the final effectiveness state label. S4: Semantically vectorize the text units carrying the final validity status label to generate the corresponding semantic vector, and store the semantic vector together with the corresponding block identifier and the final validity status label to build a semantic index library; S5: Receive the query text, perform semantic vectorization encoding on the query text to obtain the query semantic vector, retrieve candidate semantic vectors similar to the query semantic vector in the semantic index, filter based on the final validity status label associated with the candidate semantic vectors, and return the file fragments corresponding to the text units that meet the valid return conditions.

[0006] Optionally, the structural parsing and semantic segmentation of the target file in S1 specifically includes: S11: Read the layout structure information and text content information of the target file, and identify the heading level, paragraph boundary, clause number and body content of the target file; S12: Determine the structural belonging of the text content based on the heading level, paragraph boundaries and clause number, and divide the continuous text content belonging to the same structural node into initial text segments; S13: Perform semantic integrity detection on the initial text fragment; specifically, when the initial text fragment contains multiple independent semantic contents, split it into multiple text units according to semantic boundaries; when adjacent initial text fragments together constitute complete semantic content, merge the adjacent initial text fragments into one text unit; S14: Output several text units with complete semantic boundaries.

[0007] Optionally, in S1, a unique block identifier is assigned to each text unit, specifically including: S15: Extract the file identifier, hierarchical path, start and end positions of the text, and segmentation number of each text unit in the target file; S16: Combine and encode the file identifier, structural hierarchy path, text start and end positions and segmentation number to generate a block identifier that uniquely corresponds to the text unit; S17: Establish the binding relationship between block identifiers and corresponding text units.

[0008] Optionally, identifying the effectiveness dependency between text units in S2 specifically includes: S21: Extract the effectiveness trigger words, object reference information, and structural reference information from each text unit; S22: Locate the affected text unit in all text units based on object pointing information and structural reference information; S23: The text unit containing the effect trigger word is identified as the source text unit, and the text unit that will be affected is identified as the target text unit; S24: Based on the semantic type corresponding to the effectiveness trigger word, determine the type of effectiveness influence of the source text unit on the target text unit. The type of effectiveness influence includes repeal, substitution, supplementation, restriction, and citation.

[0009] Optionally, generating the effectiveness dependency chain in S2 specifically includes: S25: Obtain the source block identifier corresponding to the source text unit and the target block identifier corresponding to the target text unit; S26: Write the source block identifier, target block identifier, type of effect, direction of effect, and dependency generation time into the same chain record to form a single effect dependency record; S27: Aggregate multiple validity dependency records according to the target block identifier to generate a validity dependency chain corresponding to each target text unit.

[0010] Optionally, the preset effectiveness state propagation rules in S3 specifically include: Read the effectiveness dependency chain corresponding to each target text unit to obtain all effectiveness impact types acting on the target text unit; When the type of effect is invalidation, the effect status of the target text unit is updated to invalidation. When the type of effect is substitution, the effect status of the target text unit is updated to the substituted state, while the effect status of the source text unit remains in the valid state. When the type of effect is supplementary, the effect status of the target text unit is kept in the effective state, and a supplementary effective association is established between the source text unit and the target text unit; When the type of effect is restriction, the effect status of the target text unit is updated to partially effective. When the effect type is reference, the effect status of the target text unit is maintained as valid.

[0011] Optionally, when the same text unit is affected by multiple validity dependency records at the same time, the final validity status label is determined according to the preset validity priority; the preset validity priority is in the following order from high to low: abolished, replaced, restricted, supplemented, and cited.

[0012] Optionally, S4 specifically includes: S41: Semantically vectorize each text unit carrying the final validity status label to generate a semantic vector corresponding to the text unit; S42: Write the semantic vector, block identifier, target file identifier, text start and end positions, final validity status label, and validity dependency chain identifier into the same index record; S43: Build a vector retrieval structure for all indexed records according to semantic vectors, and build a tag filtering field according to the final validity status tag to form a semantic index library.

[0013] Optionally, step S5 retrieves candidate semantic vectors similar to the query semantic vector from the semantic index library, specifically including: S51: Perform semantic vectorization encoding on the query text to obtain the query semantic vector; S52: Calculate the similarity between the query semantic vector and each semantic vector in the semantic index; S53: Select semantic vectors that meet the similarity threshold from high to low similarity as candidate semantic vectors; S54: Based on the block identifier corresponding to the candidate semantic vector, read the final validity status label associated with the candidate semantic vector.

[0014] Optionally, the filtering in S5 based on the final effectiveness status label associated with the candidate semantic vectors specifically includes: S55: Read the final effectiveness state label associated with each candidate semantic vector; When the final validity status label is valid, the text unit corresponding to the candidate semantic vector is retained; When the final validity status label is a partially valid status, read the restricted valid range corresponding to the text unit, and retain the text unit corresponding to the candidate semantic vector when the query text hits the restricted valid range. When the final validity status label is invalid or replaced, filter the text units corresponding to the candidate semantic vectors. S56: Return the file fragments corresponding to the retained text units after sorting the results by similarity.

[0015] The beneficial effects of this invention are: This invention performs structural analysis and semantic segmentation on target files, dividing continuous file content into text units with complete semantic boundaries and assigning a unique block identifier to each text unit, enabling the file content to participate in subsequent processing in a locationable, referable, and traceable manner. At the same time, by identifying the obsolescence, substitution, supplementation, restriction, and reference relationships between text units, a validity dependency chain is generated, and the final validity status label is determined according to the validity status propagation rules, so that file fragments have a clear basis for validity judgment before entering the semantic index.

[0016] This invention, by jointly storing semantic vectors, block identifiers, and final validity status labels when constructing a semantic index library, enables semantic retrieval results to be synchronously associated with file location and validity status. During the query phase, candidate file fragments are first recalled based on semantic similarity, and then filtered according to the final validity status labels to exclude content that is invalid, replaced, or does not meet the restrictions. This avoids returning currently inapplicable file fragments solely based on semantic similarity, thereby improving the accuracy, usability, and traceability of file content retrieval results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the intelligent document content retrieval method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the preset effectiveness state propagation rules in an embodiment of the present invention. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0020] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0021] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0022] like Figures 1-2 As shown, a method for intelligent file content retrieval based on semantic indexing includes the following steps: S1: Perform structural parsing and semantic segmentation on the target file to obtain several text units with complete semantic boundaries, and assign a unique block identifier to each text unit; S1 performs structural parsing and semantic segmentation on the target file, specifically including: S11: Read the layout structure information and text content information of the target file, and identify the heading level, paragraph boundaries, clause number and body content of the target file; S12: Determine the structural affiliation of text content based on heading level, paragraph boundaries, and clause numbering. Divide continuous body text belonging to the same structural node into initial text segments. Specifically, scan the target file according to paragraph order to determine whether each paragraph has a heading attribute. The heading attribute is determined by font size, bolding, numbering format, and pre-paragraph spacing. For paragraphs that meet the heading attribute conditions, identify them as structural nodes and determine their hierarchical position in the target file based on the heading level. For paragraphs that do not meet the heading attribute conditions, identify them as body text paragraphs and assign them to the structural node that precedes them in the order and is closest in level. The heading level score is calculated using the following formula: ,in, Indicates the first The heading level score for each paragraph; Indicates the first The font size of each paragraph; This indicates the minimum font size of a paragraph in the target file; This indicates the maximum font size for paragraphs in the target file; Indicates the first The bold mark for each paragraph, with a value of 1 indicating bold and a value of 0 indicating no bold; Indicates the first The numbering level marker for each paragraph, with a value of 1 indicating that it has a heading number and a value of 0 indicating that it does not have a heading number; Indicates the first Paragraph spacing before each paragraph; This represents the minimum spacing before each paragraph in the target file; This indicates the maximum value of the pre-paragraph spacing in the target file; These represent the weights corresponding to font size, bold status, numbering level, and paragraph spacing, with values ​​of 0.35, 0.25, 0.25, and 0.15 respectively, and satisfying the following conditions: .when At that time, the first The paragraph is determined to be a title paragraph; when At that time, the first Each paragraph is identified as a body paragraph. For title paragraphs, the system determines the title level based on the clause number format; for paragraphs that do not contain explicit clause numbers but meet the title determination criteria, the system determines their level based on font size, indentation, and the relationship between adjacent title nodes. After determining the title level, a structure node stack is established; when a title paragraph is scanned, it is treated as a new structure node, and push, pop, or attach processing is performed according to the hierarchical relationship between its title level and the node levels in the stack; when a body paragraph is scanned, it is assigned to the last structure node in the structure node stack; for body paragraphs with clause numbers, the system performs attribution correction based on the consistency between the clause number and the title number prefix, so that the body paragraph is assigned to the structure node under the corresponding number level. After the structural attribution relationship is determined, the body content belonging to the same structure node and arranged continuously is divided into initial text fragments; each initial text fragment records its own structure node, parent title path, paragraph start and end positions, and clause number information, providing a positioning basis for subsequent block identifier generation and validity dependency identification.

[0023] S13: Perform semantic integrity detection on the initial text fragment; specifically, when the initial text fragment contains multiple independent semantic contents, split it into multiple text units according to semantic boundaries; when adjacent initial text fragments together constitute complete semantic content, merge the adjacent initial text fragments into one text unit; S14: Output several text units with complete semantic boundaries.

[0024] S1 assigns a unique block identifier to each text unit, specifically including: S15: Extract the file identifier, structural hierarchy path, text start and end positions, and segmentation number of each text unit in the target file; where the file identifier is used to indicate the target file to which the text unit belongs; the structural hierarchy path is used to indicate the position of the title of the text unit in the target file; the text start and end positions are used to indicate the original position range of the text unit in the main text of the target file; the segmentation number is used to indicate the arrangement order of multiple text units under the same structural node; the above four types of information can simultaneously limit the file source, structural position, content position, and segmentation order of the text unit, thereby ensuring that the block identifier has uniqueness and traceability.

[0025] S16: Combine and encode the file identifier, structural hierarchy path, text start and end positions and segmentation number to generate a block identifier that uniquely corresponds to the text unit; S17: Establish the binding relationship between block identifiers and corresponding text units, and use the binding relationship as the reference basis for the generation of validity dependency chains, the propagation of validity states, and the construction of semantic indexes.

[0026] Specifically, the combined code is represented by the following formula: ,in, Indicates the first A unique block identifier corresponding to each text unit; Represents the hash encoding function; The file identifier that represents the target file; Indicates the first The structural hierarchy path corresponding to each text unit; Indicates the first The starting position of each text unit in the object file; Indicates the first The end position of each text unit in the target file; Indicates the first The segmentation number of each text unit under the same structural node; This represents a string concatenation operation. Specifically, it includes the structural hierarchy path. It is formed by sequentially combining the title nodes of each level to which the text unit belongs, and its expression is: ,in, Indicates the first The structural hierarchy path corresponding to each text unit; Indicates the first The first-level heading node to which each text unit belongs; Indicates the first The second-level heading node to which each text unit belongs; Indicates the first The text unit belongs to the first Heading node; This indicates the number of heading levels contained in the structural path containing the text unit; This indicates a string concatenation operation.

[0027] After generating a unique block identifier, a binding relationship is established between the block identifier and the corresponding text unit. This binding relationship includes at least the unique block identifier, the original text content of the text unit, the target file identifier, the structural hierarchy path, the start and end positions of the text, the segmentation sequence number, and the text unit status field. This binding relationship is represented as follows: ,in, Indicates a unique block identifier The corresponding binding record; Indicates the first The original text content of each text unit; The file identifier that represents the target file; Indicates the first The structural hierarchy path corresponding to each text unit; Indicates the first The starting position of each text unit in the object file; Indicates the first The end position of each text unit in the target file; Indicates the first Each text unit has a segmentation sequence number under the same structural node. Through the above-mentioned unique block identifier generation and binding process, each text unit has a stable, unique and traceable positioning basis, enabling the source block identifier and target block identifier in the validity dependency chain to accurately point to the specific text unit, and ensuring data reference consistency in the subsequent validity state propagation, semantic index construction and file fragment return process.

[0028] S2: Based on the content of each text unit, identify the effectiveness dependency relationship between text units and generate an effectiveness dependency chain; S2 identifies the effectiveness dependencies between text units, specifically including: S21: Extract the effectiveness trigger words, object reference information, and structural reference information from each text unit; S22: Locate the affected text unit in all text units based on object pointing information and structural reference information; S23: The text unit containing the effect trigger word is identified as the source text unit, and the text unit that will be affected is identified as the target text unit; S24: Based on the semantic type corresponding to the effectiveness trigger word, determine the type of effectiveness influence of the source text unit on the target text unit. The effectiveness influence types include repeal, substitution, supplementation, restriction, and citation.

[0029] Specifically, by reading the original text of each text unit one by one, and extracting effectiveness trigger words, object-directing information, and structural reference information from the text units, effectiveness trigger words include repeal trigger words, substitution trigger words, supplementary trigger words, restrictive trigger words, and reference trigger words. Repealed trigger words indicate that the target text unit is no longer applicable; substitution trigger words indicate that the target text unit has been replaced by new text content; supplementary trigger words indicate that the source text unit adds supplementary content to the target text unit; restrictive trigger words indicate that the target text unit is applicable only under limited conditions; and reference trigger words indicate that the source text unit calls upon or depends on the content of the target text unit. By classifying effectiveness trigger words, different textual expressions can be uniformly mapped to processable effectiveness impact types. Object-directing information includes the name of the affected object, the number of the affected clause, the name of the affected title, and keywords of the affected content. By identifying directional expressions such as "this clause," "the above content," "Article X," "Chapter X," "the foregoing provisions," and "document fragments," the candidate objects to which the source text unit points can be determined. Structural reference information includes heading level path, clause number, paragraph number, block identifier, and text start and end positions, which are used to further map object pointing information to segmented text units. By combining object pointing information and structural reference information, pointing expressions in natural language can be converted into associations between specific text units.

[0030] To improve the certainty of the recognition results, a power trigger matching value is calculated for each text unit, using the following formula: ,in, Indicates the first The effectiveness of each text unit triggers a matching value; Indicates the first The effectiveness trigger word hit flag for each text unit, with a value of 1 indicating that the effectiveness trigger word was hit and a value of 0 indicating that the effectiveness trigger word was not hit; Indicates the first The object pointer hit flag for each text cell has a value of 1 indicating the presence of object pointer information and a value of 0 indicating the absence of object pointer information. Indicates the first The structure reference hit flag for each text unit has a value of 1 indicating the existence of structure reference information and a value of 0 indicating the absence of structure reference information. These represent the weights corresponding to the effectiveness trigger words, object pointing information, and structural reference information, respectively, with values ​​of 0.45, 0.35, and 0.20, and satisfying the following conditions: ;when At that time, the first A text unit is determined to be a candidate source text unit with the power to influence expression; when At that time, the first Each text unit serves as a source text unit in the generation of effectiveness dependencies. Through the above determination, ordinary text units that do not have an effect on effectiveness can be screened out first, thereby reducing the false matching rate of subsequent dependency identification.

[0031] After identifying candidate source text units, the affected text units are located among all text units based on object pointing information and structural reference information. For pointing content containing explicit clause numbers, title names, or block identifiers, the system directly matches according to the clause number, title path, or block identifier; for content containing relative pointing expressions such as "above," "previously," "this section," or "this content," the system searches for corresponding text units by moving forward or upward according to the structural level and paragraph order of the candidate source text units. This location method converts natural language pointing content into correspondences between specific text units; the pointing match degree between candidate source text units and candidate affected text units is calculated using the following formula: ,in, Indicates the first The candidate source text unit for the first The degree of alignment of the text units affected by the candidate text; This indicates a matching number marker, with a value of 1 indicating the first matching number. The object pointer number in the candidate source text unit is related to the first... The clause numbers of all candidate affected text units are consistent; a value of 0 indicates inconsistency. This indicates a heading path matching flag. A value of 1 indicates that the heading level paths of the two objects satisfy the reference relationship, while a value of 0 indicates that they do not. This represents a keyword matching flag; a value of 1 indicates that the object points to a keyword that matches the specified keyword. The keywords of the candidate affected text units are consistent; a value of 0 indicates inconsistency. This indicates a proximity marker, with a value of 1 indicating the nearest neighbor. The candidate affected text unit is located in the first... If a candidate source text unit is preceding and located under the same or higher-level structural node, a value of 0 indicates that the positional relationship is not satisfied. These represent the weights for number matching, title path matching, keyword matching, and position proximity, respectively, with values ​​of 0.40, 0.25, 0.20, and 0.15, and satisfying the following conditions: The matching threshold is set to 0.65. At that time, the first The candidate affected text unit was identified as the [number]th The target text unit corresponds to each candidate source text unit; when multiple candidate affected text units meet the conditions, the text unit with the highest matching degree is selected as the target text unit; when multiple candidate affected text units belong to consecutive text units under the same structural node, all text units within that consecutive range are determined as the target text unit set.

[0032] After identifying the source and target text units, the type of influence of the source text unit on the target text unit is determined based on the semantic type corresponding to the effectiveness trigger words in the source text unit. The effectiveness influence types include repeal, substitution, supplementation, restriction, and citation. Repeal indicates that the target text unit is no longer applicable; substitution indicates that the target text unit is replaced by the corresponding content of the source text unit; supplementation indicates that the source text unit adds supplementary content to the target text unit; restriction indicates that the target text unit is applicable only under limited conditions; and citation indicates that the source text unit calls upon or depends on the content of the target text unit. Through this classification process, different effectiveness expressions can be uniformly converted into influence types that can be recorded in subsequent effectiveness dependency chains. The type score of the effectiveness influence type is calculated according to the following formula: ,in, Indicates the first The source text unit for the first The target text unit in the first Type score under the influence of class effectiveness; Indicates the type of effect, with values ​​including repeal, substitution, supplementation, restriction, and reference; Indicates the first The first source text unit The hit strength of the type of effect trigger word, with a value ranging from 0 to Indicates the first The first source text unit The matching strength of the semantic class expression, with a value ranging from 0 to 1; Indicates the first The source text unit and the first Between the target text units in the first The structural matching strength under the class effect type, with a value range from 0 to 1; These represent the weights corresponding to the trigger word hit strength, semantic expression matching strength, and structural matching strength, respectively, with values ​​of [values ​​to be filled in]. And satisfy Furthermore, the type of final effect impact is determined according to the following formula: ,in, Indicates the first The source text unit for the first The final effectiveness of each target text unit affects the type of impact; This indicates that the type with the highest score is selected from all candidate effectiveness impact types; Indicates the first The source text unit for the first The target text unit in the first The system calculates type scores under each type of effectiveness influence type. When the highest type score reaches 0.60, its corresponding effectiveness influence type is determined as the final effectiveness influence type. When the highest type score is below 0.60, no corresponding effectiveness dependency is generated. Through this process, the source text unit, target text unit, and the effectiveness influence type between them can be determined from the text unit content. This enables the obsolescence, substitution, supplementation, restriction, and citation relationships between document content to form calculable effectiveness dependencies, providing a basis for subsequently generating effectiveness dependency chains and disseminating the final effectiveness status label.

[0033] S2 generates a power dependency chain, specifically including: S25: Obtain the source block identifier corresponding to the source text unit and the target block identifier corresponding to the target text unit; S26: Write the source block identifier, target block identifier, type of effect, direction of effect, and dependency generation time into the same chain record to form a single effect dependency record; S27: Aggregate multiple validity dependency records according to the target block identifier to generate a validity dependency chain corresponding to each target text unit.

[0034] S3: Based on the effectiveness dependency chain, call the preset effectiveness state propagation rules to propagate and determine the effectiveness state of each text unit, determine the final effectiveness state label of each text unit, and output the set of text units carrying the final effectiveness state label. The pre-defined rules for propagating the effectiveness status in S3 include: Read the effectiveness dependency chain corresponding to each target text unit to obtain all effectiveness impact types acting on the target text unit; When the type of effect is invalidation, the effect status of the target text unit is updated to invalidation. When the type of effect is substitution, the effect status of the target text unit is updated to the substituted state, while the effect status of the source text unit remains in the valid state. When the type of effect is supplementary, the effect status of the target text unit is kept in the effective state, and a supplementary effective association is established between the source text unit and the target text unit; When the type of effect is restriction, the effect status of the target text unit is updated to partially effective. When the effect type is reference, the effect status of the target text unit is maintained as valid.

[0035] When the same text unit is affected by multiple validity dependency records simultaneously, the final validity status label is determined according to the preset validity priority; the preset validity priority, from high to low, is: abolished, replaced, restricted, supplemented, and cited; specifically: When the same text unit is simultaneously subject to repeal and other types of effect, the final effect status label of the text unit is determined to be invalid. When the same text unit has both substitution and restriction and no repeal, the final validity status label of the text unit is determined to be the substituted state. When the same text unit has restrictions, supplements and references at the same time, and there is no repeal or replacement, the final validity status label of the text unit is determined to be a partially valid state. When the same text unit only has supplements and references, the final validity status label of the text unit is determined to be valid.

[0036] S4: Semantically vectorize the text units carrying the final validity status label to generate the corresponding semantic vector, and store the semantic vector together with the corresponding block identifier and the final validity status label to build a semantic index library; S4 specifically includes: S41: Semantically vectorize each text unit carrying the final validity status label to generate a semantic vector corresponding to the text unit; S42: Write the semantic vector, block identifier, target file identifier, text start and end positions, final validity status label, and validity dependency chain identifier into the same index record; S43: Build a vector retrieval structure for all indexed records according to semantic vectors, and build a tag filtering field according to the final validity status tag to form a semantic index library.

[0037] Specifically, each index record in the semantic index library corresponds to a text unit. When constructing the index library, the semantic vector of the text unit is used as the retrieval entry point, the block identifier is used as the unique location basis, the final validity status label is used as the filtering basis, and the validity dependency chain identifier is used as the status tracing basis.

[0038] Table 1. Examples of Semantic Indexes In Table 1 above, the index record number represents an index record in the semantic index library, used to distinguish different index records; the semantic vector represents the result of semantic vectorization encoding of the text unit, used to calculate the similarity with the query semantic vector; the block identifier represents the unique identifier of the text unit, used to locate the text unit and as a reference object in the effectiveness dependency chain; the target file identifier represents the target file to which the text unit belongs, used to determine the source file of the search results; the text start position represents the starting position of the text unit in the target file, used to backtrack and extract file fragments; the text end position represents the ending position of the text unit in the target file, used to backtrack and extract file fragments; the final effectiveness status label represents the current effectiveness status of the text unit, used to determine whether the text unit is allowed to return; and the effectiveness dependency chain identifier represents the effectiveness dependency chain associated with the text unit, used to trace the basis for the formation of the effectiveness status of the text unit.

[0039] Upon receiving the query text, it is first encoded into a semantic vector, and then its similarity is calculated between the vector and the semantic vector of each index record in the semantic index. Index records with high similarity are recalled as candidate results. Subsequently, the final validity status label of the candidate results is read: if the final validity status label is valid, the candidate result is retained; if the final validity status label is invalid or replaced, the candidate result is filtered; if the final validity status label is partially valid, the system reads the corresponding constraints based on the validity dependency chain identifier and determines whether the query text falls within the valid range of those constraints. If the conditions are met, the query text is retained; otherwise, it is filtered. Through this structure, the semantic index can merge semantic similarity retrieval and validity status determination into the same retrieval process, avoiding the return of invalid, replaced, or unsuitable file fragments simply because of content similarity, thereby improving the accuracy and usability of intelligent file content retrieval results.

[0040] S5: Receive query text, perform semantic vectorization encoding on the query text to obtain query semantic vector, retrieve candidate semantic vectors similar to the query semantic vector in the semantic index, filter based on the final validity status label associated with the candidate semantic vector, and return the file fragment corresponding to the text unit that meets the valid return conditions. S5 retrieves candidate semantic vectors similar to the query semantic vector from the semantic index, specifically including: S51: Perform semantic vectorization encoding on the query text to obtain the query semantic vector; S52: Calculate the similarity between the query semantic vector and each semantic vector in the semantic index; S53: Select semantic vectors that meet the similarity threshold from high to low similarity as candidate semantic vectors; S54: Based on the block identifier corresponding to the candidate semantic vector, read the final validity status label associated with the candidate semantic vector.

[0041] Specifically, the query text is first preprocessed to remove irrelevant symbols, whitespace, and formatting marks, retaining keywords, phrases, and sentence structures that express the query intent. Then, the preprocessed query text is input into a semantic encoding model consistent with the semantic index building phase to generate a query semantic vector. This ensures that the query semantic vector and the semantic vectors in the index are in the same vector space, facilitating subsequent similarity calculations. The query semantic vector is represented as: ,in, Represents the query semantic vector; This represents a semantic vectorized coding model; This represents the preprocessed query text. After obtaining the query semantic vector, the similarity between the query semantic vector and each semantic vector in the semantic index is calculated. This similarity represents the semantic closeness between the query text and each text unit. In this implementation, cosine similarity is used for calculation. ,in, Represents the query semantic vector With the Each text unit corresponds to a semantic vector Semantic similarity between them; Represents the query semantic vector; Indicates the first in the semantic index library The semantic vector corresponding to each text unit; Indicates the query semantic vector and the first The dot product of semantic vectors; Indicates the magnitude of the query semantic vector; Indicates the first The similarity is calculated based on the magnitude of each semantic vector and compared with a preset similarity threshold; the preset similarity threshold is set to 0.75. At that time, the judgment of the first If a semantic vector satisfies the semantic relevance condition to the query text, it is included in the candidate semantic vector set; when When this happens, it is not included in the candidate semantic vector set; by filtering by similarity threshold, text units that are not sufficiently relevant to the query intent can be removed, reducing the processing load of subsequent effectiveness status filtering.

[0042] After determining the candidate semantic vector set, the candidate semantic vectors are sorted from high to low similarity. The sorting result is used to determine the semantic matching priority between candidate text units and query text. For candidate semantic vectors with the same similarity, the original text arrangement order is maintained according to their structural hierarchy path and text start and end positions in the target file, so that the return order of adjacent content in the same file is consistent with the original file. Subsequently, the corresponding block identifier is read according to the index record where the candidate semantic vector is located, and the final validity status label associated with the candidate semantic vector is read through the block identifier. Specifically, each candidate semantic vector corresponds to an index record, which stores the semantic vector, block identifier, target file identifier, text start and end positions, final validity status label, and validity dependency chain identifier. The index record is located by the candidate semantic vector, and the corresponding text unit is determined by the block identifier in the index record, and the final validity status label of the text unit is read.

[0043] In S5, filtering is performed based on the final effectiveness status label associated with candidate semantic vectors, specifically including: S55: Read the final effectiveness state label associated with each candidate semantic vector; When the final validity status label is valid, the text unit corresponding to the candidate semantic vector is retained; When the final validity status label is a partially valid status, read the restricted valid range corresponding to the text unit, and retain the text unit corresponding to the candidate semantic vector when the query text hits the restricted valid range. When the final validity status label is invalid or replaced, filter the text units corresponding to the candidate semantic vectors. S56: Return the file fragments corresponding to the retained text units after sorting the results by similarity.

[0044] After filtering, the retained text units are sorted according to similarity. Specifically, multiple text units originating from the same target file and with consecutive start and end positions are merged into consecutive file segments in the original text order. Text units from different files or with non-consecutive positions are output as independent file segments. This ensures that the returned content has a clear file source and original text position while maintaining semantic relevance ranking. The final output includes file segment content, target file identifier, block identifier, text start and end positions, similarity value, final validity status label, and hit criteria. Among them, the file segment content is used to display the specific text matched by the search; the target file identifier is used to determine the file to which it belongs; the block identifier is used to trace back the corresponding text unit; the text start and end positions are used to locate the original text position; the similarity value is used to indicate the degree of semantic relevance; the final validity status label is used to explain the current validity status; and the hit criteria is used to explain whether the segment is returned directly due to its valid status or due to the limitation of the valid range by partially valid status. Through the above processing, invalid, substituted, and content that does not meet the restrictions can be excluded from the semantic relevance search results, so that the final returned file segments simultaneously meet the semantic matching requirements and validity status requirements.

[0045] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0046] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for intelligent retrieval of file content based on semantic indexing, characterized in that, Includes the following steps: S1: Perform structural parsing and semantic segmentation on the target file to obtain several text units with complete semantic boundaries, and assign a unique block identifier to each text unit; S2: Based on the content of each text unit, identify the effectiveness dependency relationships between text units and generate an effectiveness dependency chain; specifically including: S21: Extract the effectiveness trigger words, object reference information, and structural reference information from each text unit; S22: Locate the affected text unit in all text units based on object pointing information and structural reference information; S23: The text unit containing the effect trigger word is identified as the source text unit, and the text unit that will be affected is identified as the target text unit; S24: Based on the semantic type corresponding to the effectiveness trigger word, determine the type of effectiveness influence of the source text unit on the target text unit, wherein the type of effectiveness influence includes repeal, substitution, supplementation, restriction and citation; S25: Obtain the source block identifier corresponding to the source text unit and the target block identifier corresponding to the target text unit; S26: Write the source block identifier, target block identifier, type of effect, direction of effect, and dependency generation time into the same chain record to form a single effect dependency record; S27: Aggregate multiple validity dependency records according to the target block identifier to generate a validity dependency chain corresponding to each target text unit; S3: Based on the effectiveness dependency chain, invoke the preset effectiveness state propagation rules to propagate and determine the effectiveness state of each text unit, determine the final effectiveness state label of each text unit, and output the set of text units carrying the final effectiveness state label. S4: Semantically vectorize the text units carrying the final validity status label to generate the corresponding semantic vector, and store the semantic vector together with the corresponding block identifier and the final validity status label to build a semantic index library; S5: Receive the query text, perform semantic vectorization encoding on the query text to obtain the query semantic vector, retrieve candidate semantic vectors similar to the query semantic vector in the semantic index, filter based on the final validity status label associated with the candidate semantic vectors, and return the file fragments corresponding to the text units that meet the valid return conditions.

2. The intelligent file content retrieval method based on semantic indexing according to claim 1, characterized in that, The S1 step involves structural parsing and semantic segmentation of the target file, specifically including: S11: Read the layout structure information and text content information of the target file, and identify the heading level, paragraph boundary, clause number and body content of the target file; S12: Determine the structural belonging of the text content based on the heading level, paragraph boundaries and clause number, and divide the continuous text content belonging to the same structural node into initial text segments; S13: Perform semantic integrity detection on the initial text fragment; specifically, when the initial text fragment contains multiple independent semantic contents, split it into multiple text units according to semantic boundaries; when adjacent initial text fragments together constitute complete semantic content, merge the adjacent initial text fragments into one text unit; S14: Output several text units with complete semantic boundaries.

3. The intelligent file content retrieval method based on semantic indexing according to claim 2, characterized in that, In S1, a unique block identifier is assigned to each text unit, specifically including: S15: Extract the file identifier, hierarchical path, start and end positions of the text, and segmentation number of each text unit in the target file; S16: Combine and encode the file identifier, structural hierarchy path, text start and end positions and segmentation number to generate a block identifier that uniquely corresponds to the text unit; S17: Establish the binding relationship between block identifiers and corresponding text units.

4. The intelligent file content retrieval method based on semantic indexing according to claim 1, characterized in that, The preset effectiveness state propagation rules in S3 specifically include: Read the effectiveness dependency chain corresponding to each target text unit to obtain all effectiveness impact types acting on the target text unit; When the type of effect is invalidation, the effect status of the target text unit is updated to invalidation. When the type of effect is substitution, the effect status of the target text unit is updated to the substituted state, while the effect status of the source text unit remains in the valid state. When the type of effect is supplementary, the effect status of the target text unit is kept in the effective state, and a supplementary effective association is established between the source text unit and the target text unit; When the type of effect is restriction, the effect status of the target text unit is updated to partially effective. When the effect type is reference, the effect status of the target text unit is maintained as valid.

5. The intelligent file content retrieval method based on semantic indexing according to claim 4, characterized in that, When the same text unit is affected by multiple validity dependency records at the same time, the final validity status label is determined according to the preset validity priority; the preset validity priority is in the following order from high to low: abolished, replaced, restricted, supplemented, and cited.

6. The intelligent file content retrieval method based on semantic indexing according to claim 1, characterized in that, S4 specifically includes: S41: Semantically vectorize each text unit carrying the final validity status label to generate a semantic vector corresponding to the text unit; S42: Write the semantic vector, block identifier, target file identifier, text start and end positions, final validity status label, and validity dependency chain identifier into the same index record; S43: Build a vector retrieval structure for all indexed records according to semantic vectors, and build a tag filtering field according to the final validity status tag to form a semantic index library.

7. The intelligent file content retrieval method based on semantic indexing according to claim 6, characterized in that, S5 retrieves candidate semantic vectors similar to the query semantic vector from the semantic index library, specifically including: S51: Perform semantic vectorization encoding on the query text to obtain the query semantic vector; S52: Calculate the similarity between the query semantic vector and each semantic vector in the semantic index; S53: Select semantic vectors that meet the similarity threshold from high to low similarity as candidate semantic vectors; S54: Based on the block identifier corresponding to the candidate semantic vector, read the final validity status label associated with the candidate semantic vector.

8. The intelligent file content retrieval method based on semantic indexing according to claim 7, characterized in that, The filtering in S5 based on the final effectiveness status label associated with the candidate semantic vectors specifically includes: S55: Read the final effectiveness state label associated with each candidate semantic vector; When the final validity status label is valid, the text unit corresponding to the candidate semantic vector is retained; When the final validity status label is a partially valid status, read the restricted valid range corresponding to the text unit, and retain the text unit corresponding to the candidate semantic vector when the query text hits the restricted valid range. When the final validity status label is invalid or replaced, filter the text units corresponding to the candidate semantic vectors. S56: Return the file fragments corresponding to the retained text units after sorting the results by similarity.

Images