A long text context deep directional retrieval method and system

Abstract

This invention discloses a method and system for deep context-oriented retrieval of long texts, relating to the field of data processing technology. The method includes: normalizing sentences in the long text to obtain a sequence of structured text blocks; performing hierarchical semantic structure extraction to construct standardized structured data; performing global consistency verification on the attributes of characters throughout the text to obtain structured semantic data; obtaining core semantic parameters and performing dimensionality reduction encoding to generate hierarchical semantic vectors; obtaining a fused retrieval result set, determining the target text blocks and target semantic information, and constructing deep context-oriented retrieval results for long texts. This invention solves the technical problems in existing technologies, such as semantic fragmentation in long texts, loose global logic, disconnect between complex retrieval intent and shallow semantic matching, and insufficient retrieval efficiency and accuracy due to the lack of hierarchical indexes for unstructured data. It achieves the technical effect of improving the retrieval accuracy, contextual coherence, processing efficiency, and result reliability of long texts.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a method and system for deep contextual retrieval of long texts. Background Technology

[0002] Current long-text retrieval methods primarily rely on keyword matching, vector space models, or semantic similarity calculations using general pre-trained language models (such as BERT and GPT series). However, these methods exhibit multiple limitations in real-world applications. Long texts typically possess complex hierarchical structures (such as chapters, paragraphs, and subtopics) and long-distance dependencies. Existing retrieval methods neglect the logical connections and thematic continuity between paragraphs and chapters, leading to the omission of key information or retrieval results that are disconnected from the context. Furthermore, the lack of a unified maintenance and consistency verification mechanism for global character attributes often results in misclassifying different states of the same entity as different individuals. In addition, the search results may return contradictory attribute information in different paragraphs, which seriously reduces the credibility of the search results. The search requests initiated by users contain complex logical constraints and deep intentions. Existing vector search is difficult to parse complex queries, resulting in poor semantic relevance of search results and a large amount of redundant and irrelevant content, which still requires users to manually screen them again. Furthermore, long texts come from diverse sources, have messy formats, and lack a unified semantic skeleton. The search system cannot use the text's own logical framework (such as the table of contents and heading system) to guide the search path, resulting in a large waste of computing resources in irrelevant areas. The search efficiency drops sharply as the text length increases.

[0003] Therefore, current technologies suffer from several technical problems, including semantic fragmentation of long texts, loose global logic, disconnect between complex search intents and shallow semantic matching, and insufficient search efficiency and accuracy due to the lack of hierarchical indexes for unstructured data. Summary of the Invention

[0004] This application provides a deep contextual retrieval method and system for long texts, which solves the technical problems in the prior art, such as semantic fragmentation of long texts, loose global logic, disconnect between complex retrieval intent and shallow semantic matching, and insufficient retrieval efficiency and accuracy due to the lack of hierarchical indexes for unstructured data. It achieves the technical effect of improving the retrieval accuracy, contextual coherence, processing efficiency and result reliability of long texts.

[0005] This application provides a method for deep context-oriented retrieval of long texts. The method includes: normalizing sentences in the long text; reconstructing paragraphs based on the normalization results to obtain a structured text block sequence, wherein the structured text block sequence contains a global index framework; performing hierarchical semantic structure extraction on the structured text block sequence to construct standardized structured data; performing global consistency verification on the full-text character attributes based on the global index framework and the standardized structured data; automatically aligning and correcting based on the verification results to obtain structured semantic data; extracting semantic-level key information from the structured semantic data; performing dimensionality reduction encoding on the core semantic parameters to generate hierarchical semantic vectors; introducing a dedicated retrieval task package and combining it with the hierarchical semantic vectors to perform full-text semantic retrieval to obtain a fused retrieval result set; determining target text blocks and target semantic information based on the fused retrieval result set and concatenating and filling them to construct a deep context-oriented retrieval result for the long text.

[0006] This application also provides a long-text context-based deep-targeted retrieval system, the system comprising: a structured text block sequence acquisition module, used to perform sentence normalization on the long text, and reconstruct paragraphs based on the sentence normalization results to obtain a structured text block sequence, the structured text block sequence containing a global index framework; a standardized structured data construction module, used to perform hierarchical semantic structured extraction on the structured text block sequence to construct standardized structured data; a structured semantic data acquisition module, used to perform global consistency verification of full-text character attributes based on the global index framework and the standardized structured data, and perform automatic alignment correction based on the verification results to obtain structured semantic data; a hierarchical semantic vector generation module, used to extract semantic-level key information from the structured semantic data, obtain core semantic parameters, perform dimensionality reduction encoding, and generate hierarchical semantic vectors; and a deep-targeted retrieval result acquisition module, used to introduce a dedicated retrieval task package and combine it with the hierarchical semantic vectors to perform full-text semantic retrieval, obtain a fused retrieval result set, determine target text blocks and target semantic information based on the fused retrieval result set, and concatenate and fill them to construct a long-text context-based deep-targeted retrieval result.

[0007] This application proposes a method and system for deep context-oriented retrieval of long texts. The method involves: normalizing sentences in long texts to obtain a sequence of structured text blocks; performing hierarchical semantic structure extraction to construct standardized structured data; performing global consistency verification on the attributes of characters throughout the text to obtain structured semantic data; obtaining core semantic parameters and performing dimensionality reduction encoding to generate hierarchical semantic vectors; obtaining a fused retrieval result set; identifying target text blocks and target semantic information; and constructing deep context-oriented retrieval results for long texts. This method addresses the technical problems in existing technologies, such as semantic fragmentation in long texts, loose global logic, disconnect between complex retrieval intent and shallow semantic matching, and insufficient retrieval efficiency and accuracy due to the lack of hierarchical indexes for unstructured data. It achieves the technical effect of improving the retrieval accuracy, contextual coherence, processing efficiency, and result reliability of long texts. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0009] Figure 1 This is a schematic diagram of a long text context-based deep-targeted retrieval method provided in an embodiment of this application.

[0010] Figure 2 This is a schematic diagram of the structure of a long text context-based deep-targeted retrieval system provided in an embodiment of this application.

[0011] Figure labeling: Module 10 for obtaining structured text block sequences, Module 20 for constructing standardized structured data, Module 30 for obtaining structured semantic data, Module 40 for generating hierarchical semantic vectors, and Module 50 for obtaining deep-targeted retrieval results. Detailed Implementation

[0012] To further illustrate the technical means and effects adopted by the present invention in order to achieve the intended purpose, the following detailed description is provided in conjunction with the accompanying drawings and preferred embodiments, based on the specific implementation methods, structures, features and effects of the present invention.

[0013] This application provides a method for deep context-based targeted retrieval of long texts, such as... Figure 1 As shown, the method includes: Step S100: Normalize the long text by sentences, reconstruct paragraphs based on the normalization results, and obtain a structured text block sequence, wherein the structured text block sequence contains a global index framework.

[0014] Preferably, each natural sentence in the original long text undergoes format normalization and expression standardization processing to ensure that all sentences follow unified expression norms in terms of syntax, punctuation, number format, date format, unit expression, and special symbols. Specifically, this includes punctuation normalization, number and date format normalization, abbreviation and full name normalization, special symbol processing, redundant whitespace character removal, and sentence boundary identification and segmentation. Then, based on the semantic relevance, thematic continuity, and logical connection between sentences in the standardized sentence sequence after normalization, sentences belonging to the same logical paragraph unit are re-aggregated and organized into independent paragraphs. This includes removing original paragraph markers, calculating semantic relevance, detecting thematic continuity, determining paragraph boundaries, and re-aggregating. The output is a re-divided paragraph sequence, where each paragraph consists of several consecutive sentences, and the sentences within each paragraph have relatively high semantic relevance. High semantic cohesion, with clear semantic boundaries between paragraphs; all paragraph units obtained after paragraph reconstruction are arranged according to their natural order in the original long text to form a structured text block sequence, and each paragraph unit is assigned a unique position identifier and hierarchical information, upgrading it from an ordinary paragraph to a structured unit that can be accurately located and referenced by the computer system. Each element carries a globally unique number and position mapping information; based on the structured text block sequence, a complete index system covering all paragraph units of the entire long text is established, enabling the system to directly locate and quickly access any text block using multiple dimensions such as paragraph number, chapter number, hierarchical depth, and position range. It includes paragraph number index, chapter hierarchical index, position range index, hierarchical depth index, and jump pointer chain, and outputs a multi-mapping index table that can be directly called by the retrieval system.

[0015] Step S200: Perform hierarchical semantic structure extraction on the structured text block sequence to construct standardized structured data.

[0016] Step S200 further includes performing top-level semantic extraction on the structured text block sequence to obtain overall information of the whole text; performing mid-level semantic extraction on the structured text block sequence to obtain story context information; performing bottom-level semantic extraction on the structured text block sequence to obtain single-point text information; and structurally integrating the overall information of the whole text, the story context information, and the single-point text information to construct standard structured data.

[0017] Preferably, the entire sequence of structured text blocks is taken as input, from which global attributes, macro-features, and overall summary information covering the entire text are extracted. This includes extracting macro-attributes, full-text theme tags, full-text time span, full-text list of main characters / entities, full-text summary, full-text sentiment and tone, and full-text logical structure type, outputting overall information about the entire text. The sequence of structured text blocks is then analyzed from the perspective of event development, logical progression, or argument evolution, extracting the overall process, evolutionary stages, and main plot developments throughout the text. This includes extracting the main stage divisions, core events / arguments of each stage, transitional relationships between stages, role evolution of main characters / entities, key time node sequences, causal logic chains, and the distribution of sub-themes in each stage, outputting the story's context. The process involves a layer-by-layer fine-grained scan of each paragraph and even each sentence in a structured text block sequence to extract all specific factual knowledge units and independent semantic atoms. This includes extracting entity-type information, attribute-type information, relation-type information, event-type information, numerical and statistical data, citation and source information, and local factual assertions, outputting single-point textual information. Then, the top-level semantic extraction results, middle-level semantic extraction results, and bottom-level semantic extraction results are merged, associated, and reorganized using a unified metadata framework to form standard structured data containing semantic information at all levels, with each level being indexable and interconnected. The top-level information provides the global context, the middle-level information provides the evolutionary context, and the bottom-level information provides specific facts. The three are linked through pointers to enable cross-level linked queries.

[0018] Step S300: Based on the global indexing framework and the standardized structured data, perform a global consistency check on the attributes of the characters in the full text, and automatically align and correct them according to the check results to obtain structured semantic data.

[0019] Step S300 further includes: constructing a globally unique identifier for a person based on the global index framework and the standardized structure data; using the globally unique identifier as an index to construct a set of attributes to be verified for the person; performing coreference resolution processing based on the set of attributes to be verified to generate a coreference chain for the person across text blocks; traversing the standardized structure data according to the coreference chain to perform text block matching and extract the person attribute information; performing a global traversal comparison based on the person attribute information to detect whether there are conflicts in the person attribute information of the same task in different text blocks, and generating a conflict detection result; when the conflict detection result indicates that there is a conflict, performing conflict resolution analysis based on the person attribute information to determine a baseline attribute value; automatically correcting the person attribute information according to the baseline attribute value to generate a task attribute correction record; performing a second global traversal verification on the person attribute information based on the task attribute correction record, and generating the structured semantic data when the verification passes.

[0020] Preferably, by utilizing the paragraph location capabilities provided by the global indexing framework and the character entity information extracted from the standardized structure data, each different character appearing in the long text is assigned an absolutely unique and non-repeating identifier code within the entire text. Specifically, all character references are extracted from the standardized structure data, initially grouped and merged, a unique code is assigned to each different character, a mapping between the identifier and the paragraph position is established, the identifier is written into the standardized structure data, and a globally unique identifier for the character is output. Then, using the globally unique identifier for the character as the index key, all attribute information of each candidate character is extracted from the standardized structure data to form a set to be verified. The similarity of different character references in this set is calculated and merged. Different references that are determined to point to the same real character are linked together into a coreference chain. Each chain uses a globally unique identifier for the character as the head, and the chain records all equivalent references of the character and their respective paragraph numbers in sequence.

[0021] Preferably, each reference in the coreference chain and its corresponding paragraph number are used as query conditions to locate the corresponding text block (paragraph) in the standardized structured data. All explicit and implicit attribute descriptions of the person are extracted from the text block. Specifically, all reference records in the current person's coreference chain are traversed, and the paragraph numbers corresponding to each reference are collected. These are then summarized to form a set of all paragraph numbers that need to be retrieved for the person. Each paragraph number is used as an index key to locate the corresponding complete paragraph content from the structured text block sequence. For each located paragraph, all statements in the paragraph that use the person's reference as the subject or object are identified. The predicate-object structure directly related to the person in the statement is extracted, the modifying descriptions of the person in the statement are extracted, and the actions that the person participates in in the statement are extracted. Then, the same type of attributes extracted from different paragraphs are merged, and each extracted attribute record is labeled with its source paragraph number and source statement number. The person's attribute information is then output.

[0022] Preferably, all attribute information of the same person is grouped by attribute type. Different values ​​of the same attribute type are compared to determine whether there are contradictions in the descriptions of the same attribute of the person in different paragraphs. Specifically, for all extracted attribute information of the person, all records are grouped into the same group using "person's globally unique identifier + attribute type" as the grouping key. For each attribute record in the same group, the attribute values ​​are compared pairwise to determine whether there are inconsistencies. For attribute records determined to be conflicting, conflict details are recorded, including the conflicting attribute type, the specific value descriptions of both parties, the original text statements of each party, and the paragraph number of each party. All conflict records are summarized to form a conflict detection result. For each attribute conflict listed in the conflict detection result, the credibility of each party in the conflict is assessed and weighed based on multi-dimensional evidence. The most credible attribute value is selected as the standard benchmark value of that attribute in the text. A benchmark attribute value table organized by person and attribute type is output, along with a record of the decision basis for resolving each benchmark value.

[0023] Preferably, all attribute information entries related to the character in the standardized structured data that are found to have deviations through conflict detection are uniformly replaced with the baseline attribute values ​​determined through conflict resolution. Simultaneously, the original value, target value, and modification location of each modification are recorded in detail. Specifically, based on the conflict detection results, the storage location of all attribute information records in the standardized structured data that are determined to have conflicts is located. For each entry to be corrected, its currently stored attribute value is modified to the baseline attribute value of the corresponding attribute type. During the modification process, the original value is not directly overwritten or deleted; instead, a "Backup Original Value" field is added next to the field to store the original value before modification and its source paragraph number. An independent correction record is generated for each modification, and the task attribute correction record is output. Using the character attribute correction record as a reference, all character attribute information in the entire standardized structured data is traversed again to confirm that all entries that should be corrected have been correctly updated and there are no omissions. After verification, the currently corrected complete data set is output as structured semantic data.

[0024] Furthermore, step S300 also includes: extracting features from the referential expressions of the attribute set of the person to be verified, obtaining character features, contextual semantic features, and positional features of the referential expressions; performing semantic similarity calculation based on the character features, contextual semantic features, and positional features to generate a referential similarity matrix; when the semantic similarity of two referential expressions in the referential similarity matrix exceeds a preset similarity threshold, determining that the two referential expressions point to the same person, and assigning the same globally unique identifier for the person to the two referential expressions; traversing the referential similarity matrix, linking referential expressions that share the globally unique identifier for the person, and constructing a cross-text block co-reference chain for the person.

[0025] Preferably, each denotation is extracted from the set of attributes of the person to be verified, and multi-dimensional feature quantification is performed on the denotation itself and its environment. This is converted into a structured feature vector that can be used by the computer to calculate similarity. Specifically, character features refer to the morphological features of the Chinese characters in the denotation, which do not depend on the external context. Contextual semantic features refer to the semantic environment information of the sentence and paragraph in which the denotation is located, reflecting the language scene when the denotation is used. Positional features refer to the spatial distribution information of the denotation in the structured text block sequence. Taking the feature description object of each denotation as a unit, all denotations in the set are paired up. For each pair of denotations, the similarity at the character level, context level, and position level is calculated. The three levels of similarity are then weighted and fused into the final semantic similarity score. All pairing results are arranged and output in the form of a two-dimensional matrix. The character similarity comprehensively reflects the character overlap ratio, edit distance, prefix and suffix matching, core name similarity, etc. Finally, a denotation similarity matrix is ​​obtained. The value of each element in the matrix is ​​between 0 and 1, representing the final semantic similarity between the row corresponding denotation and the column corresponding denotation.

[0026] Preferably, the system uses a referential similarity matrix as input and a preset similarity threshold as the decision boundary. All referential pairs exceeding this threshold are marked as co-references, and these co-referenced referents are grouped together. Each group is assigned a common globally unique identifier for the person. The similarity threshold, such as 0.75, is set through experimental tuning on a labeled dataset. Specifically, all upper triangular elements in the referential similarity matrix are traversed, and the similarity score of each tuple is compared with the threshold. When the semantic similarity exceeds the preset similarity threshold, a "co-reference edge" is marked between the tuples, indicating that the two referential expressions are determined by the system to point to the same real person. A co-reference connected graph is constructed using all referential expressions as nodes and all marked "co-reference edges" as edges. All co-reference edges are traversed, and all nodes are divided into several connected components. A common component is a coreference group. For each coreference group, a new globally unique identifier for the person is generated and assigned to each denotation expression within the group. Then, based on the globally unique identifier for the person, all denotations under the same identifier are chained together in the order of their appearance in the text into an ordered chain. Each chain records all the denotations of the person and their respective position information, forming a complete coreference tracing path across paragraphs and text blocks, i.e., a cross-text block person coreference chain. Each record contains the globally unique identifier of the person, the person's standard name, a complete list of nodes, the position of the first node, and the span of the paragraph. There are several person coreference chains, and each chain completely covers all the denotations of the same person that appear in the entire text and their precise positions. This allows all the information of the person at any position in the entire text to be obtained by traversing a single chain without having to repeatedly perform coreference determination.

[0027] Step S400: Extract key semantic information from the structured semantic data, obtain core semantic parameters, perform dimensionality reduction encoding, and generate hierarchical semantic vectors.

[0028] Step S400 further includes performing multi-dimensional semantic feature analysis on the structured semantic data to separate and obtain a set of multiple types of semantic units, the set of multiple types of semantic units including entity-type semantic units, relation-type semantic units, event-type semantic units, and attribute-type semantic units; performing importance scoring on the entity-type semantic units, relation-type semantic units, event-type semantic units, and attribute-type semantic units to generate importance scores; and filtering the entity-type semantic units, relation-type semantic units, event-type semantic units, and attribute-type semantic units according to the importance scores to determine the core semantic parameters.

[0029] Preferably, structured semantic data is used as input, and all information contained therein is classified and parsed according to different semantic dimensions. The originally mixed information is separated and extracted according to four semantic types: entity, relation, event, and attribute, forming four independent semantic unit sets, including entity semantic units, relation semantic units, event semantic units, and attribute semantic units. Then, for each entity record in the entity semantic unit set, its importance score is calculated based on multiple dimensions, including global occurrence frequency, cross-segment coverage breadth, relation richness, event participation, attribute richness, and position weight. The importance score is then generated by weighting and summing according to preset weights. Based on the generated importance scores, the four semantic unit sets of entity, relation, event, and attribute are sorted and truncated according to the scores from high to low. The most important items in each category are selected, and the selection results are merged to determine the core semantic parameter set for the output of this stage. This selection effectively removes redundant and low-value information, ensuring that the input for encoding processing is concise and focused on the most critical knowledge content of the entire text. The high-dimensional original feature representations contained in each semantic unit (entity, relation, event, attribute) in the core semantic parameter set are compressed into low-dimensional dense vectors through mathematical transformations. While preserving the core semantic information of the semantic unit, the feature dimension of the data is greatly reduced, so that the semantic information is transformed from a readable natural language form into a numerical vector that can be efficiently computed by a computer. Hierarchical semantic vectors are determined, and each level vector set contains only the vector representations of the semantic units belonging to that level.

[0030] Step S500: Introduce a dedicated retrieval task package and combine it with the hierarchical semantic vector to perform full-text semantic retrieval, obtain a fusion retrieval result set, determine the target text block and target semantic information based on the fusion retrieval result set, and construct a long text context-based deep-targeted retrieval result.

[0031] Step S500 further includes: obtaining user-inputted custom keywords; determining the semantic level category of the custom keywords; extracting text features of the custom keywords and performing retrieval analysis in conjunction with the semantic level category to generate a dedicated retrieval task package; constructing a full-text semantic memory; submitting the dedicated retrieval task package to the full-text semantic memory and performing semantic selection in conjunction with the hierarchical semantic vectors to determine the target semantic vector level; performing vector similarity matching based on the target semantic vector level to obtain an initial retrieval result set; setting a preset recall threshold; when the initial retrieval result set is lower than the recall threshold, performing cross-level supplementary retrieval and incorporating the cross-level supplementary retrieval results into the initial retrieval result set until the initial retrieval result set reaches the recall threshold to obtain the fused retrieval result set.

[0032] Preferably, the system receives search keywords or phrases input by the user in natural language, analyzes the semantic information expressed by the keywords, and determines which semantic level (top, middle, or bottom) the keyword most likely corresponds to. This determines which semantic vector set the search request should be prioritized for matching. Specifically, the system receives user-inputted custom keywords through its interactive interface, performs word segmentation, part-of-speech tagging, and dependency parsing on the input keywords, identifies the core subject, core action, and core limiting conditions within the keywords, matches the parsed keyword features with predefined hierarchical judgment rules, and outputs the semantic level category. Fine-grained text feature extraction is then performed on the user-inputted custom keywords. The extracted text features are then fused and packaged with the semantic level category determined in the first step to form a structured task description object containing search intent, search scope, and search constraints. This includes extracting keyword text features, fusing hierarchical categories to generate a task package, and outputting a dedicated search task package carrying complete search intent information.

[0033] Preferably, the hierarchical semantic vectors (top-level vector, middle-level vector, and bottom-level vector) generated in step four are pre-organized and stored as a full-text semantic memory that can be efficiently retrieved. After submitting the dedicated retrieval task package to the memory, the corresponding vector level is selected as the matching range for this retrieval based on the target level label in the task package. If the target level is "top-level", the top-level semantic vector index is selected as the target level for this retrieval. If the target level is "middle-level", the middle-level semantic vector index is selected as the target level for this retrieval. If the target level is "bottom-level", the bottom-level semantic vector index is selected as the target level for this retrieval. The target semantic vector level identifier for this retrieval is then output.

[0034] Preferably, the user-inputted custom keyword text is input into the same encoding model as in step four. After dimensionality reduction encoding, a query vector with the same dimension as the hierarchical semantic vector is generated. In the vector index corresponding to the target semantic vector level, the similarity between the query vector and each semantic unit vector in the index is calculated (using cosine similarity or inner product). The similarity score between the query vector and each candidate vector is calculated. All candidate vectors are sorted in descending order of similarity score from high to low. A certain number of matching results (e.g., the top 30) are selected as the initial search result set. Each record in the initial search result set contains the similarity score of the matched semantic units. The system includes the unique identifier of the semantic unit, the level to which the semantic unit belongs, and the location of the original text block corresponding to the semantic unit. A minimum recall threshold is preset (e.g., at least 20 results). When the number of results in the initial search result set is less than this threshold, cross-level search is automatically performed to supplement the recall of semantically related vectors from other levels outside the initial target level. The supplemented search results are then merged into the initial result set. This process is repeated until the number of results in the result set reaches the preset threshold. Finally, the complete result set after supplementation and expansion (including the initial search results and all supplemented search results) is output as the fused search result set, ensuring that each search returns a sufficient number of candidate results for filtering.

[0035] Furthermore, step S500 also includes: performing a multi-dimensional association strength assessment on the fused retrieval result set to generate a multi-dimensional association strength assessment result; filtering and sorting the fused retrieval result set according to the multi-dimensional association strength assessment result to select multiple retrieval result items; extracting multiple text blocks as target text blocks based on the multiple retrieval result items; and traversing the full-text semantic memory based on the target text blocks to extract multiple semantic information as target semantic information.

[0036] Preferably, the association strength of each result item in the fused search result set is calculated from multiple dimensions, including semantic vector similarity, keyword hit, keyword weight matching, semantic level matching, entity co-occurrence, and event chain integrity. For each result item in the fused search result set, the scores calculated from each dimension are summarized to form a multidimensional association strength evaluation result for that result item. The scores of each dimension in the multidimensional association strength evaluation result are weighted and fused to obtain a comprehensive association strength score for each result item. Based on this, the fused search result set is sorted in descending order, and several result items with the highest ranking are selected from the sorted results. Then, the corresponding original text blocks are extracted from these selected result items as target text blocks for subsequent operations. The weight coefficients of each evaluation dimension are preset, such as semantic vector similarity weight 0.35, keyword hit weight 0.20, keyword weight matching weight 0.15, semantic level matching weight 0.15, entity co-occurrence weight 0.10, and event chain integrity weight 0.05. The sum of the weight coefficients is 1. For each result item, the scores of each dimension are multiplied by their corresponding weight coefficients, and all weighted results are summed to obtain the overall relevance strength score of the result item. The higher the total score, the stronger the overall relevance of the result item to the user's search intent. All result items in the fused search result set are sorted in descending order of overall relevance strength score. From the sorted fused search result set, the result items with the highest overall relevance strength scores are selected as the selected result items. If there are ties (same total score), the scores of each dimension are compared in turn until a ranking is determined or all are retained. For each selected search result item, the paragraph number of the semantic unit corresponding to the result item is obtained from the full-text semantic memory. Using the global indexing framework, the complete paragraph content corresponding to the structured text block sequence is located using the paragraph number as the index. The paragraph is extracted as the text block corresponding to the result item and arranged in descending order of overall relevance strength score to form a target text block list. Each target text block carries the search result item number and overall relevance strength score of its source.

[0037] Preferably, using the selected target text blocks as clues, the full-text semantic memory is traversed to extract the structured semantic information corresponding to each target text block, including entities, relationships, events, attributes, etc. in the text block, as well as the contextual semantic information associated with the text block at the semantic level. The extracted information is integrated into target semantic information for filling the final search results. Specifically, taking the paragraph number of the target text block as the center, the context scope of the paragraph is determined according to the chapter hierarchy marked in the global index framework. Within the determined context scope, the topic of the chapter to which the paragraph belongs is extracted by traversing the full-text semantic memory. The system includes tags and stage descriptions, entity state continuation information related to the current search intent in the preceding and following paragraphs, and the association attributes of the core entities in the paragraph in other paragraphs. For cases where the context scope of multiple target text blocks may overlap, duplicate semantic information is extracted and merged, retaining a unique semantic record and marking all its occurrence positions. Finally, the semantic information of each target text block, its direct semantic information, and context-related semantic information are organized and output according to the target text block. This output includes the original content of each target text block, the structured semantic record, the stage context information of the chapter to which it belongs, and the semantic connection information with adjacent text blocks.

[0038] In the above text, refer to Figure 1 A method for deep context-based targeted retrieval of long texts according to embodiments of the present invention has been described in detail. Next, reference will be made to... Figure 2 This invention describes a deep context-oriented retrieval system for long texts according to an embodiment of the present invention.

[0039] According to an embodiment of the present invention, a deep context-oriented retrieval system for long texts addresses the technical problems in existing technologies, such as semantic fragmentation in long texts, loose global logic, disconnect between complex retrieval intents and shallow semantic matching, and insufficient retrieval efficiency and accuracy due to the lack of hierarchical indexes for unstructured data. This system achieves the technical effect of improving the retrieval accuracy, contextual coherence, processing efficiency, and result reliability of long texts. Figure 2 As shown, a deep-directed retrieval system for long-text context includes: a structured text block sequence acquisition module 10, a standardized structured data construction module 20, a structured semantic data acquisition module 30, a hierarchical semantic vector generation module 40, and a deep-directed retrieval result acquisition module 50.

[0040] The structured text block sequence acquisition module 10 is used to normalize sentences in long texts, reconstruct paragraphs based on the normalization results, and obtain a structured text block sequence, which includes a global index framework. The standardized structured data construction module 20 is used to perform hierarchical semantic structured extraction on the structured text block sequence and construct standardized structured data. The structured semantic data acquisition module 30 is used to perform global consistency verification on the full-text character attributes based on the global index framework and the standardized structured data, and perform automatic alignment correction based on the verification results to obtain structured semantic data. The hierarchical semantic vector generation module 40 is used to extract semantic-level key information from the structured semantic data, obtain core semantic parameters, perform dimensionality reduction encoding, and generate hierarchical semantic vectors. The deep-directed retrieval result acquisition module 50 is used to introduce a dedicated retrieval task package and combine it with the hierarchical semantic vectors to perform full-text semantic retrieval, obtain a fused retrieval result set, determine the target text blocks and target semantic information based on the fused retrieval result set, and concatenate and fill them to construct a deep-directed retrieval result for the long text context.

[0041] The following will describe in detail the specific configuration of the standardized structured data construction module 20. The standardized structured data construction module 20 further includes: performing top-level semantic extraction on the structured text block sequence to obtain overall information of the entire text; performing mid-level semantic extraction on the structured text block sequence to obtain story context information; performing bottom-level semantic extraction on the structured text block sequence to obtain single-point text information; and structurally integrating the overall information of the entire text, the story context information, and the single-point text information to construct standardized structured data.

[0042] The specific configuration of the structured semantic data acquisition module 30 will be described in detail below. The structured semantic data acquisition module 30 further includes: constructing a globally unique identifier for a person based on the global index framework and the standardized structured data; using the globally unique identifier as an index to construct a set of attributes to be verified for the person; performing coreference resolution processing based on the set of attributes to be verified to generate a cross-text block coreference chain for the person; traversing the standardized structured data according to the coreference chain for text block matching to extract person attribute information; performing a global traversal comparison based on the person attribute information to detect whether there are conflicts in the person attribute information of the same task in different text blocks, and generating a conflict detection result; when the conflict detection result indicates a conflict, performing conflict resolution analysis based on the person attribute information to determine a baseline attribute value; automatically correcting the person attribute information according to the baseline attribute value to generate a task attribute correction record; performing a second global traversal verification on the person attribute information based on the task attribute correction record; and generating the structured semantic data when the verification passes.

[0043] The following will describe in detail the specific configuration of the structured semantic data acquisition module 30. The structured semantic data acquisition module 30 further includes: extracting features from the referential expressions of the set of attributes of the person to be verified, obtaining character features, contextual semantic features, and positional features of the referential expressions; performing semantic similarity calculation based on the character features, contextual semantic features, and positional features to generate a referential similarity matrix; when the semantic similarity of two referential expressions in the referential similarity matrix exceeds a preset similarity threshold, determining that the two referential expressions point to the same person, and assigning the same globally unique identifier for the person to the two referential expressions; traversing the referential similarity matrix, linking referential expressions that share the globally unique identifier for the person, and constructing a cross-text block co-reference chain for the person.

[0044] The specific configuration of the hierarchical semantic vector generation module 40 will be described in detail below. The hierarchical semantic vector generation module 40 further includes: performing multi-dimensional semantic feature analysis on the structured semantic data to separate and obtain a set of multiple types of semantic units, the set of multiple types of semantic units including entity-type semantic units, relation-type semantic units, event-type semantic units, and attribute-type semantic units; generating an importance score based on the entity-type semantic units, relation-type semantic units, event-type semantic units, and attribute-type semantic units; and filtering the entity-type semantic units, relation-type semantic units, event-type semantic units, and attribute-type semantic units according to the importance score to determine the core semantic parameters.

[0045] The following describes in detail the specific configuration of the deep-targeted retrieval result acquisition module 50. The deep-targeted retrieval result acquisition module 50 further includes: acquiring user-inputted custom keywords; determining the semantic level category of the custom keywords; extracting textual features of the custom keywords and performing retrieval analysis in conjunction with the semantic level category to generate a dedicated retrieval task package; constructing a full-text semantic memory; submitting the dedicated retrieval task package to the full-text semantic memory and performing semantic selection in conjunction with the hierarchical semantic vectors to determine the target semantic vector level; performing vector similarity matching based on the target semantic vector level to obtain an initial retrieval result set; and setting a preset recall threshold. When the initial retrieval result set is lower than the recall threshold, cross-level supplementary retrieval is performed, and the cross-level supplementary retrieval results are merged into the initial retrieval result set until the initial retrieval result set reaches the recall threshold, thereby obtaining the fused retrieval result set.

[0046] The following will describe in detail the specific configuration of the deep-targeted retrieval result acquisition module 50. The deep-targeted retrieval result acquisition module 50 further includes: performing a multi-dimensional association strength evaluation on the fused retrieval result set to generate a multi-dimensional association strength evaluation result; filtering and sorting the fused retrieval result set according to the association strength of the multi-dimensional association strength evaluation result, selecting multiple retrieval result items, and extracting multiple text blocks as target text blocks based on the multiple retrieval result items; and traversing the full-text semantic memory based on the target text blocks to extract multiple semantic information as target semantic information.

[0047] The long text context-based deep targeting retrieval system provided in this embodiment of the invention can execute the long text context-based deep targeting retrieval method provided in this embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0048] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for deep context-based targeted retrieval of long texts, characterized in that, The method includes: Sentence normalization is performed on long texts, and paragraph reconstruction is performed based on the sentence normalization results to obtain a structured text block sequence, which contains a global index framework. Hierarchical semantic structure extraction is performed on the structured text block sequence to construct standardized structured data; Based on the global indexing framework and the standardized structured data, a global consistency check is performed on the attributes of the characters in the full text, and automatic alignment and correction are performed according to the check results to obtain structured semantic data. Semantic key information is extracted from the structured semantic data to obtain core semantic parameters, which are then dimensionality-reduced and encoded to generate hierarchical semantic vectors. A dedicated retrieval task package is introduced and combined with the hierarchical semantic vector to perform full-text semantic retrieval, and a fusion retrieval result set is obtained. Based on the fusion retrieval result set, the target text block and target semantic information are determined and concatenated to fill the data, thereby constructing a deep context-oriented retrieval result for long texts.

2. The method for deep contextual targeting retrieval of long texts as described in claim 1, characterized in that, Hierarchical semantic structure extraction is performed on the structured text block sequence to construct standardized structured data. The method includes: Perform top-level semantic extraction on the structured text block sequence to obtain overall information of the entire text; Perform mid-level semantic extraction on the structured text block sequence to obtain story context information; Perform low-level semantic extraction on the structured text block sequence to obtain single-point text information; The overall information of the full text, the story context information, and the individual text information are structurally integrated to construct standard structured data.

3. The method for deep contextual targeting retrieval of long texts as described in claim 1, characterized in that, Based on the global indexing framework and the standardized structured data, a global consistency check is performed on the attributes of all individuals in the full text. Automatic alignment and correction are then performed based on the check results to obtain structured semantic data. The method includes: Based on the global indexing framework and the standardized structured data, a globally unique identifier for each person is constructed. The globally unique identifier of the person is used as an index to construct a set of attributes of the person to be verified. Based on the set of attributes of the person to be verified, coreference resolution processing is performed to generate a coreference chain of the person across text blocks. Text block matching is performed based on the standardized structure data traversed according to the coreference chain of the person to extract the person's attribute information; Based on the character attribute information, a global traversal comparison is performed to detect whether there are conflicts in the character attribute information of the same task in different text blocks, and a conflict detection result is generated. When the conflict detection result indicates that a conflict exists, a conflict resolution analysis is performed based on the character attribute information to determine the baseline attribute value. The character attribute information is automatically corrected according to the aforementioned baseline attribute values, and a task attribute correction record is generated. Based on the task attribute correction record, the character attribute information is subjected to a second global traversal verification. When the verification passes, the structured semantic data is generated.

4. The long text context-based deep targeting retrieval method as described in claim 3, characterized in that, Based on the set of attributes of the person to be verified, coreference resolution processing is performed to generate a cross-text block coreference chain of persons. The method includes: Feature extraction is performed on the referential expressions of the attribute set of the person to be verified to obtain character features, contextual semantic features, and positional features of the referential expressions; Based on the character features, the contextual semantic features, and the positional features, semantic similarity calculation is performed to generate an index similarity matrix; When the semantic similarity of two referential expressions in the referential similarity matrix exceeds a preset similarity threshold, the two referential expressions are determined to point to the same person, and the same globally unique identifier for the person is assigned to the two referential expressions. Traverse the reference similarity matrix and link the reference expressions that share the globally unique identifier of the person to construct the reference chain of the person across text blocks.

5. The method for deep contextual targeting retrieval of long texts as described in claim 1, characterized in that, The method for extracting key semantic information at the semantic level from the structured semantic data to obtain core semantic parameters includes: The structured semantic data is parsed using multi-dimensional semantic features to separate and obtain a set of multiple types of semantic units, which includes entity-type semantic units, relation-type semantic units, event-type semantic units, and attribute-type semantic units. An importance score is generated based on the entity class semantic units, the relation class semantic units, the event class semantic units, and the attribute class semantic units. The entity-type semantic units, relation-type semantic units, event-type semantic units, and attribute-type semantic units are filtered based on the importance score to determine the core semantic parameters.

6. The method for deep contextual targeting retrieval of long texts as described in claim 1, characterized in that, A dedicated retrieval task package is introduced and combined with the hierarchical semantic vector to perform full-text semantic retrieval, obtaining a fused retrieval result set. The method includes: Obtain user-inputted custom keywords, determine the semantic level of the custom keywords, and identify the semantic level category; Extract the text features of the custom keywords and combine them with the semantic hierarchy categories for retrieval analysis to generate a dedicated retrieval task package; Construct a full-text semantic memory, submit the dedicated retrieval task package to the full-text semantic memory, combine the hierarchical semantic vector to perform semantic selection, and determine the target semantic vector level; Based on the target semantic vector hierarchy, vector similarity matching is performed to obtain an initial retrieval result set; A preset recall threshold is set. When the initial search result set is lower than the recall threshold, a cross-layer supplementary search is performed, and the cross-layer supplementary search results are merged into the initial search result set until the initial search result set reaches the recall threshold, thereby obtaining the fused search result set.

7. The long text context-based deep targeting retrieval method as described in claim 6, characterized in that, Based on the fused retrieval result set, target text blocks and target semantic information are determined and concatenated to fill in the data, constructing a deep context-oriented retrieval result for a long text. The method includes: A multidimensional association strength assessment is performed on the fused retrieval result set to generate a multidimensional association strength assessment result; Based on the multidimensional association strength evaluation results, the fused retrieval result set is filtered and sorted by association strength, and multiple retrieval result items are selected. Based on the multiple retrieval result items, multiple text blocks are extracted as target text blocks. Based on the target text block, the full-text semantic memory is traversed to extract multiple semantic information as target semantic information.

8. A deep context-based targeted retrieval system for long texts, characterized in that, The system is used to implement the deep context-oriented retrieval method for long texts as described in any one of claims 1 to 7, and the system includes: The structured text block sequence acquisition module is used to normalize sentences in long texts, reconstruct paragraphs based on the sentence normalization results, and obtain a structured text block sequence, which includes a global index framework. A standardized structured data construction module is used to perform hierarchical semantic structure extraction on the structured text block sequence to construct standardized structured data; The structured semantic data acquisition module is used to perform global consistency verification on the full-text character attributes based on the global index framework and the standardized structured data, and to perform automatic alignment correction based on the verification results to obtain structured semantic data. The hierarchical semantic vector generation module is used to extract key semantic information from the structured semantic data, obtain core semantic parameters, perform dimensionality reduction encoding, and generate hierarchical semantic vectors. The deep-targeted retrieval result acquisition module is used to introduce a dedicated retrieval task package and combine it with the hierarchical semantic vector to perform full-text semantic retrieval, obtain a fused retrieval result set, determine the target text block and target semantic information based on the fused retrieval result set, and concatenate and fill them to construct a deep-targeted retrieval result for the context of a long text.

Images