Method and system for keyword-based search of docx document content

Abstract

The application belongs to the technical field of text processing, and particularly relates to a method and system for retrieving DOCX document content based on keywords, which comprises the following steps: analyzing the Office Open XML structure of the DOCX document, combining multi-dimensional features such as style names, and accurately distinguishing titles from texts by using a title classification score model, so that the semantic hierarchical structure of the document is effectively retained; secondly, a multi-level semantic expansion mechanism is introduced, Sentence-BERT, HowNet knowledge base and Word2Vec model are fused, intelligent expansion of synonyms and near-synonyms of keywords is realized, and the recall rate and semantic understanding ability of the retrieval are significantly improved; thirdly, the BM25 model is used to calculate the relevance score by combining paragraph length normalization and structure position weight, so that the ranking of the retrieval result is more accurate and reasonable; and finally, the construction of the inverted index is combined with position coding and compression optimization strategies, and the retrieval efficiency and storage performance are taken into account.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of text processing, and particularly relates to a method and system for retrieving DOCX document content based on keywords, a terminal and a storage medium. BACKGROUND

[0002] With the popularity of digital office, the DOCX format document generated by Microsoft Word has become one of the most mainstream document formats in enterprises, educational institutions and scientific research fields. However, in the face of the increasing number of electronic documents, how to quickly and accurately retrieve the required information from a large number of DOCX documents has become a technical problem to be solved in the current document processing field.

[0003] The current common document retrieval scheme mainly has the following limitations: first, although the traditional full-text search engine (such as the system based on Lucene) can realize basic keyword matching, it cannot effectively analyze the complex internal structure of the DOCX document (such as the title level, table data, style information, etc.), resulting in a lack of semantic hierarchy of the retrieval results. Secondly, most of the existing retrieval tools use literal matching strategies, which are difficult to cope with the complexity of Chinese semantics, especially the lack of recognition ability for synonyms, near-synonyms and variant forms of professional terms. In addition, the traditional method mainly relies on the edit distance algorithm when processing fuzzy queries, and fails to combine the context semantics for intelligent expansion, so that a large amount of irrelevant content often appears in the retrieval results.

[0004] It is particularly important to note that the DOCX document uses the Office Open XML (OOXML) standard for storage, and contains a large amount of XML structured data inside. The existing technical solutions usually convert the DOCX document into pure text for retrieval, which will cause the loss of style information, resulting in the inability to distinguish the importance difference between the title and the body. SUMMARY

[0005] In view of the above deficiencies of the prior art, the application provides a method, system, terminal and storage medium for retrieving DOCX document content based on keywords.

[0006] In a first aspect, the application provides a method for retrieving DOCX document content based on keywords, comprising:

[0007] S1, loading the DOCX file through a document parsing module, traversing all paragraphs and tables, extracting the text and style information of each paragraph, and classifying the paragraphs into titles or bodies according to at least one of the characteristics of style name, font size and bold attribute, to form structured text data;

[0008] S2, text preprocessing is performed on the structured text data: non-English characters and redundant spaces are removed, a word segmentation tool is used to segment the remaining text, and a stop word table is loaded to filter meaningless words, and a purified word sequence is obtained;

[0009] S3, an inverted index is constructed based on the word sequence, and a one-to-one mapping relationship of keywords to occurrence positions is established, the positions at least including paragraph numbers and paragraph internal offsets;

[0010] S4, receiving a user input keyword, using a natural language processing model to perform semantic expansion on the keyword, obtaining a synonym and / or near-synonym set of the original keyword, forming an extended keyword list;

[0011] S5, performing multi-mode matching in the inverted index according to the extended keyword list, the multi-mode at least including one of exact matching, fuzzy matching, and synonym matching, obtaining an initial matching paragraph set;

[0012] S6, calculating a relevance score for each paragraph in the initial matching paragraph set, and outputting a ranking result in descending order of the relevance score;

[0013] S7, highlighting the keyword in the ranking result in the corresponding paragraph, and extracting the matching sentence and its adjacent sentences to generate an abstract, the length of the abstract being dynamically determined by the maximum number of sentences configured by the user;

[0014] S8, exporting the highlighted paragraph and the abstract in HTML, TXT or PDF format, completing the persistence of the retrieval result.

[0015] Further improvements of the technical solution are as follows:

[0016] S11, loading a DOCX file of a specified path through a document parsing module, parsing the Office Open XML structure of the document by using a python-docx library, generating a document object model tree, the model tree including all paragraph elements and table elements, and ensuring the integrity of the document structure;

[0017] S12, traversing each paragraph element in the document object model tree, extracting the text content of the paragraph, and obtaining the style attribute information of the paragraph, the style attribute information at least including a style name, a font size, and a bold attribute, wherein the style name is extracted from the name attribute of the paragraph style, the font size is extracted from the font size attribute, and the bold attribute is extracted from the font weight attribute;

[0018] S13, based on the extracted style attribute information, calculating a title classification score of each paragraph, and classifying the paragraph as a title or a body according to the comparison result of the score and a preset threshold, wherein the calculation formula of the title classification score is:

[0019] ;

[0020] wherein, denotes the title classification score of paragraph p; is an indicator function, 1 if the style name of paragraph p belongs to the predefined title style set T, otherwise 0; denotes the font size of paragraph p in pounds; denotes the bold attribute of paragraph p, 1 if bold, otherwise 0; denotes the base font size, a preset constant; 、 、 is a weight coefficient; if , paragraph p is classified as a title, otherwise as a text, wherein θ is a preset score threshold.

[0021] Further improvements of the technical solution are that step S2 comprises:

[0022] S21, denoising the structured text data, removing non-English characters and extra spaces by regular expression matching and character encoding recognition, and retaining valid text content;

[0023] S22, performing word segmentation processing on the denoised text by using a Chinese word segmentation tool, calculating the optimal word segmentation path by dictionary matching and hidden Markov model, and dividing the text into a word sequence;

[0024] S23, loading a stop word table to filter the word sequence, and combining word frequency statistics and semantic weight calculation to determine the elimination threshold of meaningless words:

[0025] ;

[0026] wherein, denotes the filtering weight of word w; is the word frequency; N is the total number of words; is the document frequency containing word w; is the total number of documents; when word w is filtered as a meaningless word, is a preset filtering threshold.

[0027] Further improvements of the technical solution are that step S3 comprises:

[0028] S31, position coding is performed on the purified word sequence, a unique paragraph number and paragraph internal offset are assigned to each word, and a word position mapping table is established; the position coding adopts a segmented linear mapping function:

[0029] ;

[0030] wherein, denotes the position coordinates of the word; p is the paragraph number; is the offset within the paragraph; is the average paragraph length; is the sequential index of the word in the full text;

[0031] S32, based on the word position mapping table, an inverted index is constructed, and a hash mapping and a linked list structure are used to store the mapping relationship of the keyword to the occurrence position; the definition formula of the inverted index item is:

[0032] ;

[0033] wherein, denotes the inverted index item of the keyword t; is the paragraph number; is the offset within the paragraph; is the frequency of the keyword appearing in the position , m is the total number of paragraphs containing the keyword, and n is the number of keyword occurrence positions within a single paragraph;

[0034] S33, the inverted index is compressed and optimized, and a multi-level cache mechanism is established to improve the retrieval efficiency; the index compression rate calculation formula is:

[0035] ;

[0036] wherein, denotes the index compression rate; is the size of the compressed index; is the original index size; the cache hit rate optimization formula is:

[0037] ;

[0038] wherein, is the cache hit rate; is the number of times of hitting the cache; is the total number of accesses.

[0039] Further improvement of the technical solution, step S4 includes:

[0040] S41, receiving the keyword string input by the user, standardizing and preprocessing the keyword, including converting the keyword to a unified lowercase form, removing preset characters and unnecessary spaces, and using a Chinese word segmentation tool to segment the composite keyword into single word units, forming a standardized keyword sequence;

[0041] S42, using a pre-trained natural language processing model to perform semantic analysis on the standardized keyword sequence, and using a multi-level semantic expansion strategy to obtain a set of synonyms and near-synonyms for each keyword, the multi-level semantic expansion strategy including the following calculation process:

[0042] First expansion: using a Sentence-BERT sentence encoding model to calculate the semantic similarity between the keyword and the candidate word, the semantic similarity calculation formula being:

[0043] ;

[0044] wherein A and B are the vector representations of the keyword and the candidate word, respectively; is the cosine similarity value; when the candidate word is added to the expansion set; is a preset similarity threshold;

[0045] Second expansion: combining the HowNet synonym knowledge base to obtain explicit synonyms of the keyword and verifying the validity of the synonyms;

[0046] Third expansion: using a Word2Vec word vector model to calculate the cosine similarity between the keyword and the candidate word in the vector space obtained by training on the domain corpus, and selecting the top K words with the highest similarity as the near-synonyms, K being a preset expansion number;

[0047] S43, de-duplicating and merging the synonym and near-synonym sets obtained by multi-level semantic expansion to form an expanded keyword list, and assigning a weight to each expanded keyword, the weight calculation formula being:

[0048] ;

[0049] wherein, represents the weight of the expanded keyword e; is the semantic similarity between the expanded word e and the original keyword k; is an expansion level coefficient; and are weight coefficients.

[0050] Further improvements of the technical solution include that step S5 includes:

[0051] S51, parsing the expanded keyword list and constructing a query expression tree according to the user-input logical operators, wherein the logical operators include AND and OR, the leaf nodes of the query expression tree representing individual keywords, the internal nodes representing logical operations, and ensuring correct operator precedence;

[0052] S52, for each keyword node in the query expression tree, perform multi-pattern matching in the inverted index, the multi-patterns including exact match, fuzzy match and synonym match; the fuzzy match uses edit distance similarity calculation:

[0053] ;

[0054] wherein, denotes the fuzzy match similarity of words and ; denotes the edit distance; denotes the character length of word ; when , it is considered as a match; is a preset fuzzy match threshold;

[0055] The synonym match uses semantic similarity calculation:

[0056] ;

[0057] wherein, denotes the semantic similarity of words and ; denotes the word vector representation of word ; when , it is considered as a match; is a preset synonym match threshold;

[0058] S53, combine the matching results according to the logical operations of the query expression tree, take the intersection of paragraph sets for AND operation, take the union set for OR operation, and calculate the matching strength score of each paragraph:

[0059] ;

[0060] wherein, denotes the matching strength score of paragraph p; r is the total number of expanded keywords; is the weight of expanded keyword ; is an indicator function, which takes the value of 1 when paragraph p contains keyword , otherwise 0; finally, select the paragraphs with matching strength score higher than the corresponding preset threshold to form the initial matching paragraph set.

[0061] Further improvement of the technical solution is that step S6 includes:

[0062] S61, for each paragraph in the initial matching paragraph set, calculate an original relevance score based on keyword matching, which is calculated in combination with the term frequency of the keyword in the paragraph and the inverse document frequency, wherein the term frequency represents the number of occurrences of the keyword in the paragraph, and the inverse document frequency represents the rarity of the keyword in the entire document set; the original relevance score is calculated using the BM25 model, and the formula is:

[0063] ;

[0064] wherein, represents the original relevance score of the paragraph p; represents the i-th extended keyword, and r is the total number of extended keywords; represents the inverse document frequency of the keyword , and the calculation formula is , wherein M is the total number of documents in the document set; is the number of documents containing the keyword ; represents the term frequency of the keyword in the paragraph p; represents the length of the paragraph p; represents the average length of all paragraphs in the document set; and b are adjustable parameters, wherein controls the saturation of the term frequency, and is usually valued at 1.2 to 2.0, and b controls the influence of paragraph length normalization, and is usually valued at 0.5 to 0.8;

[0065] S62, the original relevance score is subjected to paragraph length normalization processing to eliminate the bias of long paragraphs due to high term frequency, and to ensure the comparability of scores of paragraphs of different lengths; the normalization processing uses a length weight adjustment formula:

[0066] ;

[0067] wherein, represents the normalized relevance score; is a normalization intensity coefficient; represents an exponential function for smoothing the influence of length difference;

[0068] S63, the normalized score is adjusted in combination with the structural position weight of the paragraph, wherein the weight of the title paragraph is higher than that of the body paragraph, and a final relevance score is calculated, and the ranking result is output in descending order of score; the final relevance score calculation formula is:

[0069] ;

[0070] wherein, represents the final relevance score of the paragraph p; is a function of indicating, taking value 1 when the paragraph p is classified as a title, otherwise 0; is a title weight addition coefficient; when ranking, the paragraphs are arranged in descending order of the value from high to low, to generate the final search result list. is a title weight addition coefficient; when ranking, the paragraphs are arranged in descending order of the value from high to low, to generate the final search result list.

[0071] In a second aspect, the application provides a system for searching DOCX document content based on keywords, comprising:

[0072] A document parsing module is configured to load a DOCX file, traverse all paragraphs and table elements in the document, extract text content and style information of each paragraph, and classify the paragraphs as titles or texts according to at least one of the following characteristics: style name, font size, and bold attribute, to form structured text data.

[0073] A text preprocessing module is connected to the document parsing module and configured to preprocess the structured text data, including removing non-English characters and redundant spaces, performing word segmentation processing on the text using a word segmentation tool, and loading a stop word table to filter meaningless words, to obtain a purified word sequence.

[0074] An inverted index construction module is connected to the text preprocessing module and configured to construct an inverted index based on the purified word sequence, to establish a one-to-one mapping relationship between keywords and occurrence positions, wherein the position information at least includes paragraph number and paragraph internal offset.

[0075] A semantic expansion module is configured to receive a keyword input by a user, perform semantic analysis on the keyword using a natural language processing model, obtain a synonym and / or near-synonym set of the original keyword, and form an expanded keyword list.

[0076] A multi-mode matching module is connected to the semantic expansion module and the inverted index construction module, and is configured to perform multi-mode matching in the inverted index according to the expanded keyword list, wherein the matching mode at least includes one of exact matching, fuzzy matching, and synonym matching, to obtain an initial matching paragraph set.

[0077] A relevance calculation module is connected to the multi-mode matching module and is configured to calculate a relevance score for each paragraph in the initial matching paragraph set, and arrange the paragraphs in descending order of the score to generate a ranking result.

[0078] A result display module is connected to the relevance calculation module and is configured to highlight the keywords in the corresponding paragraphs in the ranking result, and extract matching sentences and their adjacent sentences to generate an abstract, wherein the length of the abstract is dynamically determined by a maximum sentence number configured by the user.

[0079] A result export module is connected to the result display module and is configured to export the highlighted paragraphs and the abstract in HTML, TXT, or PDF format, to realize persistent storage of the search result.

[0080] In a third aspect, the present application provides a terminal, comprising:

[0081] a processor, a memory, wherein,

[0082] the memory is configured to store a computer program,

[0083] the processor is configured to call and run the computer program from the memory, so that the terminal executes the method of the terminal described above.

[0084] In a fourth aspect, the present application provides a computer storage medium, the computer readable storage medium stores instructions, when the instructions are run on a computer, the computer executes the method described in the above aspects.

[0085] The present application has the beneficial effects of:

[0086] The present application accurately distinguishes between titles and texts by deeply analyzing the Office Open XML structure of DOCX documents, combining multi-dimensional features such as style name, font size, and bold attributes, and using title classification score models. It effectively preserves the semantic hierarchical structure of the document and overcomes the semantic loss problem caused by text flattening in traditional methods. Secondly, a multi-level semantic expansion mechanism is introduced, combining Sentence-BERT, HowNet knowledge base, and Word2Vec model to realize intelligent expansion of synonyms and near synonyms of keywords, significantly improving the recall rate and semantic understanding ability of retrieval. Thirdly, the BM25 model is used to calculate the relevance score by combining paragraph length normalization and structure position weight (such as title weighting), making the retrieval result sorting more accurate and reasonable. The construction of inverted index combines position coding and compression optimization strategy, taking into account the retrieval efficiency and storage performance. In addition, the system supports highlight display, dynamic abstract generation, and multi-format export, greatly improving the user interaction experience. The overall scheme realizes the whole process optimization from structure analysis, semantic understanding to intelligent sorting, significantly improving the accuracy, intelligence level and practicality of DOCX document retrieval. BRIEF DESCRIPTION OF DRAWINGS

[0087] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, for those of ordinary skill in the art, without paying creative labor, other drawings can also be obtained from these drawings.

[0088] Figure 1 A schematic flowchart of the method of an embodiment of the present application.

[0089] Figure 2 A schematic block diagram of the system of an embodiment of the present application.

[0090] Figure 3 A structure diagram of a terminal provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0091] In order to make the objectives, features and advantages of the present application more obvious and easy to understand, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the embodiments. Obviously, the embodiments described below are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0092] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0093] The method for retrieving DOCX document content based on keywords provided by the embodiment of the present application is executed by a computer device, and accordingly, the system for retrieving DOCX document content based on keywords runs in the computer device.

[0094] Figure 1 is a schematic flow chart of the method of an embodiment of the present application. Wherein, Figure 1 The execution subject can be a system for retrieving DOCX document content based on keywords. The order of steps in the flow chart can be changed, and some can be omitted according to different needs.

[0095] As Figure 1 shown, the method comprises:

[0096] S1, loading a DOCX file through a document parsing module, traversing all paragraphs and tables, extracting paragraph text and style information, and classifying the paragraphs into titles or texts according to at least one of the characteristics of style name, font size and bold attribute, and forming structured text data;

[0097] S2, text preprocessing on the structured text data: removing non-English characters and redundant spaces, using a word segmentation tool to segment the remaining text, and loading a stop word table to filter meaningless words, to obtain a purified word sequence;

[0098] S3, constructing an inverted index based on the word sequence, establishing a one-to-one mapping relationship from keywords to occurrence positions, and the positions at least including paragraph numbers and paragraph internal offsets;

[0099] S4. Receive keywords input by the user, use a natural language processing model to perform semantic expansion on the keywords, obtain a set of synonyms and / or near-synonyms of the original keywords, and form an expanded keyword list;

[0100] S5. Perform multi-pattern matching in the inverted index based on the expanded keyword list. The multi-pattern matching includes at least one of exact matching, fuzzy matching, and synonym matching to obtain the initial set of matching paragraphs.

[0101] S6. Calculate the relevance score for each paragraph in the initial set of matched paragraphs, and output the sorting results after sorting them in descending order of relevance score.

[0102] S7. Highlight the keywords in the sorting results in the corresponding paragraphs, and extract the matching sentences and their adjacent sentences to generate a summary. The length of the summary is dynamically determined by the maximum number of sentences configured by the user.

[0103] S8. Export the highlighted paragraphs and summaries as HTML, TXT, or PDF formats to persist the search results.

[0104] To facilitate understanding of the present invention, the following description further illustrates the method for retrieving DOCX document content based on keywords, using the principle of the present invention and the process of retrieving DOCX document content based on keywords in the embodiments.

[0105] First, step S1 includes:

[0106] S11. Load the DOCX file at the specified path through the document parsing module, use the python-docx library to parse the Office Open XML structure of the document, generate a document object model tree, which contains all paragraph elements and table elements, and ensures the integrity of the document structure.

[0107] S12. Traverse each paragraph element in the document object model tree, extract the text content of the paragraph, and obtain the style attribute information of the paragraph. The style attribute information includes at least the style name, font size, and bold attribute. The style name is extracted from the name attribute of the paragraph style, the font size is extracted from the font size attribute, and the bold attribute is extracted from the font weight attribute.

[0108] S13. Based on the extracted style attribute information, calculate the title classification score for each paragraph, and classify the paragraph into title or body text according to the comparison result between the score and the preset threshold. The formula for calculating the title classification score is as follows:

[0109] ;

[0110] in, denotes the title classification score of paragraph p; is an indicator function, when the style name of paragraph p is 1 if it belongs to the predefined set of title styles T, otherwise 0; denotes the font size of paragraph p in pounds; denotes the bold attribute of paragraph p, which is 1 if bold, otherwise 0; denotes the base font size, which is a pre-defined constant; is a weight coefficient; if then paragraph p is classified as a title, otherwise as a body text, where θ is a pre-defined score threshold.

[0111] The python-docx library (version 0.8.11, which is well compatible with the OOXML standard and supports all versions of DOCX 2007-2021) is used as the core parsing tool, and it can be installed by pip install python-docx==0.8.11. If the document to be parsed is in the old.doc format, it needs to be converted first by the unoconv tool (version 0.9.0) with the command unoconv -f docx source_file.doc to ensure that the input document is in the.docx format.

[0112] Loading logic: The absolute path (D: / document_repo / technical_document.docx) is obtained by the file path construction function get_doc_path(doc_name), and the document is loaded by calling docx.Document(docx_path).

[0113] Exception handling: For the three common exceptions docx.opc.exceptions.PackageNotFoundError (file corruption or illegal format), FileNotFoundError (path error), and PermissionError (insufficient permissions), the processing flow of "logging error (including document path, error time, error details) -> returning empty document object -> terminating the current parsing task" is executed respectively. The log storage path is. / log / doc_parse_error.log, which is convenient for subsequent problem troubleshooting.

[0114] After loading the document, the following two verifications are performed to ensure the integrity of the OOXML structure:

[0115] ​​Element Integrity: Check if doc.element.body contains at least one paragraph element (w:p tag) or table element (w:tbl tag). If both are empty, determine as "empty document" and output a warning.

[0116] Attribute Integrity: Randomly select 5% of paragraph elements and verify if they contain core sub-tags such as w:style, w:r, etc. If the missing rate exceeds 10%, determine as "structurally damaged document" and trigger reloading or format repair processes (call docx.oxml.ns module to repair missing tags).

[0117] Use doc.element.body.iter() to traverse all sub-elements under the document body (w:body), distinguish element types by element tag suffix (p for paragraph, tbl for table), and store them in the elements = [] list. Each list item is a tuple (element type, element object) (e.g., ("paragraph", para_obj), ("table", tbl_obj)).

[0118] Text Extraction: For each paragraph object para, call para.text.strip() to get the text content. If the text is empty (length < 1), mark it as "empty paragraph" and skip subsequent style extraction.

[0119] Style Name Extraction: Get the style name through para.style.name. If para.style is None (style information missing), default to "Normal".

[0120] Font Size Extraction: Traverse the paragraph's text run para.runs and take the font size (run.font.size.pt) of the first non-empty run in "pounds (pt)". If the font size of all runs is None, default to 12pt (common font size for Chinese documents).

[0121] Bold Attribute Extraction: Determine the run.font.bold attribute of the first non-empty run. If it is True or None (default bold when not explicitly set in some documents), take the value 1. If it is False, take the value 0.

[0122] Recursive parsing table: for each table object tbl, traverse its rows (tbl.rows) and cells (row.cells), call cell.text.strip() to extract each cell text, and merge them into "table sub-paragraphs" (format as "table-row-number-column-number: text content", e.g. "table1-2-3: search accuracy ≥ 90%");

[0123] Context association: put "table sub-paragraphs" into the "associated content" of the previous non-empty paragraph where the table is located, ensuring that the table text is semantically coherent with the context, for example, the table is located after paragraph P5, and the table sub-paragraph is stored as an item of the "content" field of P5.

[0124] Score calculation: for each paragraph p, substitute the above formula to calculate ;

[0125] Type determination: if (i.e. ), it is determined as "title", and further subdivided into levels according to the score, as a first-level title, as a second-level title, as a third-level title; if , it is determined as "text".

[0126] Mixed style paragraph: if a paragraph contains both title style and non-title style runs, the style corresponding to the run with a proportion ≥ 50% is taken as .

[0127] Short paragraph determination: if the paragraph text length < 10 characters and , even if , it is still determined as "potential title", which needs to be manually reviewed (the review prompt can be output through the system interface).

[0128] Secondly, step S2 includes:

[0129] S21, denoising the structured text data, removing non-English characters and extra spaces by regular expression matching and character encoding recognition, and retaining valid text content;

[0130] S22, using Chinese word segmentation tool to process the denoised text, calculating the optimal segmentation path by dictionary matching and hidden Markov model, and dividing the text into word sequences;

[0131] S23, load the stop word list to filter the word sequence, combine word frequency statistics and semantic weight calculation to remove meaningless words:

[0132] ;

[0133] wherein, represents the filtering weight of the vocabulary w; is the word frequency; N is the total number of words; is the document frequency containing the vocabulary w; is the total number of documents; when the vocabulary w is filtered as a meaningless vocabulary, is the preset filtering threshold.

[0134] The noise to be removed includes:

[0135] Special characters: Emojis, symbols (such as ￥, %, #, @), full-width / half-width special symbols (such as _, —, ~);

[0136] Redundant spaces: 2 or more consecutive spaces (including full-width space “ ”), paragraph head / tail spaces;

[0137] Invisible characters: line feed, tab (\t), carriage return (\r);

[0138] Encoding abnormal characters: garbled characters due to encoding incompatibility.

[0139] The chardet library (version 3.0.4, installed through pip install chardet==3.0.4) is used to detect the text encoding;

[0140] Execution process: traverse the text field of each paragraph in the structured text data output by S1, call chardet.detect(text.encode()) to obtain the encoding type (such as “utf-8”, “gbk”, “gb2312”); if the encoding is not “utf-8”, call text.encode(encoding='utf-8',errors='replace').decode('utf-8') to convert to UTF-8 encoding, where errors='replace' ensures that the encoding error characters are replaced with a preset character and then uniformly processed.

[0141] Regular expression: the pattern for matching non-Chinese and English characters and numbers is r'[^\u4e00-\u9fa5a-zA-Z0-9]';

[0142] Execution steps:

[0143] ① Call re.sub(pattern=r'[^\u4e00-\u9fa5a-zA-Z0-9]', repl=' ', text) to replace all non-Chinese and English characters and non-digits with half-width spaces;

[0144] ②Call re.sub(pattern=r'\s+', repl=' ', text) to merge more than one space (including half-width space converted from full-width space) into a single space.

[0145] ③Call text.strip() to remove spaces at the beginning and end of the paragraph. If the length of the processed text is less than 1 (only containing spaces or an empty string), mark it as "invalid text" and remove it.

[0146] Preserve key symbols: If the text contains technical parameter symbols (such as "≥" "≤" "%"), you can preserve them by extending the regular expression r'[^\u4e00-\u9fa5a-zA-Z0-9≥≤%]' and enabling the "technical symbol preservation" switch in the system configuration file config.ini.

[0147] Long text segmentation processing: For paragraphs longer than 1000 characters, segment them by punctuation (., ;) and remove noise segment by segment to avoid high memory usage in single processing (threshold can be configured by max_text_length=1000).

[0148] Use jieba segmentation library (version 0.42.1, installed through pip install jieba==0.42.1), which supports custom dictionary and HMM model, suitable for Chinese technical document segmentation scenarios.

[0149] Dictionary format: Use "word frequency word property" three column format (for example "DOCX document 20 n" "inverted index 15n" "semantic expansion 12 v"), where "word frequency" is based on 5000 documents retrieval field corpus statistics, "word property" refers to "Chinese information processing segmentation specification";

[0150] Loading method: Call jieba.load_userdict('domain_dict.txt') to load the domain custom dictionary (path is. / dict / domain_dict.txt), which has higher priority than the default jieba dictionary, ensuring that technical terms are not segmented (for example, "document parsing module" is not segmented into "document / parse / module").

[0151] Use "exact mode + HMM auxiliary", call jieba.cut(text, cut_all=False, HMM=True), where:

[0152] cut_all=False: Turn off full mode to avoid over-segmentation (for example, "search system" is not segmented into "search / system / system");

[0153] HMM=True: Enable HMM model, handle out-of-vocabulary words (e.g. "Sentence-BERT", "BM25 model" and other foreign technical terms).

[0154] Generate initial segmentation candidate set based on dictionary matching, calculate the probability of each candidate word sequence through HMM model, and select the sequence with the highest probability as the optimal path;

[0155] HMM parameters: state set is {B (beginning of word), M (middle of word), E (end of word), S (single word)}, and the observation set is Chinese characters;

[0156] Example: For the text "key word-based DOCX retrieval system", the initial candidate set contains "based / keyword / of / DOCX / retrieval / system" "based / keyword / of / DOCX / retrieval / system" and other words, and the HMM calculates the highest probability "based / keyword / of / DOCX / retrieval / system" as the final word sequence.

[0157] English term reservation: If the segmentation result contains consecutive English characters (e.g. "DOCX", "BERT"), call re.match(r'[a-zA-Z0-9]+', word) to detect and determine as an English term, and reserve the original word without splitting;

[0158] Number and unit combination: For "200ms" "92%" and other number+unit combinations, match through regular expression r'[0-9]+[a-zA-Z%≤≥]+', and merge into a single word (e.g. "200ms" is not split into "200 / ms").

[0159] Basic stop list: derived from open source standard stop list (Harbin Institute of Technology stop list, Baidu stop list, Sichuan University stop list), containing 1208 general meaningless words, such as "of", "of", "in", "and", "is", "I", "you";

[0160] Domain stop list: customized for document retrieval field, containing 32 high-frequency meaningless words, such as "document", "file", "module", "system", "method", "step" (these words appear frequently in retrieval field documents, but have no actual retrieval distinction);

[0161] Dynamic stop list: based on user search log real-time update, if the search click rate (click times / search times) of a word is less than 5% for 7 consecutive days, it is automatically added to the dynamic stop list, updated once a week, initially empty.

[0162] Traverse all document segmentation results, count through collections.Counter() ;

[0163] A "word-document ID" mapping table is constructed, and the number of document IDs in which each word appears is counted to obtain ;

[0164] For each word w, the formula is calculated ; if (i.e. ), or w exists in the three-level stop word list, it is determined as "meaningless word" and removed; otherwise, it is retained to the purified word sequence;

[0165] Result storage: the purified word sequence is associated with the corresponding paragraph ID and document ID, and stored in JSON format (for example, {"doc_id":"doc_001","para_id":"P1","word_seq":["based on","key words","DOCX","retrieval system"]}).

[0166] Next, step S3 includes:

[0167] S31, position coding is performed on the purified word sequence, a unique paragraph number and paragraph internal offset are assigned to each word, and a word position mapping table is established; the position coding uses a segmented linear mapping function:

[0168] ;

[0169] Wherein, represents the position coordinates of the word ; p is the paragraph number; is the paragraph internal offset; is the average paragraph length; is the sequential index of the word in the full text;

[0170] S32, based on the word position mapping table, an inverted index is constructed, and a hash mapping and linked list structure are used to store the mapping relationship of keywords to occurrence positions; the definition formula of the inverted index item is:

[0171] ;

[0172] Wherein, represents the inverted index item of the keyword t; is the paragraph number; is the paragraph internal offset; is the frequency of the keyword in the position , m is the total number of paragraphs containing the keyword, and n is the number of keyword occurrence positions in a single paragraph;

[0173] S33, the inverted index is compressed and optimized, and a multi-level cache mechanism is established to improve the retrieval efficiency; the index compression rate calculation formula is:

[0174] ;

[0175] in, Indicates the index compression ratio; This is the size of the compressed index; The original index size is used; the cache hit rate optimization formula is:

[0176] ;

[0177] in, For cache hit rate; This represents the number of times the cache was hit. This represents the total number of visits.

[0178] It uses JSON Lines (.jsonl) format for storage, with each line corresponding to the position information of a word (e.g., {"doc_id":"doc_20251022_001","word":"keyword","p":1,"o_i":2,"i":2,"L":2}), supports reading and writing line by line to avoid excessive memory consumption;

[0179] Exception handling: If the number of words in a paragraph is 0 (invalid text removed by S2), skip the paragraph numbering assignment and ensure that p increases continuously (for example, if P2 is an invalid paragraph, P1 corresponds to p=1, and P3 corresponds to p=2).

[0180] Hash mapping: Python's built-in dict type is used (average search time complexity O(1)), the key is the keyword t (string type, case sensitive, e.g. "DOCX" ≠ "docx"), and the value is a linked list (simulated using Pythonlist, which supports dynamic node appending).

[0181] Linked list nodes: Each node is a tuple type with the format (p_j, list_o, f_jk) (e.g., (1, [3],1) indicates that the keyword is in paragraph 1, offset 3, and frequency 1).

[0182] Initialize index: Create an empty hash map: `inverted_index={}`;

[0183] Traverse the position mapping table: Read the .jsonl file generated by S31 line by line, and extract the word (t) and p (p) of each word. ), o_i (i.e. );

[0184] Update index logic: If t is not in inverted_index: create a new key-value pair, and initialize the linked list to [(p_j,[o_i], 1)] (first occurrence, frequency 1);

[0185] If t is already in inverted_index: Traverse its linked list nodes and check if it exists. Consistent nodes:

[0186] If it exists: will The f_jk is appended to the list_o of this node and updated to len(list_o);

[0187] If it does not exist: append a new node (p_j, [o_i], 1) to the linked list;

[0188] Index verification: After the index is built, randomly select 10% of the keywords and check whether the p_j and list_o in their index entries are consistent with the position mapping table to ensure that there are no omissions or errors.

[0189] Duplicate words: If the same word appears consecutively in the same paragraph (e.g., "search search"), it is recorded according to the actual offset, and f_jk is accumulated normally;

[0190] Mixed Chinese and English vocabulary: For example, “DOCX search”, the S2 segmentation results are used as independent keywords t, and its index terms are logically consistent with Chinese vocabulary.

[0191] Delta encoding is used for paragraph numbering in inverted indexes. The "orderly increasing" characteristic can significantly reduce the number of bytes to be stored.

[0192] Regarding paragraph numbering For each keyword t in the linked list, store the first paragraph number. Subsequent paragraph numbers are stored as "current" -Previous (difference) For example: original Sequence [1,2,5] → Compressed and stored as [1, 1(2-1), 3(5-2)].

[0193] For offset If the offsets within a paragraph are ordered (e.g., [3,6,8]), similarly store the first offset. Subsequent storage difference (e.g., [3,3 (6-3),2 (8-6)]);

[0194] frequency No compression required, store directly (because) Compression yields no significant benefits.

[0195] Hit rate optimization strategy:

[0196] L1 cache update: update the "last access time" of each keyword t every time it is retrieved, if it hits L1; if it misses L1 but hits L2, move t to L1 (if L1 is full, remove the keyword with the earliest "last access time");

[0197] L2 cache update: if t misses L1 and L2, load it from the disk index and store it in L2 (if L2 is full, remove the keyword with the earliest "last access time");

[0198] High-frequency word statistics: update the keyword list of L1 and L2 every morning by counting the retrieval logs of the previous day (ensure that high-frequency words are stored in L1 first).

[0199] By using a piecewise linear mapping function to convert "paragraph number + offset" into a unique position coordinate, the defect of the prior art that "only recording paragraph ID cannot locate specific words within the paragraph" is avoided. For example, when a user searches for "DOCX", the position coordinate can be used to directly locate the 3rd word in P1, without the need to traverse the entire paragraph, and the positioning accuracy is improved from "paragraph level" to "word level".

[0200] In addition, step S4 includes:

[0201] S41, receiving the keyword string input by the user, standardizing and preprocessing the keyword, including converting the keyword to a unified lowercase form, removing preset characters and unnecessary spaces, and using a Chinese word segmentation tool to segment the composite keyword into single word units to form a standardized keyword sequence;

[0202] S42, using a pre-trained natural language processing model to perform semantic analysis on the standardized keyword sequence, and using a multi-level semantic expansion strategy to obtain the synonym and near-synonym set of each keyword, the multi-level semantic expansion strategy including the following calculation process:

[0203] First-level expansion: use the Sentence-BERT sentence encoding model to calculate the semantic similarity between the keyword and the candidate word, and the semantic similarity calculation formula is:

[0204] ;

[0205] Where A and B are the vector representations of the keyword and the candidate word, respectively; is the cosine similarity value; when the candidate word is added to the expansion set; is a preset similarity threshold;

[0206] Second-level expansion: obtain the explicit synonyms of the keyword by combining the HowNet synonym knowledge base, and verify the validity of the synonyms;

[0207] Three-level expansion: using the vector space obtained by training the Word2Vec word vector model on the domain corpus, calculate the cosine similarity between the key words and the candidate words, select the top K words with the highest similarity as the synonyms, K is the preset expansion number;

[0208] S43, the synonym and synonym set obtained by multi-level semantic expansion is de-duplicated and merged to form an expanded keyword list, and each expanded keyword is assigned a weight, and the weight calculation formula is:

[0209] ;

[0210] Among them, The weight of the expanded keyword e is represented; The semantic similarity between the expanded word e and the original keyword k; The expansion level coefficient; And The weight coefficient.

[0211] Call keyword_str.lower() to convert the user input keyword string (e.g. "DOCX document retrieval" "retrieval system") to all lowercase (e.g. "docx document retrieval" "retrieval system") to avoid missing matches due to case differences (e.g. "DOCX" and "docx").

[0212] Pre-set character removal:

[0213] Define "preset characters" as: non-Chinese and English characters, meaningless separators (e.g. "_" "-" "#" "@"), and special symbols (e.g. "!" "?" ";" );

[0214] Use the regular expression r'[^\u4e00-\u9fa5a-zA-Z0-9]' to match the preset characters, and call re.sub(pattern, repl='', keyword_str) to remove them;

[0215] Example: input "DOCX-Document Retrieval!" → processed as "docx document retrieval".

[0216] Excess space cleaning: call re.sub(r'\s+', repl=' ',keyword_str).strip() to merge consecutive spaces (including full-width spaces) into a single space and remove leading and trailing spaces; If the length of the processed string is <1 (contains only spaces), prompt the user "Please enter a valid keyword" and terminate the process.

[0217] Compound keyword segmentation and disassembly:

[0218] jieba.cut(keyword_str, cut_all=False, HMM=True), combined with the document retrieval domain dictionary loaded in step S2 (containing terms such as "docx document" and "retrieval system"), the composite keyword is split into single tokens;

[0219] Rule: If the length of the split token is <2 (e.g. "of" "in"), and not in the domain core term table, remove the token; finally form the "standardized keyword sequence" (list form);

[0220] Example: Composite keyword "docx document retrieval system" → split into ["docx document", "retrieval system"] (remove meaningless tokens and keep domain terms).

[0221] Exception handling:

[0222] Empty input processing: If the user does not input the keyword (string length is 0), trigger the front-end prompt "keyword cannot be empty" and re-receive input;

[0223] Single character input processing: If the input is a single character (e.g. "check" "find"), automatically supplement the common collocations in the domain (e.g. "check" "retrieve"), form a candidate keyword sequence for the user to choose from, to avoid poor expansion due to short input.

[0224] First-level expansion: Sentence-BERT semantic similarity expansion.

[0225] Select the paraphrase-multilingual-MiniLM-L12-v2 model in the sentence-transformers library (version 2.2.2, installed through pip install sentence-transformers==2.2.2), which supports Chinese semantic encoding, has small parameter size (about 120MB) and fast inference speed (single sentence encoding <10ms);

[0226] Candidate word source: Based on the "document retrieval domain corpus" (containing 5000 DOCX documents with purified word sequences, total vocabulary 100,000), a candidate word pool is constructed to ensure that the candidate words are strongly related to the domain.

[0227] Second-level expansion: HowNet synonym knowledge base expansion.

[0228] Use HowNet 2024 official version (contains 1.5 million word semantic relationships, covering office documents and retrieval domain terms);

[0229] Access method: locally deploy HowNet dictionary file (path. / knowledge / HowNet_2024.dat), call dictionary query interface through pynetdicom library (version 2.0.2), avoid network dependence.

[0230] Synonym query: for each word in the standardized keyword sequence, call HowNet.query_synonyms(word) to get explicit synonyms (e.g. "retrieve" → "find" "search", "DOCX document" → "DOCX file");

[0231] Effectiveness verification:

[0232] Verification rule: count the frequency of synonyms in the "document retrieval field corpus", if the frequency ≥ 5 times (indicating that the word is commonly used in the field), it is determined as "effective synonym"; otherwise, it is excluded (e.g. the synonym "search" of "retrieve" appears only 2 times in the corpus, which is excluded);

[0233] Example: the synonym "DOCX file" of the keyword "DOCX document" appears 32 times in the corpus → judged effective, added to the "second-level expansion set".

[0234] Three-level expansion: Word2Vec field word vector expansion.

[0235] Near-synonym screening logic:

[0236] Default expansion quantity K: value is 5 (tested, K=5 can balance between "expansion breadth" and "precision", avoid excessive expansion leading to redundancy);

[0237] Execution steps:

[0238] Call model.wv.most_similar(word, topn=K) to get the top K near-synonyms of the keyword word and the corresponding cosine similarity;

[0239] If the near-synonym already exists in the first / second-level expansion set, only update the highest similarity value; otherwise, add it to the "third-level expansion set".

[0240] De-duplication and merging logic:

[0241] De-duplication rule: use "word string complete match" for de-duplication (e.g. "docx file" and "DOCX file" are considered the same word, as S41 has unified lowercase, so directly compare the string);

[0242] Merge strategy: merge the primary, secondary, and tertiary expansion sets into an "initial expansion set", keeping the highest semantic similarity value for each vocabulary (e.g., a vocabulary exists in both the primary and secondary sets, take the higher similarity value between the two).

[0243] Final expansion keyword list output:

[0244] Output format: stored in a list, each element is a tuple (expanded keyword, weight value, original keyword), sorted in descending order of weight value (convenient for subsequent matching to prefer high-weight vocabulary);

[0245] Example: the expansion list of the original keyword "docx document" is [("docx file", 0.895, "docx document"), ("word document", 0.836, "docx document"), ("office document", 0.752, "docx document")].

[0246] Then, step S5 includes:

[0247] S51, parse the expansion keyword list, construct a query expression tree according to the user input logical operators, which include AND and OR, the leaf nodes of the query expression tree represent single keywords, the internal nodes represent logical operations, and ensure the correct operator precedence;

[0248] S52, for each keyword node in the query expression tree, perform multi-pattern matching in the inverted index, multi-patterns include exact match, fuzzy match and synonym match; fuzzy match uses edit distance similarity calculation:

[0249] ;

[0250] Where, represents the fuzzy match similarity of vocabulary and ; represents the edit distance; represents the character length of vocabulary ; when , it is considered to match; is the preset fuzzy match threshold;

[0251] Synonym matching uses semantic similarity calculation:

[0252] ;

[0253] Where, represents the semantic similarity of vocabulary and ; represents the semantic similarity of vocabulary word vector representation; when is considered to match; is a preset synonym matching threshold;

[0254] S53, merge the matching results according to the logical operations of the query expression tree, take the intersection of paragraph sets for AND operations, take the union for OR operations, and calculate the matching strength score of each paragraph:

[0255] ;

[0256] wherein, represents the matching strength score of paragraph p; r is the total number of extended keywords; is the weight of the extended keyword ; is an indicator function, taking the value of 1 when paragraph p contains the keyword , otherwise 0; finally, the paragraphs with matching strength scores higher than the corresponding preset threshold form the initial matching paragraph set.

[0257] Logical operator rules:

[0258] Supported operators: only include AND (logical AND, high priority), OR (logical OR, low priority), and operators must be in uppercase (to avoid confusion with keywords, such as "and" as a keyword);

[0259] Priority rules: AND has higher priority than OR (for example, "a AND b OR c" is equivalent to "(a AND b) OR c");

[0260] Input format: the user needs to input in the format "keyword 1 operator keyword 2 [operator keyword 3...] ", for example, "DOCX document AND search OR find"; if no operator is entered, it is processed by default as OR (for example, "DOCX document search" is equivalent to "DOCX document OR search").

[0261] Use Python's pyparsing library (version 3.1.1, installed through pip install pyparsing==3.1.1) to support custom grammar rules and avoid the complexity of manual parsing;

[0262] Expression tree structure:

[0263] Leaf node: data structure is LeafNode(keyword, weight), where keyword is an extended keyword (e.g. "docx file"), and weight is the weight assigned by step S4 (e.g. 0.94);

[0264] Internal node: The data structure is InternalNode(op, left, right), where op is a logical operator (AND / OR), and left / right are child nodes (which can be leaf nodes or other internal nodes).

[0265] Example: For "DOCX document AND retrieval OR search", the expression tree structure is:

[0266] InternalNode(op="OR",left=InternalNode(op="AND",left=LeafNode("docx document",0.895),right=LeafNode("retrieval",0.82)), right=LeafNode ("search",0.94)).

[0267] Load the list of (extended keyword, weight, original keyword) output by step S4, and construct a "keyword-weight" mapping dictionary (e.g., {"docx file":0.94,"retrieval":0.82});

[0268] User input parsing:

[0269] Call pyparsing.Word(pyparsing.alphanums+"Chinese characters") to define the keyword grammar, and pyparsing.CaselessLiteral("AND")|pyparsing.CaselessLiteral("OR") to define the operator grammar;

[0270] Construct the grammar rule: query = pyparsing.OneOrMore(LeafNode_expr +(InternalNode_expr + LeafNode_expr)[...]), where LeafNode_expr matches the extended keyword, and InternalNode_expr matches the operator;

[0271] Exception handling:

[0272] If the user enters a word not in the extended keyword list (e.g., "invalid keyword"), prompt "The keyword is not in the extended list. Please re-enter.";

[0273] If an invalid operator is entered (e.g., "NOT", "XOR"), prompt "Only AND / OR operators are supported. Please correct.";

[0274] If the expression syntax is incorrect (for example, "DOCX document AND OR search"), the message "Logical expression syntax error. Please check the operator position" is prompted.

[0275] Exact match (priority):

[0276] Core logic: directly compare whether the extended keyword and the keyword in the inverted index are exactly the same (no need for case adaptation because step S4 has been standardized to lowercase);

[0277] Execution steps:

[0278] For the keyword e of the leaf node, check whether the key e exists in the inverted index inverted_index;

[0279] If it exists, extract all paragraph number sets P_exact = {p_j | (p_j,[...],...) ∈ I(e)} corresponding to the keyword;

[0280] If it does not exist, the exact match result is an empty set, and the subsequent synonym matching is entered.

[0281] Use the python-Levenshtein library (version 0.21.1, installed through pip install python-Levenshtein==0.21.1) to call Levenshtein.distance(w1, w2) to obtain the edit distance;

[0282] Execution steps:

[0283] Traverse all keywords w_idx of the inverted index;

[0284] Calculate Sim_fuzzy(e, w_idx), and if Sim_fuzzy≥θ_f, extract the paragraph number set P_fuzzy corresponding to w_idx;

[0285] De-duplication: if P_fuzzy and P_exact have overlapping paragraphs, only keep one.

[0286] Word vector acquisition: call the Sentence-BERT model loaded in step S to encode the extended keyword e and the inverted index keyword w_idx, and obtain v_e and v_idx;

[0287] Similarity calculation: calculate Sim_syn(e, w_idx) according to the formula, and if Sim_sy≥θ_s, extract the paragraph number set P_syn corresponding to w_idx;

[0288] Matching order: exact match first, synonym match second, fuzzy match last, to avoid repeated processing (already matched paragraphs will not participate in subsequent pattern matching);

[0289] Result merging: the final matched paragraph set of a single extended keyword is P_e = P_exact ∪ P_syn ∪ P_fuzzy.

[0290] Result merging of logical operations (intersection / union):

[0291] Data structure: paragraph set uses Python's set type (supports efficient intersection and union operations);

[0292] Merging rules:

[0293] If the internal node is AND: merge the left child node paragraph set P_left and the right child node paragraph set P_right, take the intersection P_merge = P_left ∩ P_right (only keep paragraphs that contain both keywords);

[0294] If the internal node is OR: take the union P_merge = P_left ∪ P_right (keep paragraphs that contain any keyword);

[0295] Recursive merging: recursively perform merging from the leaf node to the root node of the expression tree, and finally get the total paragraph set P_total of the root node (all possible matched paragraphs).

[0296] Initial matched paragraph set screening

[0297] Score calculation: traverse each paragraph p in P_total, for all extended keywords , judge whether p is in , substitute into the formula to calculate ;

[0298] Screening rules: if , add p to the "initial matched paragraph set"; if , remove the paragraph (considered as insufficient correlation);

[0299] Result storage: the initial matched paragraph set is stored in the format of "paragraph number-matching strength score-matched keyword list" (for example {"p1": {"score":1.78, "matched_keywords":["docx file", "search"]}}), which is convenient for relevance sorting in subsequent step S6.

[0300] Finally, step S6 includes:

[0301] S61, for each paragraph in the initial matching paragraph set, calculate an original relevance score based on keyword matching, which is calculated in combination with the term frequency of the keyword in the paragraph and the inverse document frequency, wherein the term frequency represents the number of occurrences of the keyword in the paragraph, and the inverse document frequency represents the rarity of the keyword in the entire document set; the original relevance score is calculated using the BM25 model, and the formula is:

[0302] ;

[0303] wherein, represents the original relevance score of the paragraph p; represents the i-th extended keyword, and r is the total number of extended keywords; represents the inverse document frequency of the keyword , and the calculation formula is , wherein M is the total number of documents in the document set; is the number of documents containing the keyword ; represents the term frequency of the keyword in the paragraph p; represents the length of the paragraph p; represents the average length of all paragraphs in the document set; and b are adjustable parameters, wherein controls the saturation of the term frequency, and is usually valued at 1.2 to 2.0, and b controls the influence of paragraph length normalization, and is usually valued at 0.5 to 0.8;

[0304] S62, the original relevance score is subjected to paragraph length normalization processing to eliminate the bias of long paragraphs due to high term frequency, and to ensure the comparability of scores of paragraphs of different lengths; the normalization processing uses a length weight adjustment formula:

[0305] ;

[0306] wherein, represents the normalized relevance score; is a normalization intensity coefficient; represents an exponential function for smoothing the influence of length difference;

[0307] S63, the normalized score is adjusted in combination with the structural position weight of the paragraph, wherein the weight of the title paragraph is higher than that of the body paragraph, and a final relevance score is calculated, and the ranking result is output in descending order of score; the final relevance score calculation formula is:

[0308] ;

[0309] wherein, represents the final relevance score of the paragraph p; is a function that takes value 1 when paragraph p is classified as a title, otherwise 0; is the title weight bonus coefficient; when ranking, paragraphs are sorted by in descending order, generating the final search result list.

[0310] IDF calculation: first, count the total number of documents M (e.g. M=100 if the user uploads 100 documents) and the number of documents containing by traversing the S3 inverted index; call the Python math.log function (natural logarithm) to calculate; if (the keyword does not appear in any document), take 0.5 to avoid the meaninglessness of logarithm; Term frequency

[0311] : get the field of paragraph p corresponding to the keyword from the S3 inverted index (e.g. = "docx file" in p=1);

[0312] Paragraph length ∣p∣: extract from the "paragraph ID-purified word sequence" mapping output by S2 (e.g. the length of the purified word sequence of p=1 is 4);

[0313] Average length avgdl: traverse all paragraphs of all documents, accumulate ∣p∣ and divide by the total number of paragraphs (e.g. 100 documents have a total of 1200 paragraphs, with a total length of 6360, then avgdl=6360 / 1200=5.3).

[0314] For each initial matching paragraph p, substitute the formula to calculate ;

[0315] Descending order ranking: sort by from high to low, if equal, then sort by paragraph number in ascending order (prefer to show the earlier paragraphs in the document);

[0316] Result output: generate a structured list of "paragraph ID-final score-matched keywords-paragraph type" (e.g. {"para_id":"p1","final_score":4.29,"matched_keywords":["docx file","search"],"type":"h1"}), as input to S7.

[0317] Highlight execution logic:

[0318] ​Load the "Paragraph ID - Final Score - Matched Keywords" list output from S6 (e.g., p1's matched keywords are ["docx file", "retrieve", "search"]), and the "Paragraph ID - Original Text" mapping stored from S1 (e.g., p1's original text is "keyword-based DOCX retrieval system");

[0319] Keyword positioning: For the paragraph's original text, call re.findall(keyword, text, flags=re.IGNORECASE) (ignore case) to locate the start / end positions of each matched keyword in the text;

[0320] Style embedding:

[0321] If it's HTML format: replace the matched keyword with <mark style="background-color:#ffeb3b;color:#000;">Keywords< / mark> ;

[0322] If it's TXT format: replace it with "

keyword

[0323] If it's PDF / Word format: set the background color through run.font.highlight_color = docx.enum.text.WD_COLOR_INDEX.YELLOW (python-docx);

[0324] Exception handling: If no matched keyword is found in the paragraph's original text (due to S5's multi-pattern matching based on synonyms / fuzzy words), extract the keyword's offset from the S3 inverted index, locate the text position, and then perform highlighting (e.g., "search" matches "retrieve", so highlight "retrieve").

[0325] Matched sentence recognition and context expansion:

[0326] 1. Matched sentence definition and recognition tool.

[0327] Matched sentence: refers to a complete sentence in the paragraph's original text that "contains at least one expanded keyword (or its synonym / fuzzy matching word)";

[0328] Sentence segmentation tool: Use the snownlp library (version 0.12.3, installed through pip install snownlp==0.12.3), call SnowNLP(text).sentences to segment the paragraph text into independent sentences (supports Chinese punctuation sentence segmentation, such as "。、;!?").

[0329] 2. Context expansion rules (based on the maximum number of sentences configured by the user).

[0330] User configuration parameter: max_sentence_num (maximum number of summary sentences, ranging from 1 to 10 sentences, with a default of 3 sentences, set via the system interface or configuration file);

[0331] Extended logic:

[0332] For the sentence list sent_list after paragraph segmentation, mark all matching sentences (denoted as match_sent_idx, for example, if the second sentence is a matching sentence, then match_sent_idx=[1], and the index starts from 0);

[0333] Expand the sentences forward and backward, centering on each matching sentence:

[0334] If len(sent_list) ≤ max_sentence_num: extract all sentences as a summary;

[0335] If len(sent_list) > max_sentence_num: expand by "take 1 sentence before + matching sentence + take 1 sentence after" (default 3 sentences). If the matching sentence is at the beginning / end of the sentence, expand backward / forward first (for example, if the matching sentence is the first sentence, then take "the first + second + third sentences").

[0336] Deduplication: If multiple matching sentences contain duplicate sentences after expansion, only one duplicate is retained.

[0337] Dynamic summary generation and output:

[0338] Abstract format: Output in a structured format as "Paragraph ID - Highlighted Paragraph Text - Abstract Text - Final Score", where the abstract text must be marked with "Matching Sentences" (e.g., using "... (prefix)

[0339] Example: Summary of p1 (max_sentence_num=3):

[0340] The highlighted text reads: "A DOCX retrieval system based on keywords."

[0341] Abstract text: A DOCX retrieval system based on keywords (the paragraph contains only one sentence, so it was extracted directly).

[0342] Export in HTML format (supports web browsing, retains highlighting styles):

[0343] Template definition: Create a Jinja2 template file result_template.html and define a fixed structure:

[0344] html

[0345] Preview

[0346] <!DOCTYPE html>

[0347]

[0348]

[0349] <meta charset="UTF-8">

[0350] <title>DOCX search results< / title>

[0351] <style>mark {background-color: #ffeb3b; color: #000;}< / style>

[0352]

[0353]

[0354] <h1>Search results summary (total of {{total_para}} paragraphs)< / h1>

[0355] {% for para in result_list %}

[0356]

[0357] <h3>Paragraph {{para.para_id}} (score: {{para.final_score|round(3)}})< / h3>

[0358] <strong>Highlighted paragraph:< / strong> {{para.highlighted_text}}

[0359] <strong>Dynamic summary:< / strong> {{para.summary}}

[0360]

[0361] {% endfor %}

[0362]

[0363]

[0364] Data rendering: The "paragraph ID-highlighted text-summary-score" list output by S7 is used as result_list, passed to the template, and jinja2.Template.render() is called to generate complete HTML content;

[0365] File saving: Call `with open("search results.html", "w", encoding="utf-8") as f:f.write(html_content)` to save to the user-specified path (the default path is `. / export / `).

[0366] Persistent storage and exception handling:

[0367] Storage path: By default, the . / export / folder is created in the user's current working directory. If the folder does not exist, it will be created automatically by calling os.makedirs(". / export / ").

[0368] File name rules: Name files in the format "Search Results_YYYYMMDD_HHMMSS" (e.g., "Search Results_20251022_153045.html") to avoid file overwriting;

[0369] Abnormality handling: If the user-specified path has no write permission (PermissionError), automatically switch to the system temporary directory (tempfile.gettempdir()) and prompt the user; if the exported file is too large (> 100MB), prompt "The file size is large, it is recommended to export in batches."

[0370] In some embodiments, the system 200 for keyword-based retrieval of DOCX document content can include a plurality of function modules composed of computer program segments. The computer programs of each program segment in the system 200 for keyword-based retrieval of DOCX document content can be stored in the memory of the computer device and executed by at least one processor to perform (see detailed description) Figure 1 ) the function of keyword-based retrieval of DOCX document content.

[0371] In this embodiment, the system 200 for keyword-based retrieval of DOCX document content can be divided into a plurality of function modules according to the functions it performs, as shown in Figure 2 The function modules can include a document parsing module 210, a text preprocessing module 220, an inverted index construction module 230, a semantic expansion module 240, a multi-mode matching module 250, a relevance calculation module 260, a result display module 270, and a result export module 280. The module referred to in the present application refers to a series of computer program segments that can be executed by at least one processor and can complete a fixed function, which are stored in the memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0372] Specifically, the document parsing module 210 is configured to load a DOCX file, traverse all paragraphs and table elements in the document, extract text content and style information of each paragraph, and classify the paragraphs into a title or a body according to at least one of the following features: a style name, a font size, and a bold attribute, to form structured text data; the text preprocessing module 220 is connected to the document parsing module and configured to preprocess the structured text data, including removing non-English characters and redundant spaces, performing word segmentation processing on the text by using a word segmentation tool, and loading a stop word table to filter meaningless words, to obtain a purified word sequence; the inverted index construction module 230 is connected to the text preprocessing module and configured to construct an inverted index based on the purified word sequence, and establish a one-to-one mapping relationship from a keyword to a position, wherein the position information at least includes a paragraph number and a paragraph internal offset; the semantic expansion module 240 is configured to receive a keyword input by a user, perform semantic analysis on the keyword by using a natural language processing model, obtain a synonym and / or near-synonym set of the original keyword, and form an expanded keyword list; the multi-mode matching module 250 is connected to the semantic expansion module and the inverted index construction module, and configured to perform multi-mode matching in the inverted index according to the expanded keyword list, wherein the matching mode at least includes one of the following: exact matching, fuzzy matching, and synonym matching, to obtain an initial matching paragraph set; the relevance calculation module 260 is connected to the multi-mode matching module, and configured to calculate a relevance score for each paragraph in the initial matching paragraph set, and generate a ranking result in descending order of the scores; the result display module 270 is connected to the relevance calculation module, and configured to highlight the keyword in the corresponding paragraph in the ranking result, and extract a matching sentence and adjacent sentences before and after the matching sentence to generate an abstract, wherein the length of the abstract is dynamically determined according to a maximum sentence number configured by the user; and the result export module 280 is connected to the result display module, and configured to export the highlighted paragraph and the abstract in an HTML, TXT or PDF format, to realize persistent storage of the retrieval result.

[0373] Figure 3 A structure schematic diagram of a terminal 300 is provided for an embodiment of the present application, and the terminal 300 can be used to execute the method for retrieving DOCX document content based on a keyword provided by the embodiment of the present application.

[0374] The terminal 300 can include a processor 310, a memory 320 and a communication module 330. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the server shown in the figure does not constitute a limitation on the present application, which can be a bus structure or a star structure, and can include more or fewer components than shown in the figure, or combine some components, or different component arrangements.

[0375] The memory 320 can be used to store the execution instructions of the processor 310, and the memory 320 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 can execute part or all of the steps in the following method embodiments.

[0376] The processor 310 is the control center of the storage terminal, and connects various parts of the entire electronic terminal through various interfaces and lines, and executes the software programs and / or modules stored in the memory 320 and the data stored in the memory, so as to execute various functions of the electronic terminal and / or process data. The processor can be composed of an integrated circuit (IC), for example, can be composed of a single packaged IC, or can be composed of a plurality of packaged ICs with the same function or different functions. For example, the processor 310 can only include a central processing unit (CPU). In the embodiments of the present application, the CPU can be a single operation core or can include multiple operation cores.

[0377] The communication module 330 is used to establish a communication channel, so that the storage terminal can communicate with other terminals. The communication module 330 receives user data sent by other terminals or sends user data to other terminals.

[0378] The present application also provides a computer storage medium, wherein the computer storage medium can store a program, and the program can include part or all of the steps in the embodiments provided by the present application when executed. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0379] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present application can be implemented by means of software plus necessary universal hardware platforms. Based on such an understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a USB flash disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, and includes a plurality of instructions for causing a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, or the like) to execute all or part of the steps of the methods described in the embodiments of the present application.

[0380] In the present specification, the same or similar parts among various embodiments can be referred to each other. In particular, for the terminal embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0381] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are merely schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, systems or modules, and can be electrical, mechanical or other forms.

[0382] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, that is, they can be located in one place, or can be distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0383] In addition, the functional modules in each of the embodiments of the present application can be integrated in one processing module, or each module can exist physically separately, or two or more modules can be integrated in one module.

[0384] Although the present application has been described in detail with reference to the preferred embodiments, it should be understood that the application is not limited to those preferred embodiments. Various modifications and equivalents can be made by those skilled in the art without departing from the spirit and scope of the application. Any and all modifications and equivalents are intended to be included within the scope of the present application.

Claims

1. A method for keyword-based search of DOCX document content, characterized by, Comprise: S1, load the DOCX file through the document parsing module, traverse all paragraphs and tables, extract the text and style information of each paragraph, and classify the paragraphs into titles or texts according to at least one of the style name, font size and bold attribute, and form structured text data; S2, text preprocessing is performed on the structured text data: remove non-English characters and extra spaces, use a word segmentation tool to segment the remaining text, and load a stop word list to filter meaningless words to obtain a purified word sequence; S3, based on the word sequence, an inverted index is constructed, and a one-to-one mapping relationship between the keyword and the position is established, and the position at least includes paragraph number and paragraph internal offset; S4, receiving the keyword input by the user, using a natural language processing model to perform semantic expansion on the keyword, obtaining a synonym and / or near synonym set of the original keyword, forming an extended keyword list; S5, according to the extended keyword list, performing multi-mode matching in the inverted index, the multi-mode at least including one of exact matching, fuzzy matching, synonym matching, obtaining an initial matching paragraph set; S6, calculating the relevance score of each paragraph in the initial matching paragraph set, and outputting the sorting result in descending order of relevance score; S7, highlighting the keyword in the sorting result in the corresponding paragraph, and extracting the matching sentence and its adjacent sentences to generate an abstract, the length of the abstract is dynamically determined by the maximum sentence number configured by the user; S8, exporting the highlighted paragraph and the abstract into HTML, TXT or PDF format, completing the persistence of the retrieval result; Step S1 includes: S11, load the DOCX file in the specified path through the document parsing module, parse the Office Open XML structure of the document using the python-docx library, generate a document object model tree, the model tree contains all paragraph elements and table elements, and ensure the integrity of the document structure; S12, traverse each paragraph element in the document object model tree, extract the text content of the paragraph, and obtain the style attribute information of the paragraph, the style attribute information at least includes style name, font size and bold attribute, wherein the style name is extracted from the name attribute of the paragraph style, the font size is extracted from the font size attribute, and the bold attribute is extracted from the font weight attribute; S13, based on the extracted style attribute information, calculate the title classification score of each paragraph, and classify the paragraph as a title or text according to the comparison result of the score and the preset threshold, wherein the calculation formula of the title classification score is: ； wherein, represents the title classification score of paragraph p; is an indicator function that takes value 1 when the style name of paragraph p belongs to the predefined set of title styles T, and 0 otherwise; represents the font size of paragraph p in pounds; represents the bold attribute of paragraph p, taking value 1 if bold, and 0 otherwise; represents the base font size, which is a pre-defined constant; , , is a weight coefficient; and paragraph p is classified as a title if, and as a body text otherwise, where θ is a pre-defined score threshold.

2. The method for keyword-based retrieval of DOCX document content according to claim 1, characterized in that, Step S2 includes: S21, denoising processing is performed on the structured text data, non-English characters and extra spaces are removed through regular expression matching and character encoding recognition, and valid text content is retained; S22, use a Chinese word segmentation tool to segment the denoised text, calculate the optimal segmentation path through dictionary matching and hidden Markov model, and divide the text into a word sequence; S23, load the stop word list to filter the word sequence, combine word frequency statistics and semantic weight calculation to remove meaningless words: ； wherein, represents a filtering weight of a word w; is a word frequency; N is a total number of words; is a document frequency including the word w; is a total number of documents; when the word w is filtered as a meaningless word, is a preset filtering threshold.

3. The method for keyword-based retrieval of DOCX document content according to claim 2, characterized in that, Step S3 includes: S31, position coding is performed on the purified word sequence, a unique paragraph number and paragraph internal offset are assigned to each word, and a word position mapping table is established; the position coding adopts a segmented linear mapping function: ； wherein, the position coordinates of the word; the position coordinates of the word; the intra-paragraph offset; the average paragraph length; the sequential index of the word in the text; S32, an inverted index is constructed based on the word position mapping table, and a hash mapping and linked list structure are used to store the mapping relationship of the keyword to the occurrence position; the definition formula of the inverted index item is: ； wherein, represents an inverted index item for the keyword t; is a paragraph number; is an intra-paragraph offset; is a frequency of occurrence of the keyword at position m is a total number of paragraphs containing the keyword, and n is a number of positions of the keyword within a single paragraph. S33, the inverted index is compressed and optimized, and a multi-level cache mechanism is established to improve the retrieval efficiency; the index compression rate calculation formula is: ； wherein, denotes the index compression rate; is the size of the compressed index; is the size of the original index; and the cache hit rate optimization formula is: ； wherein, is the cache hit rate; is the number of cache hits; is the total number of accesses.

4. The method for keyword-based retrieval of DOCX document content according to claim 3, characterized in that, Step S4 includes: S41, receiving the keyword string input by the user, standardizing and preprocessing the keyword, including converting the keyword to a unified lowercase form, removing preset characters and extra spaces, and using a Chinese word segmentation tool to segment the composite keyword into single word units to form a standardized keyword sequence; S42, the pre-trained natural language processing model is used to perform semantic analysis on the standardized keyword sequence, and a multi-level semantic expansion strategy is used to obtain the synonym and near-synonym set of each keyword, the multi-level semantic expansion strategy includes the following calculation process: First-level expansion: use the Sentence-BERT sentence encoding model to calculate the semantic similarity between the keyword and the candidate word, and the semantic similarity calculation formula is: ； wherein A and B are vector representations of the keyword and the candidate word, respectively; is a cosine similarity value; when the candidate word is added to the expanded set; is a preset similarity threshold value; Second-level expansion: combine the HowNet synonym knowledge base to obtain the explicit synonyms of the keyword, and verify the validity of the synonyms; Third-level expansion: use the Word2Vec word vector model to calculate the cosine similarity between the keyword and the candidate word in the vector space trained on the domain corpus, select the top K words with the highest similarity as the near-synonyms, and K is a preset expansion number; S43, the synonym and near-synonym set obtained by the multi-level semantic expansion is de-duplicated and merged to form an expanded keyword list, and a weight is assigned to each expanded keyword, and the weight calculation formula is: ； wherein, represents the weight of the expansion keyword e; is the semantic similarity between the expansion keyword e and the original keyword k; is the expansion level coefficient; and is the weight coefficient.

5. The method for keyword-based retrieval of DOCX document content according to claim 4, characterized in that, Step S5 includes: S51, parse the expanded keyword list, construct a query expression tree according to the logical operators input by the user, wherein the logical operators include AND and OR, the leaf nodes of the query expression tree represent single keywords, the internal nodes represent logical operations, and the operator priority is correct; S52, for each keyword node in the query expression tree, perform multi-pattern matching in the inverted index, the multi-pattern includes exact matching, fuzzy matching and synonym matching; the fuzzy matching uses edit distance similarity calculation: ； wherein, a fuzzy match similarity degree of the words and ; denotes an edit distance; a character length of the words ; when is considered as a match; is a preset fuzzy match threshold value; The synonym matching uses semantic similarity calculation: ； wherein, representing the semantic similarity of the vocabulary and ; representing the word vector representation of the vocabulary ; when is considered to be a match; is a preset synonym matching threshold value; S53, combine the matching results according to the logical operations of the query expression tree, take the intersection of the paragraph set for the AND operation, take the union set for the OR operation, and calculate the matching strength score of each paragraph: ； wherein, represents the matching strength score of paragraph p; r is the total number of expanded keywords; is the weight of expanded keyword ; is an indicator function, which takes the value 1 when paragraph p contains keyword , and 0 otherwise; finally, the initial matching paragraph set is selected as the set of paragraphs whose matching strength scores are higher than a corresponding preset threshold .

6. The method for keyword-based retrieval of DOCX document content according to claim 5, characterized in that, Step S6 includes: S61, for each paragraph in the initial matching paragraph set, calculate the original relevance score based on keyword matching, which is calculated in combination with the term frequency and inverse document frequency of the keyword in the paragraph, wherein the term frequency represents the number of occurrences of the keyword in the paragraph, and the inverse document frequency represents the rarity of the keyword in the entire document set; the original relevance score is calculated using the BM25 model, and the formula is: ； wherein, represents the original relevance score of passage p; represents the ith extended keyword, r is the total number of extended keywords; represents the inverse document frequency of keyword , the calculation formula is , wherein M is the total number of documents in the document set; is the number of documents containing keyword ; represents the term frequency of keyword in passage p; represents the length of passage p; represents the average length of all passages in the document set; and b are adjustable parameters, wherein controls the saturation of the term frequency, the value is 1.2 to 2.0, b controls the influence of passage length normalization, the value is 0.5 to 0.8; S62, paragraph length normalization is performed on the original correlation score to eliminate the bias caused by high word frequency of long paragraphs, and to ensure the comparability of scores of paragraphs of different lengths; the normalization processing adopts a length weight adjustment formula: ； wherein, denotes the normalized correlation score; is the normalized intensity coefficient; denotes an exponential function to smooth the impact of length differences; S63, the normalized score is adjusted in combination with the structural position weight of the paragraph, wherein the weight of the title paragraph is higher than that of the text paragraph, and the final correlation score is calculated, and the sorting result is output in descending order of score; the final correlation score calculation formula is: ； wherein, represents the final relevance score of paragraph p; is an indicator function that takes the value 1 when paragraph p is classified as a title, and 0 otherwise; is a title weight bonus coefficient; when ranking, paragraphs are ordered by in descending order of value, generating the final search result list.

7. A system for keyword-based search of DOCX document content, characterized by, Including: The document parsing module is configured to load a DOCX file, traverse all paragraphs and table elements in the document, extract text content and style information of each paragraph, and classify the paragraphs into titles or texts according to at least one of the following characteristics: style name, font size, and bold attribute, to form structured text data; The text preprocessing module is connected with the document parsing module and is configured to preprocess the structured text data, including removing non-English characters and redundant spaces, performing word segmentation processing on the text using a word segmentation tool, and loading a stop word table to filter meaningless words, to obtain a purified word sequence; The inverted index construction module is connected with the text preprocessing module and is configured to construct an inverted index based on the purified word sequence, and establish a one-to-one mapping relationship from a keyword to a position, wherein the position information at least includes a paragraph number and a paragraph internal offset; The semantic expansion module is configured to receive a keyword input by a user, perform semantic analysis on the keyword using a natural language processing model, obtain a synonym and / or near-synonym set of the original keyword, and form an expanded keyword list; The multi-mode matching module is connected with the semantic expansion module and the inverted index construction module, and is configured to perform multi-mode matching in the inverted index according to the expanded keyword list, wherein the matching mode at least includes one of exact matching, fuzzy matching and synonym matching, to obtain an initial matching paragraph set; The correlation calculation module is connected with the multi-mode matching module and is configured to calculate a correlation score for each paragraph in the initial matching paragraph set, and generate a sorting result in descending order of score; The result display module is connected with the correlation calculation module and is configured to highlight the keyword in the corresponding paragraph in the sorting result, and extract a matching sentence and its adjacent sentences to generate an abstract, wherein the length of the abstract is dynamically determined by a maximum sentence number configured by a user; The result export module is connected with the result display module and is configured to export the highlighted paragraph and the abstract in HTML, TXT or PDF format, to realize persistent storage of the retrieval result; The execution method of the document parsing module includes: S11, load the DOCX file in the specified path through the document parsing module, parse the Office Open XML structure of the document using the python-docx library, generate a document object model tree, the model tree contains all paragraph elements and table elements, and ensure the integrity of the document structure; S12, traversing each paragraph element in the document object model tree, extracting the text content of the paragraph, and obtaining style attribute information of the paragraph, the style attribute information at least including a style name, a font size, and a bold attribute, wherein the style name is extracted from a name attribute of a paragraph style, the font size is extracted from a font size attribute, and the bold attribute is extracted from a font weight attribute; S13, based on the extracted style attribute information, calculating a title classification score of each paragraph, and classifying the paragraph as a title or a body according to a comparison result of the score and a preset threshold, wherein a calculation formula of the title classification score is: ； wherein, denotes the title classification score of paragraph p; is an indicator function that takes value 1 when the style name of paragraph p belongs to the predefined set of title styles T, and 0 otherwise; denotes the font size of paragraph p in pounds; denotes the bold attribute of paragraph p, taking value 1 if bold, and 0 otherwise; denotes the base font size, which is a pre-defined constant; , , is a weight coefficient; and paragraph p is classified as a title if, and as a body text otherwise, where θ is a pre-defined score threshold.

8. A terminal, characterized by comprising: comprising: a processor; a memory for storing execution instructions of the processor; wherein the processor is configured to execute the method of any one of claims 1-6.

9. A computer readable storage medium storing a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-6. The program is executed by the processor to implement the method of any one of claims 1-6.

Images