Extraction optimization method and device for research report text, equipment and medium

Abstract

The invention relates to the technical field of data processing, and discloses a research report text extraction optimization method and device, equipment and a medium, which can be applied to the financial field, and the method comprises the following steps: obtaining research report text data, and processing to obtain a segmented text fragment set; establishing a fragment vector index database by utilizing the segmented text fragment set; receiving a query request based on the fragment vector index database and processing the query request to obtain candidate fragment sequence data; and processing the candidate fragment sequence data to obtain context data, injecting a cue word template to generate a target cue word, inputting the target cue word into a language model for reasoning, and outputting a text optimization result. In the invention, aiming at the problem that the output of the existing research and report text is lack of standardization, the target cue word can be generated by utilizing the context data obtained by calculation, and is input into the language model for reasoning, and finally the text optimization result is output, so that a multi-level abstract and key information summary can be quickly generated, the manual reading cost is reduced, and the efficiency is improved. The working efficiency of personnel is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for extracting and optimizing research report text. Background Technology

[0002] In quantitative forex trading, market research reports are typically written by financial institutions, analysts, or professional research teams. These reports cover multiple dimensions, including macro and micro market dynamics, technical analysis, and fundamental analysis, serving as crucial information for traders to acquire and formulate strategies. With the rapid increase in the number of cross-market and multilingual research reports, these reports are often in PDF or formatted documents, characterized by complex layouts, significant structural differences, dense technical jargon, and inconsistent writing styles. This results in high costs and low efficiency for manual reading and comparison, and makes it difficult to consistently extract structured information that can be directly used for decision-making within a limited timeframe. Summary of the Invention

[0003] This invention provides a method, apparatus, device, and medium for extracting and optimizing research report texts, in order to solve the technical problem that the lack of standardization in the output of existing research report texts leads to low work efficiency for personnel.

[0004] Firstly, a method for extracting and optimizing research report text is provided, including:

[0005] Obtain the research report document and perform text extraction processing on the research report document to obtain the research report text data;

[0006] The research report text data is semantically segmented and indexed to obtain a set of segmented text fragments;

[0007] The segmented text fragment set is vectorized and indexed to obtain a fragment vector index library;

[0008] Based on the fragment vector index library, query requests are received and vector retrieval and reordering processes are performed respectively to obtain candidate fragment sequence data;

[0009] The candidate fragment sequence data are subjected to context construction and window cropping and splicing processes respectively to obtain context data;

[0010] The contextual data is injected into a preset prompt word template to generate target prompt words, and the target prompt words are then input into a language model for inference to generate text optimization results.

[0011] Secondly, a research report text extraction and optimization device is provided, including:

[0012] The document acquisition module is used to acquire research report documents and perform text extraction processing on the research report documents to obtain research report text data;

[0013] The text processing module is used to perform semantic segmentation and indexing on the research report text data to obtain a set of segmented text fragments;

[0014] The index building module is used to vectorize and encode the segmented text fragment set and build an index to obtain a fragment vector index library;

[0015] The fragment sorting module is used to receive query requests based on the fragment vector index library and perform vector retrieval and reordering processes respectively to obtain candidate fragment sequence data;

[0016] The data splicing module is used to perform context construction and window clipping splicing processing on the candidate fragment sequence data to obtain context data;

[0017] The text output module is used to inject the context data into a preset prompt word template to generate target prompt words, and input the target prompt words into a language model for inference to generate text optimization results.

[0018] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-mentioned method for extracting and optimizing research report text.

[0019] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-mentioned method for extracting and optimizing research report text.

[0020] The aforementioned method, apparatus, device, and medium for extracting and optimizing research report text can acquire research report documents and perform text extraction processing on the documents to obtain research report text data; perform semantic segmentation and indexing processing on the research report text data to obtain a set of segmented text fragments; perform vectorization encoding and indexing on the set of segmented text fragments to obtain a fragment vector index library; receive query requests based on the fragment vector index library and perform vector retrieval and reordering processing respectively to obtain candidate fragment sequence data; perform context construction and window clipping and splicing processing on the candidate fragment sequence data to obtain context data; inject the context data into a preset prompt word template to generate target prompt words, and input the target prompt words into a language model for inference to generate text optimization results. In this invention, addressing the problem of low work efficiency caused by the lack of standardization in the output of existing research report texts, target prompt words can be generated using the calculated context data, and the target prompt words can be input into a language model for inference to finally output text optimization results. This allows for the rapid generation of multi-level summaries and key information abstracts, reducing manual reading costs and improving work efficiency. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a method for extracting and optimizing research report text according to an embodiment of the present invention;

[0023] Figure 2 yes Figure 1 A detailed implementation flow diagram of step S10 Figure 1 ;

[0024] Figure 3 yes Figure 1 A detailed implementation flow diagram of step S20 Figure 2 ;

[0025] Figure 4 yes Figure 1 A detailed implementation flow diagram of step S30 Figure 3 ;

[0026] Figure 5 yes Figure 1 A detailed implementation flow diagram of step S40 Figure 4 ;

[0027] Figure 6 yes Figure 1 A detailed implementation flow diagram of step S50 Figure 5 ;

[0028] Figure 7 yes Figure 1 A detailed implementation flow diagram of step S60 Figure 6 ;

[0029] Figure 8 This is a schematic diagram of a research report text extraction and optimization device according to an embodiment of the present invention;

[0030] Figure 9 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0031] Figure 10 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Please see Figure 1 As shown, Figure 1 The flowchart of a research report text extraction and optimization method provided in this embodiment of the invention includes the following steps:

[0034] S10: Obtain the research report document and perform text extraction processing on the research report document to obtain research report text data.

[0035] The research report documents (including formatted files such as PDFs) are obtained from a preset data source. The research report documents are parsed and the text is extracted. The basic text format is unified to form research report text data for subsequent processing.

[0036] Combination Figure 2 As shown, step S10 specifically includes the following steps:

[0037] S101: Retrieve research report documents from the target data source.

[0038] The system accesses research report documents from a pre-defined target data source, which can include institutional research report repositories, object storage, enterprise cloud storage, or HTTP / HTTPS download endpoints. During acquisition, metadata associated with the document is recorded and stored, including the source institution, title, author and publication date, language identifier, page number, file type, and version number. The system also performs integrity verification and deduplication on the original files, saving the verification summary value and source path for traceability during subsequent processing.

[0039] S102: Decapsulate the research report document to obtain a decapsulated document object.

[0040] Based on the file header and MIME information of the research report document, the document type is determined, and decapsulation is performed according to the type: For PDF documents, the cross-reference table and object directory are parsed, the page content stream is decompressed, and embedded fonts, images, and attachment resources are extracted; for container files with similar layouts, container unpacking, resource decoding, and dependency resource association are completed to form a unified data carrier. The decapsulated document object includes a set of page objects, a set of resource objects, and document-level metadata, which serve as input for page-level parsing.

[0041] S103: Perform page-level parsing on the decapsulated document object to obtain page structure data.

[0042] For each decapsulated document object, page structure data is parsed and generated. This data includes at least: page size and rotation information, reading direction, text drawing instruction sequence, positions of graphics and image objects, candidate areas for tables and charts, candidate boxes for headers, footers, and page numbers, footnotes, endnotes, and their references. The page structure data is organized at the page level and provides initial constraints on candidate areas and reading order for subsequent page element segmentation and recognition.

[0043] S104: Based on the page structure data, perform page element segmentation and text recognition processing respectively to obtain element content data.

[0044] The page structure data is segmented at the region level, dividing the page into title areas, body paragraph areas, list and key point areas, table areas, chart areas, formula areas, symbol areas, header and footer areas, and footnote areas. For areas where text can be directly extracted, the text and its geometric borders are read; for areas where text is stored as an image, text recognition is performed. Table boundaries and cell grids are extracted from table areas, and cell text is recognized. For charts and images, the text content of titles and captions is extracted and its association with the graphics is indexed. Placeholders are generated for formulas and special symbols, and their positions are recorded. The obtained element content data, along with the corresponding region type, page number, coordinate boundaries, and reading order, are output together.

[0045] S105: Based on the element content data, perform character encoding unification, noise removal and text line reconstruction processing respectively to obtain text line data.

[0046] The text in the element content data is encoded and normalized at the character level. Multi-source and multi-language characters are unified into the target encoding and normalized to Unicode. Differences between font suffixes and full-width and half-width characters are eliminated, and currency symbols, punctuation, and whitespace are standardized. Noise is cleaned up based on region type and layout characteristics, and duplicate lines in headers and footers, watermarks, decorative separators, and isolated page numbers are removed. The identified text is aggregated according to the baseline and column boxes to complete the text line reconstruction. During the reconstruction process, cross-line word breaks and incorrect line breaks are repaired according to the rules of continuous punctuation, word breaks, and indentation. The structural information of list numbers, paragraph titles, and footnotes is preserved. The output is line-ordered text line data, and the mapping relationship between each line and the original page position is preserved.

[0047] S106: Perform cross-line merging processing on the text line data to obtain research report text data.

[0048] Based on line-level reconstruction, text lines are merged according to reading order and paragraph boundaries to form complete paragraphs; multi-column layouts are strung across columns according to column area and reading direction; paragraphs interrupted by pagination are spliced ​​across pages while maintaining hierarchical relationships such as title-body text, body text-footnote, body text-table, and figure caption; tables and lists retain their structural placeholders and hierarchical labels; finally, the research report text data is output, and traceable indexing from paragraph to page number, coordinates, and area type is retained for subsequent semantic segmentation and indexing processing.

[0049] S20: Perform semantic segmentation and indexing on the research report text data to obtain a set of segmented text fragments.

[0050] The research report text data is semantically segmented and labeled to ensure that each text fragment has traceable source location and hierarchical information, forming a set of segmented text fragments for subsequent encoding and retrieval.

[0051] Combination Figure 3 As shown, step S20 specifically includes the following steps:

[0052] S201: Perform character normalization on the research report text data to obtain normalized text data.

[0053] The output research report text data undergoes character-level and encoding-level standardization processing. Specifically, this includes: unifying the encoding representation of texts from multiple sources (e.g., standardizing to the same Unicode standard); eliminating differences between full-width and half-width characters; unifying capitalization; repairing hyphenation breaks and removing soft line breaks; merging synonymous punctuation and currency symbols; standardizing the writing of percent signs, thousandths places, and units of measurement; standardizing the delimiter styles for numbers, dates, and proper nouns in multilingual texts; and retaining structural tags such as list numbers, section titles, footnote labels, and table and figure caption placeholders to ensure the stability of subsequent boundary detection. This results in standardized text data oriented towards paragraph analysis.

[0054] S202: Detect and divide the paragraph boundaries of the normalized text data to obtain a set of paragraph units.

[0055] Paragraph boundary detection is performed on standardized text data, comprehensively utilizing layout features such as lines, paragraph wrapping, indentation styles, heading numbering patterns (e.g., "1.", "1.1"), list markers (e.g., "·", "-"), sentence-ending punctuation, blank line density, and page numbers and coordinates from step S106 to determine the start and end positions of paragraphs. Headers, footers, and repeated decorative lines are suppressed to avoid false boundaries. Tables, figure captions, and formula placeholders are modeled separately from the main text. Natural paragraphs broken by pagination are spliced ​​across pages. The final output is a set of paragraph units at the natural paragraph level, with metadata for each paragraph unit including the source page number, intra-paragraph offset, and hierarchy (e.g., heading, main text, list, figure caption).

[0056] S203: Perform semantic vector representation processing on the paragraph unit set to obtain a paragraph semantic representation set.

[0057] For a set of paragraph units, a pre-trained semantic encoding model can be used to generate semantic representations for each paragraph. Before encoding, word segmentation and tokenization are performed, and the atomicity of financial domain and cross-language proper nouns (such as "FOMC", "EPS", "current account") is preserved. Fixed-length representations are obtained through paragraph-level pooling, and the vectors are normalized and aligned in terms of norm and dimension. The paragraph semantic representations are bound to their source metadata (paragraph ID, page number range, hierarchical labels) to form a set of paragraph semantic representations, which serves as the basis for adjacent similarity calculation and subsequent segmentation and merging.

[0058] S204: Calculate the similarity between adjacent units of the semantic representation set of the paragraph to obtain candidate segmentation boundary data.

[0059] Semantic relevance indices are calculated for adjacent paragraphs (and, if necessary, neighboring paragraphs within a sliding window) in the paragraph sequence according to reading order. Relevance can be primarily based on cosine similarity, supplemented by multi-dimensional features such as term overlap, topic distribution distance, hierarchical transition penalties (e.g., low penalty for natural transitions from title to body text, and high penalty for cross-type transitions from body text to tables / captions), and page spanning costs. The similarity curves of adjacent pairs are smoothed and thresholded, and local troughs, inflection points, and hierarchical transition points are extracted and summarized to form candidate segmentation boundary data. This candidate segmentation boundary data consists of boundary location and its confidence level, boundary type (segmentation, merging hints), and triggering features, for use in the next step of decision-making.

[0060] S205: Based on the candidate segmentation boundary data, the paragraph unit set is segmented and merged respectively to obtain a text fragment set.

[0061] Using candidate segmentation boundary data as constraints, the paragraph sequence is segmented from left to right: when the similarity between adjacent paragraphs decreases significantly or a hierarchical shift occurs (e.g., switching from a "methodology" paragraph to "case studies, charts"), a segmentation point is identified; when the similarity between adjacent paragraphs is high and the paragraph length or information density is insufficient to form an independent semantic unit, they are merged into the same segment; list items are aggregated as a whole list, and table text and figure captions are bound according to semantic association; constraints such as minimum or maximum segment length, cross-page splicing, and chapter continuity are applied to avoid overly fragmented or excessively long segments. The final result is a set of text segments oriented towards retrieval and evidence alignment, where each segment maintains a mapping relationship with the source paragraph and page coordinates.

[0062] S206: The text fragment set is segmented and identified to generate a segmented text fragment set.

[0063] Generate a globally unique segment identifier for each segment in the text segment set. The identifier includes document ID, chapter level, page number range, and segment offset, and may include a hash check value. Synchronously write the segment's metadata index, including: segment type (title, body text, list, table, figure caption, formula placeholder), source page number range, original text coordinates, level label, upstream and downstream segment references, and pointer information required for subsequent vector index construction. Archive the segment content, identifier, and their metadata to form a segmented text segment set.

[0064] S30: The segmented text fragment set is vectorized and indexed to obtain a fragment vector index library.

[0065] A vectorized representation can be used to encode a collection of segmented text fragments, and a searchable index structure can be built accordingly. In the index structure, the fragment content is bound to its metadata to support efficient retrieval and location in the future, forming a fragment vector index library.

[0066] Combination Figure 4 As shown, step S30 specifically includes the following steps:

[0067] S301: Perform text vectorization encoding on the segmented text fragment set to obtain the initial fragment vector set.

[0068] The output set of segmented text fragments is tokenized and vector-encoded with each fragment as the smallest processing unit. Specifically, this includes: performing tokenization processing while maintaining the atomicity of fragments and the integrity of proper nouns, preserving the original tokens of financial abbreviations and cross-linguistic terms (e.g., currency symbols, identifiers, institutional abbreviations, etc.); dividing excessively long fragments into sliding window blocks according to the input constraints of the encoding model and performing pooling aggregation at the fragment level (e.g., using first-tag aggregation or average pooling) to obtain fixed-length representations; generating a fixed-length vector representation for each fragment, forming an initial set of fragment vectors, and maintaining index consistency with the fragment content and fragment identifier; and recording auxiliary statistics related to encoding (e.g., the number of effective tokens and the number of blocks) to provide a basis for subsequent quality control and normalization processing.

[0069] S302: Perform dimension alignment and normalization on the initial set of fragment vectors to obtain a standardized set of fragment vectors.

[0070] For initial vectors that may originate from different encoding configurations (such as multilingual, different windows, or different batches), unified spatial alignment and scale normalization are performed. Specifically, all initial vectors are mapped to a unified dimension of the target vector space. Linear transformation or equivalent orthogonal projection is preferred to complete cross-configuration alignment to ensure that the fragment representations from different sources are comparable in the same metric space. Centering and norm normalization are performed on the vectors (e.g., zero mean and L2 normalization) to avoid vector scale bias caused by differences in length or block partitioning strategies. Anomalous vectors (such as extremely short fragments or fragments with too many placeholders) are labeled or removed using the auxiliary statistics in step S301, and a set of standardized fragment vectors is output.

[0071] S303: Generate a fragment identifier based on the fragment standardized vector set and bind it to metadata to obtain a vector-metadata mapping record.

[0072] A globally unique fragment identifier is generated for each standardized vector and bound to the fragment's related metadata to form a searchable mapping record. The metadata includes at least: document identifier, chapter level, page number range, original text coordinates, fragment type, intra-fragment offset, language information, content hash summary, timestamp, and quality marker. The final result is a vector-metadata mapping record that supports bidirectional tracing from vectors to original text evidence.

[0073] S304: Construct an approximate nearest neighbor index for the vector-metadata mapping record to obtain a fragment nearest neighbor index structure.

[0074] Based on fragment-standardized vector sets and vector-metadata mapping records, an Approximate Nearest Neighbor (ANN) index structure for high-dimensional semantic retrieval is constructed: Clustering or partitioning is performed on vector samples to obtain several candidate centers and partition boundaries, used to organize the entry points for inverted lists or graph structures; each vector undergoes allocation and compression encoding (e.g., allocation to several inverted lists or graph nodes, and generation of compressed codewords) to reduce storage and computational overhead while ensuring recall; adjacency or routing relationships are established between index units (e.g., intra-cluster, inter-cluster pointers, or graph adjacency lists), and a reference link is established between each index unit and its vector-metadata mapping record; key parameters and version information of index construction (e.g., number of partitions, codebook size, graph metric threshold) are recorded to ensure consistency in subsequent retrieval and reconstruction, ultimately resulting in a fragment nearest neighbor index structure.

[0075] S305: Merge and store the fragment nearest neighbor index structure with the vector-metadata mapping record to obtain a fragment vector index library.

[0076] A partitioned and hierarchical layout can be adopted, storing the fragment nearest neighbor index structure (center, inverted index, graph links, compression code) and vector-metadata mapping records (fragment identifier, metadata) hierarchically, and establishing primary key-foreign key references through fragment identifiers. During the write process, atomic updates are performed on the index partitions and metadata tables, ensuring consistent access to the vector entry and metadata records of any fragment. A unified retrieval and backtracking interface specification is exposed at the database level, supporting nearest neighbor retrieval using query vectors as keys and returning fragment identifiers and metadata pointers. The database version, build time, and build parameters are recorded, reserving entry points for incremental updates and reconstruction to accommodate subsequent incremental data imports and index retraining. After the above merging and solidification, a fragment vector index library capable of vector retrieval and reordering is formed. This fragment vector index library achieves a unified organization of semantic representation, traceable metadata, and efficient nearest neighbor retrieval at the fragment level.

[0077] S40: Receive query requests based on the fragment vector index library and perform vector retrieval and reordering processes respectively to obtain candidate fragment sequence data.

[0078] The system receives query requests and performs representation transformations. It then uses the constructed fragment vector index library to complete vector retrieval and obtain preliminary relevant fragments. Subsequently, it reorders the retrieval results by integrating multidimensional relevance factors and outputs candidate fragment sequence data arranged by relevance.

[0079] Combination Figure 5 As shown, step S40 specifically includes the following steps:

[0080] S401: Receive a query request and parse the content of the query request to obtain query text data.

[0081] It receives query requests from users or upstream processes (such as issue analysis and task orchestration), parses and normalizes the query request payload to obtain query text data. The parsing process includes: identifying the query language and character encoding, cleaning up control characters and redundant whitespace, retaining proper nouns (such as "non-farm payrolls," "FOMC," "dollar index," and "term spread") and numerical symbols, extracting keywords and constraints (such as institution name, time range, chapter limitation, and page number range), and generating a unified query structure (including original text, standardized text, intent tags, and constraints) according to a predetermined template, providing consistent input for subsequent vectorization and retrieval.

[0082] S402: Perform vectorization encoding processing on the query text data to obtain query vector data.

[0083] The same semantic encoding model and tokenization strategy as in step S30 can be used to tokenize and encode the query text data, ensuring that the query vector and fragment vector are in the same metric space. For very long queries, a sliding window is used to aggregate them into a single query vector; for short queries, lightweight expansions using domain synonyms and aliases (e.g., "dollar index" and "Federal Reserve") are introduced before encoding to reduce representational bias caused by word form differences. After centering and norm normalization, the encoded results are output as query vector data, carrying metadata such as query ID, language label, and constraint parameters.

[0084] S403: Based on the query vector data, perform vector retrieval in the fragment vector index library to obtain a preliminary set of fragments.

[0085] Using the query vector as the key, an approximate nearest neighbor search is performed on the fragment vector index. A recall size and a minimum similarity threshold are set to achieve rapid candidate recall in high-dimensional space. The search process should follow the routing and access strategies of the index structure (corresponding to the index partitioning in step S305), returning several fragment identifiers and their similarity scores. To improve usability, a one-time hard filter is applied to the results based on query constraints (such as institution, time, and chapter level). Highly adjacent and semantically repetitive fragments within the same document are lightly deduplicated and merged to form a preliminary fragment set containing fragment identifiers, similarity scores, and basic metadata pointers.

[0086] S404: Extract fragment text using the initial selected fragment set and calculate multidimensional correlation features to obtain a multidimensional feature set.

[0087] Based on the identifiers in the initial fragment set, the fragment text and its metadata are retrieved back, and multidimensional relevance features for ranking are constructed, including at least: cosine similarity between query vector and fragment vector, fragment block aggregation similarity, semantic neighborhood density, standardized phrase matching score, domain term hit rate, amount, ratio, consistency of time expression, chapter level matching degree, offset from the nearest title, page number proximity, whether it spans pages, fragment length and information density, semantic coherence with adjacent fragments, whether it contains charts, formula placeholders and traceability, institution name, underlying asset (currency pair), and matching status of time window and regional label.

[0088] The above features are output together with the fragment identifier in vector or scalar form to form a multidimensional feature set.

[0089] S405: Perform rule reordering on the multidimensional feature set to obtain candidate fragment sequence data.

[0090] The initial selection results are then re-sorted and refined according to preset sorting rules, outputting candidate fragment sequence data. The sorting rules can employ a combination of hierarchical weighting and hard or soft constraints: first, hard rules are applied to eliminate fragments that do not meet key constraints (such as strong mismatches in organizational or temporal constraints); then, a weighted scoring function is used to score the remaining fragments, integrating features such as semantic similarity, lexical matching, structural position, and contextual availability, while introducing redundancy removal and diversity constraints to prevent homogeneous fragments from occupying the top positions; when scores are the same or close, chapter-level priority (abstracts and titles take precedence over general text) and page number position are used as the criteria for elimination. After sorting, candidate fragment sequence data is generated, arranged from high to low relevance, and the score decomposition and metadata pointer of each candidate are retained for subsequent context construction and window trimming and splicing processing.

[0091] S50: Perform context construction and window clipping and splicing processing on the candidate fragment sequence data to obtain context data.

[0092] By combining the original text location, chapter level, and page number information of the fragment, a contextual relationship can be constructed; based on this, window division, cropping, and sequential splicing can be performed, and splicing boundaries can be resolved to obtain contextual data that can be directly used for reasoning.

[0093] Combination Figure 6 As shown, step S50 specifically includes the following steps:

[0094] S501: Perform sequence normalization on the candidate fragment sequence data to obtain normalized candidate sequence data.

[0095] First, the candidate segment sequences are deduplicated and their order corrected. This involves removing completely duplicated and highly overlapping segments based on their segment identifiers, page number ranges, and intra-segment offsets. For reading order deviations caused by page or column crossings, the segments are rearranged according to the priority of "Document ID → Page Number → Paragraph Offset." Then, character-level representations are standardized (e.g., encoding style, merging full-width, half-width, and whitespace punctuation), while preserving the mapping relationship between each segment in the sequence and its original candidate ranking, similarity score, and source pointer. After these processes, a standardized candidate sequence data set with consistent structure, correct order, and traceability is formed.

[0096] S502: Based on the normalized candidate sequence data, extract the original text position, chapter level and page number information respectively to obtain the context metadata set.

[0097] For each segment within the normalized candidate sequence data, the following metadata is extracted back from the segment vector index database: the start and end page numbers of the original text, the coordinate boundaries within the page, the paragraph hierarchy path (e.g., "heading → second-level heading → body text"), the type of page area to which it belongs (heading, abstract, body text, list, table, figure caption, formula placeholder), and the anchoring relationship with adjacent headings or figures. For segments distributed across different pages or levels, they are uniformly converted into a path-based indexing format of "document – ​​chapter – page number – offset," and the consistency of source and hierarchical comparability between segments are maintained at the set level, thereby obtaining a contextual metadata set.

[0098] S503: Construct fragment adjacency relationships based on the aforementioned context metadata set to obtain context relationship graph data.

[0099] Using fragments as graph nodes, a weighted graph is constructed to represent multiple types of adjacency relationships: first, sequential adjacency (fragments within the same document that are adjacent or near-adjacent in reading order); second, hierarchical adjacency (fragments sharing the same title path or having parent-child or sibling hierarchical relationships); and third, referential adjacency (reference relationships between the main text and its associated tables, figure captions, and footnotes). Each edge is assigned a weight, determined by a combination of factors including page spacing, paragraph spacing, hierarchical consistency, and reference strength, with attenuation applied to weakly related edges across pages and hierarchical levels. The final output context graph data is stored as an edge list or a sparse adjacency structure, maintaining a one-to-one mapping with fragment identifiers.

[0100] S504: Divide the context relationship graph data into windows and clip the windows to obtain a set of window fragments.

[0101] Based on the input constraints and evidence continuity requirements of the language model, window partitioning is performed using context graph data: High-scoring segments of candidate sequences are prioritized as "window entry segments," expanding outwards to both sides according to edge weights until a set window budget (e.g., word count, tag count limit, or page spacing threshold) is reached; expansion stops and the window closes when a hierarchical breakpoint (title switching) or a weakly related cross-page edge is encountered. For each initial window, a pruning strategy is implemented to meet the input budget while maintaining semantic consistency: the first matching title within the window and its immediately following introductory paragraph are retained; for lengthy lists or tables, the title and overview rows are prioritized; necessary connecting sentences are retained for consecutive paragraphs spanning multiple pages; and recurring adjacent low-information-density segments are compressed. After pruning, the output consists of a set of several semantically continuous, well-defined window segments.

[0102] S505: The set of window segments is spliced ​​together in sequence and boundary resolution is performed to obtain spliced ​​window data.

[0103] For each window in the window fragment set, fragments are linearly stitched together according to the rule of "chapter level priority, and page number and paragraph offset increment within the same level". Boundary resolution is performed simultaneously: merging sentence and word breaks caused by pagination or column splitting, unifying list numbers and indentation levels, removing duplicate fragments caused by window overlap, and maintaining consistency in citation and annotation markers. Lightweight placeholder markers are inserted at the joints between tables, figure captions, and the main text to indicate the source type and location. The result is a window-stitched data set with continuous content, a clear structure, and traceable anchor points (page number, coordinates, and level labels).

[0104] S506: The window splicing data is tokenized and encoded to obtain context data.

[0105] According to the predefined tokenization specifications, the windowed data is converted into a sequence representation that can be directly consumed by the language model. Without altering the original semantics, structural markers (such as chapter start and end points, page number hints, and chart placeholders) are introduced to preserve source information. Cross-language terms and domain abbreviations are marked as is. For content exceeding the limits, a sliding offset and beginning / end retention truncation strategy is employed to ensure the overall sequence remains within the input budget. Simultaneously, a reverse mapping table from fragments to the tokenized sequence is retained to support subsequent evidence backtracking and template injection. After tokenization, the output contextual data serves as direct input for subsequent prompt word template injection and model inference.

[0106] S60: Inject the context data into a preset prompt word template to generate a target prompt word, and input the target prompt word into a language model for inference to generate a text optimization result.

[0107] Based on the preset prompt word template, the obtained context data is mapped and filled into the target prompt word, which is then input into the language model for inference calculation. The model output is then formatted and standardized to obtain a structured and reusable text optimization result.

[0108] Combination Figure 7 As shown, step S60 specifically includes the following steps:

[0109] S601: Perform structural parsing on the context data, extract the research report input information, context information and preprocessed knowledge assistant enhancement information respectively, and obtain the injection parameter set.

[0110] Contextual data was analyzed and cleaned for security purposes to identify and extract: Research report input information. message This includes information such as the applicant's name, the institution from which the application was submitted, the publication date, the currency, the subject matter, chapter titles, and original text snippets; and contextual information. content This refers to the continuous evidence text after window splicing, along with its page numbers, coordinates, and hierarchical anchor points; Knowledge Assistant Enhanced Information Inputs RAG The RAG knowledge assistant generates enhanced terminology definitions, indicator definitions, and methodological entries, retaining their source pointers and confidence markers. The three types of information undergo deduplication, conflict resolution, and label budget evaluation to form a set of injection parameters that can be directly used for template injection. For ease of description, this embodiment uses the following optimization relationship to uniformly represent the input of the large model:

[0111] Outputs query =LLM(Inputs) message +Inputs content +Inputs RAG ).

[0112] S602: Based on the injected parameter set, the preset prompt word template is mapped and generated to obtain a placeholder mapping table.

[0113] Following the core framework of "Role—Goals—Constraints—OutputFormat—Attention," a one-to-one mapping relationship is established between each placeholder in the template and the injected parameter set: Role specifies the domain role and working boundary of the large model; Goals indicate the goals of generating multi-level directories, paragraph summaries, and structured key points; Constraints constrain language, terminology consistency, citation traceability, and length budget; OutputFormat clarifies the format of the directory hierarchy, numbering style, key point fields, and evidence pointer list; Attention emphasizes that the evidence should not deviate from the context, and that missing information should be placed as "not detected" while maintaining traceability. After mapping, the output is a placeholder mapping table covering all placeholders.

[0114] S603: Use the placeholder mapping table to fill the injected parameter set into the prompt word template to obtain the target prompt word.

[0115] Based on the placeholder mapping table, inputs message Inputs content With Inputs RAG Fill the corresponding areas of the template as follows: fill the research report input information into the "Task Context Area" to indicate the source; fill the contextual evidence into the "Evidence Area" in the anchor format of "Fragment ID - Page Number - Level - Text"; fill the knowledge assistant enhancement information into the "Background Knowledge Area" in the key-value pair format of "Term - Definition, Definition - Source"; simultaneously write the five-segment instructions of Role, Goals, Constraints, OutputFormat and Attention, and perform consistency verification and label budget verification to generate target prompt words that are structurally complete and can be stably consumed by the language model.

[0116] S604: Input the target prompt word into the language model for inference calculation to obtain the model output data.

[0117] The target prompt word is submitted as a single input to the language model for inference, maintaining the same language and word segmentation assumptions as during vectorized encoding; parameters such as decoding upper limit and temperature are set to ensure that style and length are controllable; abnormal instructions or unauthorized generation are intercepted during inference and retried according to the established strategy, and model output data containing multi-level directories, key paragraph points, terminology and indicator entries and corresponding evidence citations are obtained, and runtime metadata such as model version and timestamp are recorded.

[0118] S605: Standardize the format of the model output data to obtain the text optimization result.

[0119] The model output data undergoes structural validation and format standardization, specifically: heading levels, numbering, field names, and list structure are corrected according to OutputFormat; terminology, units, time, and currency expressions are standardized according to Constraints; the validity of fragment IDs and page number anchors for each piece of evidence citation is verified and a citation index is generated; excess content is truncated and supplemented with placeholders according to the budget; and control characters and redundant tags irrelevant to the output are removed. After the above standardization process, the output is a text optimization result that meets template requirements, has traceable evidence, and can be directly used for downstream consumption.

[0120] As can be seen, this invention can be applied to markets such as finance, constructing an end-to-end process of "text extraction—semantic segmentation and indexing—fragment vector indexing—vector retrieval and reordering—context construction—template-based reasoning." Combined with the RAG knowledge assistant and the "Role–Goals–Constrains–OutputFormat–Attention" cue word framework, it uniformly injects evidence fragments and domain knowledge from multilingual research reports into a large model, achieving automatic generation of multi-level outlines and summaries, structured abstracts of key paragraphs, and standardized output of terminology and indicator definitions, while maintaining a traceable mapping to the original page number and fragment ID. This significantly improves the accuracy and consistency of key information extraction in interbank quantitative trading research, reduces comprehension biases caused by differences in cross-institutional and cross-language expressions, shortens the link from research report reading to strategy judgment, improves analytical efficiency and compliance verification, and facilitates integration with downstream factor libraries and strategy development systems.

[0121] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0122] In one embodiment, a research report text extraction and optimization device is provided, which corresponds one-to-one with the research report text extraction and optimization method in the above embodiments. For example... Figure 8 As shown, the text extraction and optimization device for this research report includes: a document acquisition module 100, a text processing module 200, an index building module 300, a fragment sorting module 400, a data splicing module 500, and a text output module 600. Detailed descriptions of each functional module are as follows:

[0123] The document acquisition module 100 is used to acquire research report documents and perform text extraction processing on the research report documents to obtain research report text data;

[0124] The text processing module 200 is used to perform semantic segmentation and indexing on the research report text data to obtain a set of segmented text fragments;

[0125] The index building module 300 is used to perform vectorization encoding on the segmented text fragment set and build an index to obtain a fragment vector index library;

[0126] The fragment sorting module 400 is used to receive query requests based on the fragment vector index library and perform vector retrieval and reordering processes respectively to obtain candidate fragment sequence data.

[0127] The data splicing module 500 is used to perform context construction and window clipping splicing processing on the candidate fragment sequence data to obtain context data;

[0128] The text output module 600 is used to inject the context data into a preset prompt word template to generate a target prompt word, and input the target prompt word into a language model for inference to generate a text optimization result.

[0129] In one embodiment, the document acquisition module 100 is specifically used for:

[0130] Retrieve research report documents from the target data source;

[0131] The research report document is decapsulated to obtain a decapsulated document object;

[0132] The decapsulated document object is parsed at the page level to obtain page structure data;

[0133] Based on the page structure data, page element segmentation and text recognition processing are performed to obtain element content data;

[0134] Based on the element content data, character encoding unification, noise removal, and text line reconstruction are performed respectively to obtain text line data;

[0135] The text lines are merged across lines to obtain the research report text data.

[0136] In one embodiment, the text processing module 200 is specifically used for:

[0137] The research report text data is normalized to obtain normalized text data;

[0138] Detect and delineate the paragraph boundaries of the normalized text data to obtain a set of paragraph units;

[0139] The semantic vector representation of the paragraph unit set is processed to obtain the paragraph semantic representation set;

[0140] The similarity between adjacent units of the semantic representation set of the paragraph is calculated to obtain candidate segmentation boundary data;

[0141] Based on the candidate segmentation boundary data, the paragraph unit set is segmented and merged respectively to obtain a text fragment set;

[0142] The text fragment set is labeled to generate a segmented text fragment set.

[0143] In one embodiment, the index building module 300 is specifically used for:

[0144] The segmented text fragment set is encoded using text vectorization to obtain an initial set of fragment vectors;

[0145] The initial set of fragment vectors is subjected to dimension alignment and normalization to obtain a standardized set of fragment vectors.

[0146] Based on the standardized vector set of fragments, a fragment identifier is generated and bound to metadata to obtain a vector-metadata mapping record;

[0147] An approximate nearest neighbor index is constructed on the vector-metadata mapping record to obtain a fragment nearest neighbor index structure;

[0148] The fragment nearest neighbor index structure is merged and stored with the vector-metadata mapping record to obtain the fragment vector index library.

[0149] In one embodiment, the fragment sorting module 400 is specifically used for:

[0150] Receive a query request and parse the content of the query request to obtain query text data;

[0151] The query text data is vectorized and encoded to obtain query vector data.

[0152] Based on the query vector data, a vector retrieval is performed in the fragment vector index library to obtain a preliminary set of fragments;

[0153] The text of the selected segments is extracted using the initial set of segments, and multidimensional relevance features are calculated to obtain a multidimensional feature set.

[0154] The multidimensional feature set is reordered according to rules to obtain candidate fragment sequence data.

[0155] In one embodiment, the data splicing module 500 is specifically used for:

[0156] The candidate fragment sequence data is normalized to obtain normalized candidate sequence data;

[0157] Based on the normalized candidate sequence data, the original text position, chapter level and page number information are extracted to obtain a set of context metadata;

[0158] Based on the aforementioned context metadata set, fragment adjacency relationships are constructed to obtain context relationship graph data;

[0159] The context graph data is divided into windows and then clipped to obtain a set of window fragments;

[0160] The window fragment set is concatenated in sequence and boundary resolution is performed to obtain the window concatenation data;

[0161] The concatenated window data is tokenized and encoded to obtain context data.

[0162] In one embodiment, the text output module 600 is specifically used for:

[0163] The context data is structurally parsed to extract the research report input information, context information, and preprocessed knowledge assistant enhancement information to obtain the injection parameter set;

[0164] Based on the injected parameter set, a preset prompt word template is mapped and generated to obtain a placeholder mapping table;

[0165] The placeholder mapping table is used to fill the injected parameter set into the prompt word template to obtain the target prompt word;

[0166] The target prompt words are input into the language model for inference calculation to obtain the model output data;

[0167] The output data of the model is formatted to obtain the text optimization result.

[0168] Specific limitations regarding the research report text extraction and optimization device can be found in the limitations of the research report text extraction and optimization method described above, and will not be repeated here. Each module in the aforementioned research report text extraction and optimization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0169] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a research report text extraction optimization method on the server side.

[0170] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements client-side functions or steps of a research report text extraction optimization method.

[0171] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, can perform the steps provided in the above embodiments.

[0172] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0173] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0174] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0175] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for extracting and optimizing research report text, characterized in that, include: Obtain the research report document and perform text extraction processing on the research report document to obtain the research report text data; The research report text data is semantically segmented and indexed to obtain a set of segmented text fragments; The segmented text fragment set is vectorized and indexed to obtain a fragment vector index library; Based on the fragment vector index library, query requests are received and vector retrieval and reordering processes are performed respectively to obtain candidate fragment sequence data; The candidate fragment sequence data are subjected to context construction and window cropping and splicing processes respectively to obtain context data; The contextual data is injected into a preset prompt word template to generate target prompt words, and the target prompt words are then input into a language model for inference to generate text optimization results.

2. The method for extracting and optimizing research report text according to claim 1, characterized in that, The process of acquiring research report documents and performing text extraction processing on the research report documents to obtain research report text data includes: Retrieve research report documents from the target data source; The research report document is decapsulated to obtain a decapsulated document object; The decapsulated document object is parsed at the page level to obtain page structure data; Based on the page structure data, page element segmentation and text recognition processing are performed to obtain element content data; Based on the element content data, character encoding unification, noise removal, and text line reconstruction are performed respectively to obtain text line data; The text lines are merged across lines to obtain the research report text data.

3. The method for extracting and optimizing research report text according to claim 1, characterized in that, The semantic segmentation and indexing of the research report text data yields a set of segmented text fragments, including: The research report text data is normalized to obtain normalized text data; Detect and delineate the paragraph boundaries of the normalized text data to obtain a set of paragraph units; The semantic vector representation of the paragraph unit set is processed to obtain the paragraph semantic representation set; The similarity between adjacent units of the semantic representation set of the paragraph is calculated to obtain candidate segmentation boundary data; Based on the candidate segmentation boundary data, the paragraph unit set is segmented and merged to obtain a text fragment set; The text fragment set is labeled to generate a segmented text fragment set.

4. The method for extracting and optimizing research report text according to claim 1, characterized in that, The step of vectorizing and indexing the segmented text fragment set to obtain a fragment vector index library includes: The segmented text fragment set is encoded using text vectorization to obtain an initial set of fragment vectors; The initial set of fragment vectors is subjected to dimension alignment and normalization to obtain a standardized set of fragment vectors. Based on the standardized vector set of fragments, a fragment identifier is generated and bound to metadata to obtain a vector-metadata mapping record; An approximate nearest neighbor index is constructed on the vector-metadata mapping record to obtain a fragment nearest neighbor index structure; The fragment nearest neighbor index structure is merged and stored with the vector-metadata mapping record to obtain the fragment vector index library.

5. The method for extracting and optimizing research report text according to claim 1, characterized in that, The process of receiving query requests based on the fragment vector index and performing vector retrieval and reordering to obtain candidate fragment sequence data includes: Receive a query request and parse the content of the query request to obtain query text data; The query text data is vectorized and encoded to obtain query vector data. Based on the query vector data, a vector retrieval is performed in the fragment vector index library to obtain a preliminary set of fragments; The text of the selected segments is extracted using the initial set of segments, and multidimensional relevance features are calculated to obtain a multidimensional feature set. The multidimensional feature set is reordered according to rules to obtain candidate fragment sequence data.

6. The method for extracting and optimizing research report text according to claim 1, characterized in that, The process of performing context construction and window cropping and splicing on the candidate fragment sequence data to obtain context data includes: The candidate fragment sequence data is normalized to obtain normalized candidate sequence data; Based on the normalized candidate sequence data, the original text position, chapter level and page number information are extracted to obtain a set of context metadata; Based on the aforementioned context metadata set, fragment adjacency relationships are constructed to obtain context relationship graph data; The context graph data is divided into windows and then clipped to obtain a set of window fragments; The window fragment set is concatenated in sequence and boundary resolution is performed to obtain the window concatenation data; The concatenated window data is tokenized and encoded to obtain context data.

7. The method for extracting and optimizing research report text according to claim 1, characterized in that, The step of injecting the context data into a preset prompt word template to generate a target prompt word, and then inputting the target prompt word into a language model for inference to generate text optimization results, includes: The context data is structurally parsed to extract the research report input information, context information, and preprocessed knowledge assistant enhancement information to obtain the injection parameter set; Based on the injected parameter set, a preset prompt word template is mapped and generated to obtain a placeholder mapping table; The placeholder mapping table is used to fill the injected parameter set into the prompt word template to obtain the target prompt word; The target prompt words are input into the language model for inference calculation to obtain the model output data; The output data of the model is formatted to obtain the text optimization result.

8. A research report text extraction and optimization device, characterized in that, include: The document acquisition module is used to acquire research report documents and perform text extraction processing on the research report documents to obtain research report text data; The text processing module is used to perform semantic segmentation and indexing on the research report text data to obtain a set of segmented text fragments; The index building module is used to vectorize and encode the segmented text fragment set and build an index to obtain a fragment vector index library; The fragment sorting module is used to receive query requests based on the fragment vector index library and perform vector retrieval and reordering processes respectively to obtain candidate fragment sequence data. The data splicing module is used to perform context construction and window clipping splicing processing on the candidate fragment sequence data to obtain context data; The text output module is used to inject the context data into a preset prompt word template to generate target prompt words, and input the target prompt words into a language model for inference to generate text optimization results.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the research report text extraction and optimization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the research report text extraction and optimization method as described in any one of claims 1 to 7.

Images