Method of automated processing and search of relevant documents based on list of procurement objects

Abstract

FIELD: data processing.SUBSTANCE: invention relates to the field of automated processing and searching for relevant documents based on structured data in the form of a matrix with fields comprising string representations of information. The method includes the following stages: importing structured data in the form of a matrix with fields comprising string representations of information, performing exact matching of terms, including normalization of texts using n-grams and excluding documents comprising predefined exclusion words, performing semantic analysis using machine learning and vector representations, in which unique terms are extracted and repeated terms are combined to form a set of unique terms and unique terms are converted into vector representations, in three different modes: a combining mode, in which the matrix fields are combined into a single row, a separate mode, in which vector representations of unique terms are used, and a hybrid mode, in which the results of the separate mode are adjusted by recalculating weights, as well as forming a final list of documents by combining the results of the modes with the exclusion of duplicates and ranking based on the relevance assessment.EFFECT: increasing the completeness of extraction of relevant documents.3 cl, 1 dwg

Description

[0001] The field of technology to which the invention relates.

[0002] The invention relates to the field of automated processing and search of relevant documents, for example, according to lists of goods, services, any objects for procurement, and can be used to search for relevant goods, services, tenders.

[0003] State of the art

[0004] Methods for automated processing and retrieval of relevant documents from a procurement list exist. For example, a method for automated processing and retrieval of relevant documents from a procurement list is known from the prior art, described in US Patent Application US20240312451A1, published in 2024. This method uses multimodal embedding of patent documents to search for prior art, but is not adapted for structured procurement data and does not take into account the specifics of the categories or subjects of the objects. Similarly, patent US11288319B2 describes a hybrid semantic search for document classification, but it is focused on general information retrieval without a focus on procurement. Publication EP2583203A2 proposes semantic content search, but lacks multimodal analysis with relevance correction.

[0005] The disadvantages of existing solutions include their inability to work with structured procurement data and the lack of specific features for working with procurement categories / entities, resulting in low-completeness of relevant document extraction. In particular, a basic (single) search based on a simple vector representation without multimodality yields significantly fewer relevant results and is also of lower quality.

[0006] Disclosure of invention.

[0007] The present invention is primarily aimed at proposing a method for automated processing and searching for relevant documents in a list of procurement objects, which makes it possible to increase the completeness of the extraction of relevant documents, which is the technical problem being solved.

[0008] To achieve this goal:

[0009] - import structured data of the list of procurement objects in the form of a matrix with fields that include, at least, the category of procurement objects, the subjects of the procurement objects, and the name of the procurement objects;

[0010] - perform an exact match, in which:

[0011] • Perform text normalization using n-grams,

[0012] • check the inclusion of terms from the specified list in the descriptions of procurement object documents,

[0013] • exclude documents containing predefined exclude words;

[0014] - performs semantic analysis of documents regarding the list of procurement objects using machine learning in three different modes:

[0015] • merge mode, in which the data of the specified fields is combined into a single string, converted into a vector representation based on the attention mechanism model, and compared with documents using the cosine similarity metric, providing contextual search with stable results (e.g., 193 relevant documents with an average score of 62.94%);

[0016] • split mode, in which unique terms are extracted from the specified fields, vector representations are formed using the “bag of words” model, and comparison is performed taking into account synonymous matches, ensuring the completeness of the volume (for example, 239 documents with an average score of 62.95%);

[0017] • hybrid mode, in which the results of the split mode are adjusted by dynamically increasing the relevance score when terms from the “subject of procurement object” field are detected in the document, ensuring focus on highly relevant matches (e.g. 239 documents with an average score of 63.33%, all added >70%);

[0018] - filter documents by discarding elements with a similarity score below the threshold;

[0019] - generates a final list of documents by eliminating duplicate identifiers and ranking the found documents based on relevance, where the multimodal approach yields several times more relevant results compared to a basic single search (284 versus 39, an increase of 629%).

[0020] These advantageous features enable the ability to work with structured procurement data, include the ability to work with procurement categories / entities, ensure the completeness of relevant document extraction (47–629% increase), improve quality (0.34–1.7% increase in average similarity score and 120–1000% increase in high matches), and reduce computational load (3 times faster than the baseline).

[0021] There is an embodiment of the invention in which the threshold value of cosine similarity is set manually in the range from 0.1 to 0.9.

[0022] These advantageous characteristics allow for a balance between recall (a low threshold results in more results, but with noise) and precision (a high threshold results in less noise, but at the risk of losing relevant results). The range of 0.1–0.9 is chosen empirically: below 0.1, there are too many false matches (noise >90%), while above 0.9, there is a loss of recall (>50% of relevant results are rejected); 0.6 is optimal for typical purchasing data (recall / precision balance of ~63%).

[0023] There is a variant of the invention in which the threshold value of cosine similarity varies adaptively depending on the number of documents found.

[0024] Thanks to these advantageous characteristics, dynamic optimization becomes possible: if there are few documents, the threshold can be lowered to increase completeness; if there are many, the threshold can be raised to reduce noise, ensuring adaptation to the volume of data without manual adjustment.

[0025] There is an embodiment of the invention in which the increase in the relevance score in the hybrid mode is carried out by a fixed coefficient in the range from 0.05 to 0.2.

[0026] These advantageous characteristics make it possible to fine-tune subjective matches without overestimating them. The range of 0.05–0.2 was chosen empirically: below 0.05, the effect is weak (rating increase <1%), while above 0.2, there is a risk of false top scores (overestimation >20%). The optimal value is 0.1, which increases the average rating by 0.4% and high matches by 120–1000%.

[0027] There is a variant of the invention in which semantic analysis is carried out using a pre-trained model on a large volume of lists of procurement objects.

[0028] These advantageous characteristics make it possible to use a pre-trained model, which optimizes the search by better processing the purchasing context, increasing the accuracy by 3-10% compared to baseline models.

[0029] Brief description of the drawings.

[0030] Other distinctive features and advantages of the proposed solution clearly follow from the description given below for illustration and not as limiting, with reference to the attached drawings, in which:

[0031] - Figure 1 shows, according to the invention, a block diagram of the process.

[0032] Figure 1 illustrates the invention's sequence of steps, with parallel execution of exact matching and semantic analysis. It is represented as a directed graph, where blocks represent process steps and arrows represent connections (data flow or execution sequence). The process begins with data import, then branches into parallel branches (exact and semantic), which are merged in the filtering step. Each step has a unique identifier, a function description, and an input / output.

[0033] Step 1: Importing Structured Data

[0034] Function: Import a list of procurement objects in the form of a matrix with fields (procurement object category, procurement object subjects, procurement object name).

[0035] Input: File or database (eg spreadsheet).

[0036] Output: Structured data matrix passed in parallel to stages 2 and 3.

[0037] Stage 2: "Exact Match"

[0038] Function: Text normalization (using n-grams), checking for terms in document descriptions, excluding documents with predefined exclusion words (e.g., "repair", "services").

[0039] Input: Data matrix from stage 1, document descriptions from the source.

[0040] Output: Filtered list of documents with exact matches (passed to step 4 in parallel).

[0041] Stage 3: "Semantic analysis (multimodal)"

[0042] Function: Document similarity calculation using machine learning in three modes. This is a parallel block with branching submodes.

[0043] Input: Data matrix from stage 1, complete set of documents from the source (regardless of the exact one).

[0044] Output: Similarity results from sub-stages (3.1, 3.2, 3.3) are fed in parallel to stage 4.

[0045] Stage 3.1: "Unification Mode"

[0046] Function: Combine fields into a single string, convert to vector representation (based on the attention mechanism type model), compare by cosine similarity.

[0047] Input: Data from step 3.

[0048] Output: Similarity scores for the concatenated strings.

[0049] Stage 3.2: "Split mode".

[0050] Function: Extract unique terms from fields, generate vector representations (using the bag-of-words model), and compare using synonymous matches.

[0051] Input: Data from step 3.

[0052] Output: Similarity scores for unique terms.

[0053] Stage 3.3: "Hybrid Mode"

[0054] Function: Results from split mode with relevance correction (increase score when detecting subjects, such as producers).

[0055] Input: Data from step 3.

[0056] Output: Adjusted similarity scores.

[0057] Step 4: "Document filtering".

[0058] Function: Discard elements with similarity score below threshold (manual or adaptive), merge results from parallel branches.

[0059] Input: Results from stage 2 (exact) and stages 3.1, 3.2, 3.3 (semantic).

[0060] Output: Combined filtered list of relevant documents.

[0061] Stage 5: “Formation of the final list.”

[0062] Function: Exclude duplicate identifiers, rank found documents based on relevance score.

[0063] Input: Filtered list from step 4.

[0064] Output: Final ranked list of documents.

[0065] Communications between blocks (process flow)

[0066] Stage 1 branches off in parallel into Stage 2 and Stage 3: “Exact Match” and “Semantic Analysis”.

[0067] If an exact match is found, then after step 2, go to step 4.

[0068] If an exact match is not found, then a semantic analysis is performed, that is, stage 3 consists of three sub-stages:

[0069] Stage 3.1: "Unification Mode"

[0070] Stage 3.2: "Split mode".

[0071] Stage 3.3: "Hybrid Mode"

[0072] As a result, documents are filtered (stage 4) and a final list is formed (stage 5)

[0073] Implementation of the invention.

[0074] The present invention primarily aims to provide a method for automated processing and searching of relevant documents from a list of procurement objects.

[0075] The following detailed description of a possible embodiment of the invention provides numerous details and variations aimed at providing a clear understanding of the essence of the invention and possible methods for implementing the present invention. However, any skilled person will readily understand how the present invention can be used with or without the details and variations listed below, or with combinations of such variations.

[0076] One of the embodiments of this method of automated processing and searching for relevant documents on the list of procurement objects and illustrating the logic of application of this invention can be implemented as follows.

[0077] Import a list of procurement objects from a file (spreadsheet or database) into structured fields for data classification (type, manufacturer, name).

[0078] Retrieve procurement objects from a data source (API or procurement database) filtered by specified criteria (status, date, type, or other parameters).

[0079] Exact matching is performed: comparison of procurement item names with procurement item descriptions (line entries), with filtering of exclusion words (e.g., "repairs," "services") to eliminate irrelevant ones.

[0080] Perform semantic analysis: use machine learning (text vector representation model) to calculate similarity (cosine similarity) in three modes:

[0081] - Data merging (concatenation of fields into a single string for analysis).

[0082] - Split into words (unique words from fields for granular matching).

[0083] - Hybrid (unique words with relevance correction based on key features such as manufacturer).

[0084] Similarity threshold cuts off irrelevant, excluding words, filters out noise.

[0085] Unique output of this method: combining results by procurement object identifiers, sorting by relevance (exact matches at the top, semantic matches by similarity). Unlike a basic single search, which yields significantly fewer relevant results (39 versus 284, or 193 versus 284 in the structured data version), the proposed method provides a 47–629% increase in recall.

[0086] Examples of implementing the method.

[0087] Example 1: A test search for relevant documents (tenders) was performed using structured data (a matrix with categories, subjects, and names of procurement objects). A basic single search (without multimodality) identified 193 relevant documents with an average similarity score of 62.94% and 5 high matches (>70%). The proposed method, according to the invention, identified 284 relevant documents (a 47% increase in completeness), with an average score of 63.28% (an increase of 0.34%) and 11 high matches (an increase of 120%). The separate mode ensured completeness (171 documents in the final list), while the hybrid mode ensured quality (9 documents, all >70%, average 72.11%).

[0088] Example 2: In the basic version without structured data, a single search yielded only 39 relevant documents with an average score of 61.58% and 1 high match. The inventive method, using the same data but structured data, yielded 284 documents (a 629% increase in recall), demonstrating that the basic search yields significantly fewer results due to the lack of multimodal analysis and structured data.

[0089] Industrial applicability.

[0090] The proposed method for automated processing and searching of relevant documents within a procurement list can be implemented by a specialist in practice and, when implemented, ensures the achievement of the stated purpose, which allows for a conclusion regarding compliance with the "industrial applicability" criterion for the invention. The described method is implemented using traditional technologies and does not pose any additional technical challenges.

[0091] The proposed method for automated processing and searching for relevant documents on the list of procurement objects has been implemented in practice and ensured the implementation of the stated purpose, which allows us to conclude that the invention meets the “industrial applicability” criterion.

[0092] Structured data import (matrix with character sequence normalization for cleaning), exact matching (term entry for basic relevance), and multimodal semantics in modes (attribute merging for context; splitting into unique terms based on relationships with score correction) were performed. The method was shown to provide:

[0093] - increase in extraction completeness by 47–629% (tests: 284 documents versus 39 in the basic single mode without structured data, which is several times less relevant results; or 284 versus 193 in the single mode with structured data),

[0094] - classification accuracy of 0.34–1.7% (average score ~63.28% vs. ~61.58% in the baseline),

[0095] - 70% reduction in computational load (fewer iterations due to unique terms + correction, better graphs for optimization; in hybrid mode, the load is reduced by more than 70% due to a reduction in the number of iterations when extracting unique terms and correcting the relevance score; multimodal mode with structured data reduces the load compared to the basic single mode without structured data, where processing is less optimized and leads to large time costs for inefficient search).

[0096] Overall, it is shown that the technical result is achieved – an increase in the completeness of the extraction of relevant documents by 47–629% and a reduction in the computational load is achieved due to the triad approach, in contrast to the basic single search, which yields several times fewer relevant results (39–193 versus 284).

[0097] Also, an increase in the number of high matches of +120–1000% was found due to the adjustment of the relevance score and a combination of machine learning models.

Claims

1. A method for automated processing and searching for relevant documents based on structured data in the form of a matrix with fields containing string representations of information, in which: • import structured data; • perform exact term matching, which normalizes texts using n-grams and excludes documents containing predefined exclusion words; • perform semantic analysis using machine learning and vector representations, characterized in that: • unique terms are extracted from the matrix fields and repeating terms are combined to form a set of unique terms; • transformation into vector representations is performed only for the specified unique terms; • semantic analysis is carried out in three modes: - merging mode, in which the matrix fields are combined into a single row; - separate mode, which uses vector representations of unique terms; - and a hybrid mode, in which the results of the separate mode are adjusted by recalculating the weights of the vector representations when terms from a predetermined field are detected, and the recalculation of the weights is performed using the set of unique terms obtained at the stage of the separate mode; • form a final list of documents by combining the results of the modes, eliminating duplicate identifiers and ranking them based on relevance assessment.

2. The method according to paragraph 1, characterized in that the recalculation of weights is carried out using a coefficient in the range from 0.05 to 0.

2.

3. The method according to paragraph 1, characterized in that the threshold value of cosine similarity varies adaptively depending on the number of documents found.

Images