Multimodal duplicate checking method and system for bidding documents based on large model

Abstract

The present invention belongs to the field of natural language processing and information retrieval technology. It provides a large-scale model-based multimodal bid duplication checking method and system, which performs structured parsing and multimodal feature extraction on input bids to generate feature representations of semantic blocks and non-text elements. It uses a dynamic context-aware large language model to perform deep semantic encoding on text blocks, combining hierarchical position encoding to preserve the logical relevance of long texts. It achieves efficient matching of massive semantic vectors through a hybrid retrieval architecture, and optimizes computing efficiency by combining a distributed computing framework with hardware acceleration instructions. Finally, it reconstructs structured features of non-text content such as tables and charts, achieving cross-modal semantic association analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing and information retrieval, and in particular to a large-model-based multimodal bid duplication checking method and system. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] Current bid duplication detection technologies mainly rely on two methods: rule matching and shallow semantic analysis. Rule matching technology uses preset text patterns or fragment overlap to detect duplicates. Although it is simple to implement, it has difficulty dealing with semantic duplication problems with differentiated expressions. For example, it cannot recognize synonym replacements, sentence adjustments, or logical reorganizations. In addition, the construction and maintenance costs of the rule base are high. Methods based on shallow semantic analysis use statistical machine learning models to calculate text similarity through word frequency or shallow semantic features. Although this alleviates the problem of semantic generalization to a certain extent, due to the limited representation capabilities of the model, the accuracy of capturing complex logical relationships in long texts is insufficient. In particular, when faced with bids with dense professional terms and complex structures, the accuracy of duplicate detection drops significantly.

[0004] The limitations of existing technologies are mainly reflected in three aspects: first, the semantic understanding is not deep enough. Traditional models lack the ability to dynamically learn contextual associations and domain knowledge, resulting in a high rate of missed detection of semantically equivalent but very different expressions of content; second, domain adaptability is poor. The professional terminology and specifications involved in bids from different industries vary greatly. Existing methods need to be optimized separately for each field, and the implementation cost is high; third, computational efficiency is limited. Especially when processing large-scale bid repositories, traditional algorithms are difficult to meet real-time requirements due to the complexity of high-dimensional sparse vector calculations and long text segmentation processing.

[0005] In recent years, with the prevalence of pre-trained language models, some improvement proposals have attempted to introduce general semantic models to improve duplicate checking accuracy. However, these approaches still have significant shortcomings. For example, they fail to design differentiated processing strategies for the unique structured content of bid documents (such as technical proposals and quotation lists), resulting in poor duplicate checking results for non-text elements such as tables and clauses. Furthermore, directly using unoptimized general models can easily lead to response delays and lack support for dynamic updates to industry terminology. Furthermore, existing technologies generally ignore the fusion analysis of multimodal data, such as the combined duplication checking of charts, formulas, and text, further limiting the expansion of application scenarios. Summary of the Invention

[0006] In order to address the shortcomings of the existing technology, the present invention provides a multimodal bidding document duplication checking method and system based on a large model, which utilizes the context perception capability of a large language model to achieve deep semantic duplication checking, accurately identify the semantic equivalence between texts, and reconstruct structured features of non-text content such as tables, charts, and formulas, thereby realizing cross-modal semantic association analysis and improving the accuracy of duplication checking.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a multimodal duplication checking method for bid documents based on a large model.

[0009] A multimodal bid duplication checking method based on a large model includes the following steps:

[0010] Preprocess the original bid document to obtain text segments with hierarchical labels and non-text multimodal features;

[0011] Based on a dynamic context-aware large language model, semantic encoding is performed on the text blocks to obtain semantic vectors corresponding to the text blocks;

[0012] Building an industry-specific hierarchical index based on the semantic vector, dynamically adjusting the hierarchical structure of the index, and determining semantic similarity based on the hierarchical structure of the index;

[0013] Based on the semantic vector of the original bid, the multimodal features of non-text, and the multimodal features of the candidate documents, a cross-modal attention calculation is performed to determine the weighting coefficients of each modality. The weights of each modal feature in the candidate document are dynamically allocated according to the proportion of each modal feature in the original bid.

[0014] The non-text similarity is determined according to the weighting coefficient, the weight and the specific matching degree of various modalities, and the global similarity between the original bid and the candidate document is determined according to the weighted sum of the semantic similarity and the non-text similarity.

[0015] In an implementation of the first aspect of the present invention, the original bid is preprocessed to obtain multimodal associated metadata, and the semantic vector is associated with the multimodal associated metadata;

[0016] Extract the keyword set from the original bid document and determine multiple candidate documents through correlation calculation. This process is distributed to multiple computing nodes for parallel execution.

[0017] The semantic similarity between the semantic vector of each candidate document and the query vector of the original bid is calculated. This process is performed by parallel vector operation using the SIMD instruction set.

[0018] In an implementation of the first aspect of the present invention, preprocessing the original bid document to obtain text blocks with hierarchical labels includes:

[0019] Based on regular expression matching of titles, the weight of the title is calculated by integrating font size, bold attributes and position information. The weight is dynamically determined by a linear combination of the normalized font size value, bold flag and position weight. When the weight exceeds the preset threshold, it is marked as a title and a hierarchical index tree is generated to split the original bid into independent text blocks with hierarchical labels.

[0020] In an implementation of the first aspect of the present invention, preprocessing the original bid document to obtain multimodal features includes: converting table content into a logical tree structure;

[0021] Use the target detection model to locate the coordinate axes, legends, and data points in the chart, extract the value range vector, and construct the topological relationship matrix of the data points based on the graph neural network;

[0022] The formula or image formula obtained by the formula editor is converted into a symbolic tree structure, a serialized expression is generated through breadth-first traversal, and a hash fingerprint is generated using locality-sensitive hashing;

[0023] The logical tree structure, the topological relationship matrix and the hash fingerprint constitute the multimodal feature.

[0024] In one implementation of the first aspect of the present invention, any text block is jointly represented by word embedding, paragraph type embedding and hierarchical position coding. The hierarchical position coding dynamically adjusts the coding frequency according to the title level. Before the joint representation result is input into the large language model, the domain adaptation unit is used to dynamically splice industry-specific prompt templates.

[0025] In an implementation of the first aspect of the present invention, dynamically adjusting the hierarchical structure of the index includes: Sub-libraries, number of index levels According to the data scale Adaptive Computing: ,in, is the level sparsity factor.

[0026] In one implementation of the first aspect of the present invention, performing cross-modal attention calculation includes: ,in, Original bid No. The semantic vector corresponding to the text block, is the candidate document multimodal fusion function, Representative candidate documents No. The semantic vector corresponding to the text block, Representative candidate documents No. The non-text multimodal features corresponding to the text blocks, are all trainable projection matrices, is the vector dimension.

[0027] In a second aspect, the present invention provides a multimodal duplication checking system for bid documents based on a large model.

[0028] A multimodal bidding document duplication checking system based on a large model, comprising:

[0029] The preprocessing unit is configured to: preprocess the original bid document to obtain text blocks with hierarchical labels and non-text multimodal features;

[0030] A semantic processing unit is configured to: perform semantic encoding on the text block based on a dynamic context-aware large language model to obtain a semantic vector corresponding to the text block;

[0031] A retrieval calculation unit is configured to: construct an industry-specific hierarchical index based on the semantic vector, dynamically adjust the hierarchical structure of the index, and determine semantic similarity based on the hierarchical structure of the index;

[0032] The multimodal computing unit is configured to: perform cross-modal attention calculation based on the semantic vector of the original bid, the multimodal features of the non-text, and the multimodal features of the candidate documents to determine the weight coefficients of each modality, and dynamically allocate the weights of each modal feature in the candidate document based on the proportion of each modal feature in the original bid;

[0033] The joint duplicate checking unit is configured to: determine the non-text similarity based on the weighting coefficient, the weight and the special matching degree of various modalities, and determine the global similarity between the original bid and the candidate document based on the weighted sum of the semantic similarity and the non-text similarity.

[0034] In a third aspect, the present invention provides a computer device comprising: a processor and a computer-readable storage medium;

[0035] a processor adapted to execute a computer program;

[0036] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the multimodal duplicate checking method for bids based on a large model as described in the first aspect of the present invention is implemented.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by a processor and executing the multimodal duplication checking method for bids based on a large model as described in the first aspect of the present invention.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention innovatively proposes a large-model-based bid duplication checking method, which uses the context-awareness of a large language model to achieve deep semantic duplication checking, accurately identify the semantic equivalence between texts, reconstruct structured features of non-text content such as tables and charts, and realize cross-modal semantic association analysis. By integrating dynamic context-aware coding, domain-adaptive optimization, and multimodal joint analysis technology, an intelligent duplication checking strategy covering both text and non-text content is constructed.

[0040] 2. In view of the limitations of traditional technologies, the present invention designs a multi-level collaborative optimization mechanism. At the semantic understanding level, the sliding window and hierarchical position coding technology are used to solve the semantic discontinuity problem of long texts. For example, the multi-level titles in the tender documents are mapped into hierarchical vectors to capture the contextual dependencies across paragraphs and chapters, thereby accurately identifying semantic repetitions caused by sentence reorganization or logical adjustments. At the domain adaptation level, pluggable parameter adapters and incremental fine-tuning frameworks are introduced to support dynamic loading of terminology libraries and tender specifications for different industries. Only a small number of industry samples are required to complete the adapter parameter update, significantly reducing the cost of cross-domain migration. At the computing architecture level, a hybrid pipeline combining inverted index and dense vector retrieval is proposed. In the coarse screening stage, the candidate set range is quickly narrowed down based on keywords and metadata. In the fine ranking stage, accurate sorting is achieved through semantic vector similarity calculation. At the same time, distributed task scheduling and SIMD instruction set parallelization are used to accelerate calculations to ensure real-time response under billions of data volumes.

[0041] 3. The present invention breaks through the bottleneck of traditional methods in processing non-text elements, and proposes a multimodal joint duplicate checking solution. Structured feature extraction methods are designed for tables, charts and formulas in the bid documents: the table content is parsed into a logical tree and the row and column relationship features are extracted, the charts are matched for topological similarity through graph neural networks, and the formulas are converted into symbolic trees to generate hash fingerprints. The text and non-text features are integrated through the cross-modal attention mechanism to realize the joint analysis of text and image semantics. For example, the text description in the technical solution is associated and matched with the table data in the quotation to avoid missed detection problems caused by content dispersion.

[0042] 4. The present invention has achieved significant breakthroughs in semantic understanding, domain adaptation and multimodal processing. Through the dynamic context encoding mechanism, the system can accurately identify semantically repeated content in bid documents caused by sentence reorganization, synonym replacement or logical adjustment, especially for long texts with dense professional terms, the accuracy of duplicate detection is significantly improved; the introduction of parameter adapters and incremental fine-tuning frameworks greatly shortens the migration and deployment cycle of cross-industry bid document specifications, while avoiding the resource consumption of full model fine-tuning; the combination of hybrid retrieval architecture and distributed optimization strategy ensures that the retrieval response time of 100 million-level bid document libraries is reduced to milliseconds, meeting the real-time requirements of bidding scenarios. For non-text content, the multimodal joint duplicate detection solution effectively solves the problem of missed detection of elements such as tables and charts by traditional methods through structured feature extraction and cross-modal correlation analysis, such as semantic integrity verification of merged cells, quantitative matching of chart topology similarity, and fuzzy hash comparison of formula symbol trees, thereby forming a duplicate detection capability covering all elements.

[0043] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0045] Figure 1 A schematic flow chart of a method for checking duplicate bids based on a large model, provided as an exemplary embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a large-model-based bid duplication checking system provided by an exemplary embodiment of the present invention;

[0047] Figure 3 A schematic diagram of a computer device provided for an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0049] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0050] As described in the background, existing techniques for checking for duplicate bids rely primarily on traditional text matching algorithms (such as cosine similarity and TF-IDF) or shallow semantic analysis models. These methods suffer from core issues such as insufficient semantic understanding, poor domain adaptability, and inefficient processing of massive amounts of data. For example, traditional methods struggle to identify content with identical semantics but different expressions (such as synonym substitution and sentence reorganization), and their ability to analyze the contextual associations of professional terminology and industry standards is weak, resulting in high rates of missed detection and limited cross-scenario applicability. Furthermore, when processing large amounts of bid data, traditional algorithms struggle to achieve real-time response due to their high computational complexity. In view of this, this implementation proposes a large-model-based bid duplication checking method, which performs structured parsing and multimodal feature extraction on the input bid to generate feature representations of semantic blocks and non-text elements; uses a dynamic context-aware large language model to perform deep semantic encoding on text blocks, and combines hierarchical position encoding to retain the logical relevance of long texts; achieves efficient matching of massive semantic vectors through a hybrid retrieval architecture, and optimizes computing efficiency by combining a distributed computing framework with hardware acceleration instructions; finally, reconstructs structured features of non-text content such as tables and charts, realizes cross-modal semantic association analysis, and outputs comprehensive duplication checking results.

[0051] More specifically, Figure 1 As shown, the following process is included:

[0052] S101: Preprocess the original bid document to obtain text blocks with hierarchical labels and non-text multimodal features;

[0053] S102: Based on a dynamic context-aware large language model, semantically encode the text block to obtain a semantic vector corresponding to the text block;

[0054] S103: constructing an industry-specific hierarchical index based on the semantic vector, dynamically adjusting the hierarchical structure of the index, and determining semantic similarity according to the hierarchical structure of the index;

[0055] S104: Based on the semantic vector of the original bid, the multimodal features of the non-text, and the multimodal features of the candidate documents, a cross-modal attention calculation is performed to determine the weight coefficients of each modality, and the weight of each modal feature in the candidate document is dynamically allocated according to the proportion of each modal feature in the original bid;

[0056] S105: Determine the non-text similarity based on the weighting coefficient, the weight, and the specific matching degree of various modalities, and determine the global similarity between the original bid and the candidate document based on the weighted sum of the semantic similarity and the non-text similarity.

[0057] More specifically, step S101 of this implementation includes:

[0058] Through structured parsing and multimodal feature extraction technology, the original bidding documents are converted into standardized semantic units and non-text feature representations, providing a basis for subsequent deep semantic analysis. The core tasks of preprocessing include bidding hierarchical splitting, non-text element separation and multimodal semantic reconstruction.

[0059] Structured parsing uses a combination of a rule engine and an adaptive algorithm to identify multi-level titles and divide the tender documents into semantic blocks. It matches titles based on regular expressions (such as "1.1.1 Technical Parameters") and calculates title weights based on a combination of font size, bold attributes, and position information. The weight is dynamically determined by a linear combination of the normalized font size value, bold mark, and position weight. When the weight exceeds the preset threshold, it is marked as a title and a hierarchical index tree is generated, thereby splitting the tender document into independent semantic blocks such as technical solutions and quotations. To address the problem of broken tables across multiple pages, an association algorithm based on merged cell features and header consistency is designed: by detecting the repetitiveness of header keywords and the continuity of numerical distribution, cross-page tables are automatically spliced ​​to ensure logical integrity.

[0060] Non-text content separation is achieved through a hybrid OCR engine that combines a convolutional neural network (CNN) with a hidden Markov model (HMM) to improve recognition accuracy for scanned documents. A contextual semantic error correction mechanism is also used to correct easily confused characters. For example, for ambiguous characters in the OCR output, candidate character selection is dynamically optimized based on the probability distribution of adjacent words, ensuring accurate recognition of characters such as "0" and "O."

[0061] More specifically, differentiated feature extraction schemes are designed for tables, charts, and formulas. Table parsing converts table content into a logical tree structure, extracts row and column relationships, numerical distribution features, and merges cell ranges to generate a global feature vector. The header row is given a higher weight, and a dense vector reflecting the table content is constructed by weighted aggregation of the semantic role (such as "data item" and "unit") and statistical features (mean, variance) of each cell. Chart processing locates coordinate axes, legends, and data points through the target detection model, extracts numerical range vectors, and constructs topological relationships of data points based on graph neural networks (GNN). The adjacency matrix weights are jointly determined by the spatial distance and numerical difference of the data points, capturing the structural similarity of the chart. Formula encoding converts LaTeX or image formulas into a symbolic tree structure, generates serialized expressions through breadth-first traversal, and uses locality-sensitive hashing (LSH) to generate fuzzy fingerprints, supporting fuzzy matching of variable replacement and typesetting fine-tuning.

[0062] After preprocessing, the output is text blocks with hierarchical labels, multimodal feature sets (logical tree, topological relationship matrix, hash fingerprint) and associated metadata (such as "a certain figure corresponds to a certain section"), ensuring that downstream strategies can integrate the joint semantic information of text and non-text.

[0063] In step S102 of this implementation, more specifically, the following steps are included:

[0064] Based on a large language model with dynamic context awareness, deep semantic encoding is performed on pre-processed structured text blocks, and cross-industry semantic representation optimization is achieved through domain adaptation technology. By working together through hierarchical position encoding and pluggable parameter adapters, the problems of semantic fragmentation in long texts and generalization of industry terms are solved.

[0065] For the Text blocks (Associated hierarchical labels such as "2.3.1 Technical Solution"), which can be jointly represented by word embedding, paragraph type embedding and hierarchical position encoding. Hierarchical position encoding dynamically adjusts the encoding frequency according to the title level (chapter, section, paragraph), and captures long-range dependencies through multi-level sine / cosine functions. Specifically, Level position encoding The calculation formula is:

[0066] (1);

[0067] in, is the vector dimension, The maximum number of title levels. is the frequency adjustment factor. By superimposing the encoding results at different levels, the model can distinguish the global logic of chapters and the local details of paragraphs in the technical solution. For example, the hierarchical relationship between "1.1 Overall Design" and "1.1.2 Core Parameters" can be mapped into differentiated positional features.

[0068] The domain adaptation unit uses lightweight parameter adapters to inject industry knowledge. These adapters are superimposed on the base LLM model as residuals. Their core structure is a dimensionality reduction-activation-increase bottleneck, significantly reducing computational overhead. During training, only the adapter parameters are updated, while the base model parameters remain frozen, ensuring efficient industry migration. Industry-specific prompt templates are dynamically added before text input, guiding the model to focus on key industry terms and regulatory constraints.

[0069] The semantic vector set obtained by the present invention It integrates multi-granular semantic information: high-level encoding emphasizes the consistency of the technical solution framework, while low-level encoding captures variations in implementation details. Furthermore, the multimodal association metadata provided by the semantic vector association preprocessing process (e.g., "a certain image corresponds to a certain paragraph") provides semantic alignment anchors for subsequent cross-modal joint duplication checking. For example, the "equipment installation process" described in the technical solution will be cross-modally matched with the "equipment list table" in the quotation through association tags, avoiding missed detections due to fragmented content.

[0070] In step S103 of this implementation, more specifically, the following steps are included:

[0071] Through hybrid retrieval architecture and distributed optimization strategy, efficient semantic matching and multi-modal joint duplicate checking of 100 million-level bidding documents are achieved. Specifically: first, a hierarchical index is constructed based on the semantic vector set output by the semantic processing process, and the improved HNSW (Hierarchical Navigable Small World) algorithm is used to dynamically adjust the hierarchical structure of the index. Sub-libraries, number of index levels According to the data scale The adaptive calculation is:

[0072] (2);

[0073] in, is the hierarchical sparsity factor (default value is 500), and the number of hierarchies dynamically expands as the number of bids increases. High-level indexes store coarse-grained feature vectors to quickly narrow the search scope, while low-level indexes retain fine-grained vectors to improve matching accuracy. A greedy search algorithm navigates between hierarchies, significantly reducing retrieval latency for billions of data points. During index construction, each industry sub-database is stored independently, ensuring that cross-industry queries only require loading the target industry index, reducing inefficient computation.

[0074] The retrieval process is divided into two stages: the coarse screening stage (i.e. the first screening stage) quickly screens the candidate document set based on the inverted index and metadata (such as industry classification, bid type), and extracts the original bid Keyword collection (such as the core terms in the technical solution), calculate the document relevance score by TF-IDF weight, and filter out the Top-M candidate document set , the coarse screening fraction is calculated as:

[0075] (3);

[0076] in, is the indicator function, the screening score is above the threshold Documentation, For keywords, is the word frequency, is the inverse document frequency.

[0077] The candidate set is sorted in the refinement stage (i.e. the second screening stage). Documents in , calculate its semantic vector (From the document ) and the query vector The similarity of the original bid documents is calculated by integrating multimodal features for joint ranking. The semantic similarity (i.e., text similarity) adopts a weighted combination of cosine similarity and Manhattan distance:

[0078] (4);

[0079] in Adjust the contribution of semantic matching, is the distance scaling factor. Through cross-validation optimization, this design takes into account both the directional similarity and absolute distance difference of vectors. It is particularly suitable for scenarios where the technical solutions have different expressions but similar semantics.

[0080] To meet the real-time requirements of massive data volumes, this paper optimizes the computational process using the MapReduce distributed framework. The bid database is sharded by industry and stored on different compute nodes. Rough screening tasks are distributed to each node for parallel execution. During the fine sorting phase, SIMD instructions are used to parallelize vector operations, accelerating cosine similarity and distance calculations. A dynamic resource scheduler allocates GPU / CPU resources in real time based on query load, prioritizing high-priority tasks and automatically scaling compute nodes in high-concurrency scenarios.

[0081] In this implementation, step S104 and step S105 specifically include:

[0082] Through the cross-modal attention mechanism and dynamic feature fusion strategy, deep semantic correlation analysis of text and non-text elements is achieved, solving the problem of missed detection of text and image separation content by traditional methods. Based on the multimodal feature set extracted by preprocessing (tabular logic tree, chart topology matrix, hash fingerprint of formula) and the semantic vector generated by semantic processing, a unified cross-modal semantic space is constructed, and the global similarity is quantified through adaptive weight allocation.

[0083] In the cross-modal attention mechanism, the query-key-value (QKV) model is used to realize multimodal feature interaction. Text chunking and its associated non-text features (such as tables ), calculate its difference with the candidate bid Cross-modal attention score of:

[0084] (5);

[0085] in, Original bid No. The semantic vector corresponding to the text block, is the candidate document multimodal fusion function, Representative candidate documents No. The semantic vector corresponding to the text block, Representative candidate documents No. The non-text multimodal features corresponding to the text blocks, are all trainable projection matrices, is the vector dimension, is the candidate document multimodal fusion function. Fusion function The text vector and non-text features are mapped to the same space, and the contribution of each modality is dynamically weighted through an independent encoding network, so that the text description and the associated non-text elements (such as "equipment parameters in the technical proposal" and "model list in the quotation form") are automatically aligned.

[0086] Dynamic feature fusion generates comprehensive similarity based on attention scores for candidate documents , whose global similarity is determined by the text similarity Similarity with non-text Weighted calculation:

[0087] (6);

[0088] (7);

[0089] in, is a dynamic weight coefficient, which is adaptively adjusted by the modal distribution of the query content. Non-text similarity is further decomposed into the contribution of tables, charts, and formulas, and the weight Dynamically allocates weight based on the proportion of non-text elements in the original bid. For example, if the original bid contains multiple tables, the weight of table similarity is significantly increased to ensure adaptability to business scenarios. For the A special matching similarity of non-text modalities. Table duplication check is based on logical tree feature vector The similarity is quantified by combining the intersection over union (IoU) of row and column relationships with differences in numerical distribution (such as KL divergence); graph duplication checking uses topological features extracted by graph neural networks to calculate the structural similarity of the adjacency matrix and the similarity of node feature matching; formula duplication checking uses local sensitive hash fingerprints and symbol tree edit distance to quantify phase velocity, and supports fuzzy matching of variable replacement and typesetting fine-tuning. For the cross-modal attention pair The weight coefficient of the non-text modality is obtained by normalizing the weight distribution output by the attention mechanism. For example, when the attention score of the query text and the candidate table is high, will be close to 1, making the table similarity The proportion in non-text fusion has increased, All non-text similarity results corresponding to the weighted coefficients in the representative table are weighted and fused together with the text similarity to generate the final duplicate checking score.

[0090] Training and optimization utilize a contrastive learning framework, using a triplet loss function to drive the model's learning of consistent semantic representations of images and text. For example, through extensive training, the model can automatically associate the textual description of a "construction flow chart" in a technical proposal with the topological features of the corresponding diagram, avoiding missed detections often associated with traditional methods that separate images and text.

[0091] Figure 2 A multimodal bidding document duplication checking system based on a large model is shown, comprising:

[0092] The preprocessing unit 201 is configured to: preprocess the original bid document to obtain text blocks with hierarchical labels and non-text multimodal features;

[0093] The semantic processing unit 202 is configured to: perform semantic encoding on the text block based on a dynamic context-aware large language model to obtain a semantic vector corresponding to the text block;

[0094] The retrieval calculation unit 203 is configured to: construct an industry-specific hierarchical index based on the semantic vector, dynamically adjust the hierarchical structure of the index, and determine the semantic similarity according to the hierarchical structure of the index;

[0095] The multimodal calculation unit 204 is configured to: perform cross-modal attention calculation based on the semantic vector of the original bid, the multimodal features of the non-text, and the multimodal features of the candidate documents to determine the weight coefficients of each modality, and dynamically assign weights to each modal feature in the candidate documents based on the proportion of each modal feature in the original bid;

[0096] The joint duplicate checking unit 205 is configured to: determine the non-text similarity based on the weighting coefficient, the weight and the special matching degree of various modalities, and determine the global similarity between the original bid and the candidate document based on the weighted sum of the semantic similarity and the non-text similarity.

[0097] It is understandable that each of the above-mentioned units can be separately or completely combined into one or several other units to form a whole, or one (or more) of the units can be further divided into multiple functionally smaller units to form a whole, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present invention, the system can also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0098] According to another embodiment of the present invention, the system described in this embodiment can be constructed by running a computer program (including program code) capable of executing the steps involved in the corresponding method described in Example 1 on a general-purpose computing device such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0099] Figure 3 A computer device is shown, which includes a processor 301, a communication interface 302, and a computer-readable storage medium 303. The processor 301, the communication interface 302, and the computer-readable storage medium 303 may be connected via a bus or other means.

[0100] Among them, the communication interface 302 is used to receive and send data, the computer-readable storage medium 303 can be stored in the memory of the electronic device, the computer-readable storage medium 303 is used to store computer programs, the computer programs include program instructions, and the processor 301 is used to execute the program instructions stored in the computer-readable storage medium 303.

[0101] The processor 301 is the computing core and control core of the electronic device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement corresponding method processes or corresponding functions.

[0102] The processor 301 is configured to perform the following process:

[0103] Preprocess the original bid document to obtain text segments with hierarchical labels and non-text multimodal features;

[0104] Based on a dynamic context-aware large language model, semantic encoding is performed on the text blocks to obtain semantic vectors corresponding to the text blocks;

[0105] Building an industry-specific hierarchical index based on the semantic vector, dynamically adjusting the hierarchical structure of the index, and determining semantic similarity based on the hierarchical structure of the index;

[0106] Based on the semantic vector of the original bid, the multimodal features of non-text, and the multimodal features of the candidate documents, a cross-modal attention calculation is performed to determine the weighting coefficients of each modality. The weights of each modal feature in the candidate document are dynamically allocated according to the proportion of each modal feature in the original bid.

[0107] The non-text similarity is determined according to the weighting coefficient, the weight and the specific matching degree of various modalities, and the global similarity between the original bid and the candidate document is determined according to the weighted sum of the semantic similarity and the non-text similarity.

[0108] The present invention also provides a computer-readable storage medium, which is a memory device in an electronic device for storing programs and data. It is understood that the computer-readable storage medium herein may include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The computer-readable storage medium provides storage space that stores the processing system of the electronic device.

[0109] Furthermore, the storage space also stores one or more instructions suitable for being loaded and executed by the processor. These instructions may be one or more computer programs (including program code). It should be noted that the computer-readable storage medium herein may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device; alternatively, it may be at least one computer-readable storage medium located remotely from the processor.

[0110] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to implement the following process:

[0111] Preprocess the original bid document to obtain text segments with hierarchical labels and non-text multimodal features;

[0112] Based on a dynamic context-aware large language model, semantic encoding is performed on the text blocks to obtain semantic vectors corresponding to the text blocks;

[0113] Building an industry-specific hierarchical index based on the semantic vector, dynamically adjusting the hierarchical structure of the index, and determining semantic similarity based on the hierarchical structure of the index;

[0114] Based on the semantic vector of the original bid, the multimodal features of non-text, and the multimodal features of the candidate documents, a cross-modal attention calculation is performed to determine the weighting coefficients of each modality. The weights of each modal feature in the candidate document are dynamically allocated according to the proportion of each modal feature in the original bid.

[0115] The non-text similarity is determined according to the weighting coefficient, the weight and the specific matching degree of various modalities, and the global similarity between the original bid and the candidate document is determined according to the weighted sum of the semantic similarity and the non-text similarity.

[0116] The present invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the following process:

[0117] Preprocess the original bid document to obtain text segments with hierarchical labels and non-text multimodal features;

[0118] Based on a dynamic context-aware large language model, semantic encoding is performed on the text blocks to obtain semantic vectors corresponding to the text blocks;

[0119] Building an industry-specific hierarchical index based on the semantic vector, dynamically adjusting the hierarchical structure of the index, and determining semantic similarity based on the hierarchical structure of the index;

[0120] Based on the semantic vector of the original bid, the multimodal features of non-text, and the multimodal features of the candidate documents, a cross-modal attention calculation is performed to determine the weighting coefficients of each modality. The weights of each modal feature in the candidate document are dynamically allocated according to the proportion of each modal feature in the original bid.

[0121] The non-text similarity is determined according to the weighting coefficient, the weight and the specific matching degree of various modalities, and the global similarity between the original bid and the candidate document is determined according to the weighted sum of the semantic similarity and the non-text similarity.

[0122] Those skilled in the art will appreciate that the units and algorithmic steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0123] The above embodiments can be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. A computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, they fully or partially produce the processes or functions according to the embodiments of the present invention. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data processing device such as a server or data center that integrates one or more available media. Available media can include magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0124] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A multimodal bid duplication checking method based on a large model, characterized in that: The following processes are included: Preprocess the original bid document to obtain text segments with hierarchical labels and non-text multimodal features; Based on a dynamic context-aware large language model, semantic encoding is performed on the text blocks to obtain semantic vectors corresponding to the text blocks; Building an industry-specific hierarchical index based on the semantic vector, dynamically adjusting the hierarchical structure of the index, and determining semantic similarity based on the hierarchical structure of the index; Based on the semantic vector of the original bid, the multimodal features of non-text, and the multimodal features of the candidate documents, a cross-modal attention calculation is performed to determine the weighting coefficients of each modality. The weights of each modal feature in the candidate document are dynamically allocated according to the proportion of each modal feature in the original bid. Determining non-text similarity based on the weighting coefficient, the weight, and the specific matching degree of various modalities, and determining the global similarity between the original bid and the candidate document based on the weighted sum of the semantic similarity and the non-text similarity; For any text block, it is jointly represented by word embedding, paragraph type embedding and hierarchical position encoding. Hierarchical position encoding dynamically adjusts the encoding frequency according to the title level. Before the joint representation results are input into the large language model, a domain adaptation unit is used to dynamically splice industry-specific prompt templates.

2. The multimodal duplication checking method for bid documents based on a large model as claimed in claim 1, characterized in that: Preprocessing the original bid to obtain multimodal associated metadata, and associating the semantic vector with the multimodal associated metadata; Extract the keyword set from the original bid document and determine multiple candidate documents through correlation calculation. This process is distributed to multiple computing nodes for parallel execution. The semantic similarity between the semantic vector of each candidate document and the query vector of the original bid is calculated. This process is performed by parallel vector operation using the SIMD instruction set.

3. The multimodal duplication checking method for bid documents based on a large model as claimed in claim 1, characterized in that: Preprocess the original bid document to obtain text blocks with hierarchical labels, including: Based on regular expression matching of titles, the weight of the title is calculated by integrating font size, bold attributes and position information. The weight is dynamically determined by a linear combination of the normalized font size value, bold flag and position weight. When the weight exceeds the preset threshold, it is marked as a title and a hierarchical index tree is generated to split the original bid into independent text blocks with hierarchical labels.

4. The multimodal duplication checking method for bid documents based on a large model according to any one of claims 1 to 3, characterized in that: The original bid document is preprocessed to obtain multimodal features, including: converting the table content into a logical tree structure; locating the coordinate axes, legends and data points in the chart through the target detection model, extracting the numerical range vector, and constructing a topological relationship matrix of the data points based on the graph neural network; converting the formula or image formula obtained by the formula editor into a symbolic tree structure, generating a serialized expression through breadth-first traversal, and using local sensitive hashing to generate a hash fingerprint; the logical tree structure, the topological relationship matrix and the hash fingerprint constitute the multimodal features.

5. The multimodal duplication checking method for bid documents based on a large model according to any one of claims 1 to 3, characterized in that: Dynamically adjust the index hierarchy, including: Sub-libraries, number of index levels According to the data scale Adaptive Computing: ,in, is the level sparsity factor.

6. The multimodal duplication checking method for bid documents based on a large model according to any one of claims 1 to 3, characterized in that: Perform cross-modal attention calculations, including: ,in, Original bid No. The semantic vector corresponding to the text block, is the candidate document multimodal fusion function, Representative candidate documents No. The semantic vector corresponding to the text block, Representative candidate documents No. The non-text multimodal features corresponding to the text blocks, are all trainable projection matrices, is the vector dimension.

7. A multi-modal bidding document duplication checking system based on a large model, characterized by: The multimodal duplication checking method for bid documents based on a large model according to any one of claims 1 to 6 comprises: The preprocessing unit is configured to: preprocess the original bid document to obtain text blocks with hierarchical labels and non-text multimodal features; A semantic processing unit is configured to: perform semantic encoding on the text block based on a dynamic context-aware large language model to obtain a semantic vector corresponding to the text block; A retrieval calculation unit is configured to: construct an industry-specific hierarchical index based on the semantic vector, dynamically adjust the hierarchical structure of the index, and determine semantic similarity based on the hierarchical structure of the index; The multimodal computing unit is configured to: perform cross-modal attention calculation based on the semantic vector of the original bid, the multimodal features of the non-text, and the multimodal features of the candidate documents to determine the weight coefficients of each modality, and dynamically allocate the weights of each modal feature in the candidate document based on the proportion of each modal feature in the original bid; The joint duplicate checking unit is configured to: determine the non-text similarity based on the weighting coefficient, the weight and the special matching degree of various modalities, and determine the global similarity between the original bid and the candidate document based on the weighted sum of the semantic similarity and the non-text similarity.

8. A computer device, characterized in that: include: a processor and a computer-readable storage medium; a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the multimodal duplication checking method for bids based on a large model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing the large model-based multimodal duplication checking method for bids as described in any one of claims 1 to 6.

Images