Bid invitation purchasing bid clearing method based on multi-modal information fusion

Abstract

The invention discloses a bid invitation purchasing bid clearing method based on multi-modal information fusion. The method comprises the following steps: firstly, carrying out data preprocessing on a bid invitation file and a bidding file, unifying formats and extracting text, image and table contents; then, extracting a text feature vector, an image feature vector and a table feature vector by using natural language processing, a convolutional neural network and a statistical analysis technology respectively; then, through a fusion model based on an attention mechanism or a multi-modal Transform, the multi-modal features are fused into a comprehensive feature vector; and finally, according to a preset multi-dimensional bidding clearing rule set covering qualification, technology, commerce and reputation, carrying out automatic matching and logical judgment on the comprehensive feature vector, realizing compliance, integrity and accuracy evaluation of the bidding document, and generating a detailed bidding clearing report containing abnormal annotations. According to the method, automation and intellectualization of the bidding cleaning process are realized, and bidding cleaning efficiency and objectivity and comprehensiveness of an evaluation result are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of bidding and tendering technology, and in particular relates to a bidding and procurement evaluation method based on multimodal information fusion. Background Technology

[0002] In the bidding and procurement process, bid evaluation is a crucial step in ensuring fairness, impartiality, and transparency. Traditional bid evaluation methods primarily rely on manual review of each bid document, which is not only inefficient but also susceptible to subjective influences, making it difficult to guarantee the accuracy and objectivity of the evaluation results. While the development of information technology has led to some bid evaluation methods based on electronic document processing, most focus only on textual information, failing to adequately utilize other modal information in the bid documents, such as images, tables, and formulas, thus hindering a comprehensive and accurate assessment of the compliance and completeness of the bids. Therefore, a bidding and procurement bid evaluation method based on multimodal information fusion is needed to address these issues. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a bidding and procurement bid clearing method based on multimodal information fusion, which aims to solve the problems of low bid clearing efficiency, poor accuracy and insufficient utilization of multimodal information in the existing technology. By integrating information in various forms such as text, tables and pictures, it realizes intelligent bid clearing based on bidding documents and tender documents, thereby improving bid clearing efficiency and accuracy.

[0004] To achieve the above-mentioned technical effects, the technical solution adopted by the present invention is as follows: A bidding and procurement evaluation method based on multimodal information fusion includes the following steps: S1, Data Preprocessing: Obtain the tender documents and all bidders' bid documents, convert the bid documents into electronic documents in a unified format; use optical character recognition technology to extract the text content from the electronic documents, and preprocess the images and tables in the electronic documents respectively; S2, Multimodal Feature Extraction: Feature extraction is performed on the preprocessed text content, images, and tables to obtain text feature vectors, image feature vectors, and table feature vectors. Among them, text feature extraction adopts the term frequency-inverse document frequency method or the semantic feature extraction method based on a pre-trained language model; image feature extraction adopts a pre-trained convolutional neural network model; table feature extraction includes identifying the field names and data types in the table, and performing statistical analysis on numerical fields, including minimum value, maximum value, and quantiles. S3, Multimodal Information Fusion: Text feature vectors, image feature vectors, and table feature vectors are input into a multimodal fusion neural network model for fusion to obtain a comprehensive feature vector representing the content of the tender document; the multimodal fusion neural network model adopts a fusion method based on an attention mechanism or a multimodal Transformer model; S4, Bid Clearing Rule Matching and Evaluation: Based on the preset bid clearing rule set, the comprehensive feature vector is matched and logically judged; the bid clearing rule set includes rules under the dimensions of qualification requirements, technical review, commercial review, and reputation and risk; based on the matching and judgment results, the compliance, completeness and accuracy of the bid documents are evaluated, and the bid documents that meet the requirements are selected. S5, Result Generation and Output: Generate a bid clearing report, which includes at least the compliance assessment results, completeness assessment results, accuracy assessment results, and annotations and prompts for abnormal information for each bid document.

[0005] Preferably, in step S1, the data preprocessing further includes: Convert tender documents of different formats to PDF format; The extracted text content is cleaned and standardized. Preprocessing operations, including denoising, binarization, and tilt correction, are performed on the image. Use a table recognition algorithm to convert tabular information into structured data.

[0006] Preferably, in step S2, the semantic feature extraction method based on the pre-trained language model adopts the BERT model.

[0007] Preferably, in step S2, the pre-trained convolutional neural network model is a ResNet model.

[0008] Preferably, in step S3, the attention-based fusion method specifically includes: Calculate the similarity between each pair of the text feature vector, image feature vector, and table feature vector, and assign attention weights to each feature vector based on the similarity. The comprehensive feature vector is obtained by weighting and summing the corresponding feature vectors according to the attention weights.

[0009] Preferably, in step S4, the rule parameters for qualification requirements include enterprise qualification level, project experience, and personnel certificates; the rule parameters for technical review include technological innovation, feasibility of construction technology, and reasonableness of construction period; the rule parameters for commercial review include the total quotation range, reasonableness of itemized quotations, and payment terms; and the rule parameters for reputation and risk include enterprise credit rating, litigation records, and performance capability.

[0010] Preferably, in step S4, the compliance assessment is used to determine whether the content of the tender document complies with the mandatory requirements of the tender document; the completeness assessment is used to determine whether the tender document contains all the required content items; and the accuracy assessment is used to determine whether the data in the tender document is accurate and free of logical contradictions.

[0011] Preferably, step S4 further includes: sorting the qualified tender documents according to a preset sorting rule, wherein the sorting rule includes sorting by total price from low to high or by technical solution score from high to low.

[0012] Preferably, in step S5, the abnormal information includes missing required documents, bid content that does not conform to the rules, logically contradictory data, and quotations below cost.

[0013] Preferably, a multimodal Transformer fusion method combined with an attention mechanism is used to fuse the three feature vectors together: An attention mechanism is used to dynamically fuse data from different modalities. By calculating the correlation between data from different modalities, different weights are assigned to each modality; the formula is as follows: Calculate attention weights: ; in It is attention weight. It is a similarity function. It is an eigenvector; Calculate the attention result: ; in It is a result of attention; The multi-head self-attention mechanism of the multimodal Transformer model is used to process three feature vectors simultaneously by establishing attentional connections between different modalities.

[0014] The beneficial effects of this invention are as follows: 1. This invention fundamentally changes the traditional work mode that relies on manual page-by-page and item-by-item review by introducing a systematic data preprocessing and multimodal feature automatic extraction process. In the data preprocessing stage, automated format conversion and OCR text extraction technologies enable rapid and standardized processing of massive amounts of multi-format bid documents. In the feature extraction and fusion stage, pre-trained deep learning models and statistical methods are used to extract key features from text, images, and tables in parallel and efficiently, and then integrated and analyzed through a multimodal fusion model. The implementation of this series of automated technical solutions frees bid review personnel from tedious and repetitive preliminary screening and data collection work, allowing them to focus on higher-value complex judgments and decision review, thereby significantly shortening the overall bid review cycle and achieving a qualitative leap in bid review efficiency.

[0015] 2. The core of this invention lies in its adoption of a multimodal information fusion technology, effectively solving the problem of one-sided evaluation caused by existing technologies that only focus on text or structured data. By extracting and fusing semantic features of text, visual features such as graphics and seals in images, and structured data features in tables, the system can construct a more complete and three-dimensional digital representation of the tender document content. Based on this, an attention-based fusion model or multimodal Transformer is used to intelligently weigh the importance of different modalities of information, ensuring that no key information is omitted. Finally, the fused comprehensive features are precisely matched and logically judged with objective, pre-set bid evaluation rules, strictly avoiding inconsistencies in evaluation standards caused by subjective emotions, experience differences, or negligence, making the bid evaluation conclusion more reliable and fair, and significantly improving the quality control capabilities of bidding and procurement.

[0016] 3. This invention establishes a clear and transparent mechanism for recording and outputting the bid evaluation process through a rule-driven automated evaluation and structured report generation scheme. The system not only outputs simple "pass / fail" conclusions but also generates detailed bid evaluation reports, specifically showcasing the quantitative or qualitative evaluation results of each bid document in terms of compliance, completeness, and accuracy, and accurately locating and alerting to all identified anomalies. This provides valuable, data-driven decision-making support for the bidding party's subsequent manual review, dispute resolution, and process auditing. Simultaneously, the entire processing flow, rule application, and judgment logic are recorded by the system, enhancing the traceability and transparency of the bid evaluation activities, helping to further standardize the bidding and procurement process, prevent risks, and demonstrating the profound empowerment of management processes by intelligent technology. Attached Figure Description

[0017] Figure 1 This is a block diagram of the method of the present invention; Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] Example 1: like Figure 1 As shown, a bidding and procurement evaluation method based on multimodal information fusion includes the following steps: S1, Data Preprocessing: Obtain the tender documents and all bidders' bid documents, convert the bid documents into electronic documents in a unified format; use optical character recognition technology to extract the text content from the electronic documents, and preprocess the images and tables in the electronic documents respectively; S2, Multimodal Feature Extraction: Feature extraction is performed on the preprocessed text content, images, and tables to obtain text feature vectors, image feature vectors, and table feature vectors. Among them, text feature extraction adopts the term frequency-inverse document frequency method or the semantic feature extraction method based on a pre-trained language model; image feature extraction adopts a pre-trained convolutional neural network model; table feature extraction includes identifying the field names and data types in the table, and performing statistical analysis on numerical fields, including minimum value, maximum value, and quantiles. S3, Multimodal Information Fusion: Text feature vectors, image feature vectors, and table feature vectors are input into a multimodal fusion neural network model for fusion to obtain a comprehensive feature vector representing the content of the tender document; the multimodal fusion neural network model adopts a fusion method based on an attention mechanism or a multimodal Transformer model; S4, Bid Clearing Rule Matching and Evaluation: Based on the preset bid clearing rule set, the comprehensive feature vector is matched and logically judged; the bid clearing rule set includes rules under the dimensions of qualification requirements, technical review, commercial review, and reputation and risk; based on the matching and judgment results, the compliance, completeness and accuracy of the bid documents are evaluated, and the bid documents that meet the requirements are selected. S5, Result Generation and Output: Generate a bid clearing report, which includes at least the compliance assessment results, completeness assessment results, accuracy assessment results, and annotations and prompts for abnormal information for each bid document.

[0019] Preferably, in step S1, the data preprocessing further includes: Convert tender documents of different formats to PDF format; The extracted text content is cleaned and standardized. Preprocessing operations, including denoising, binarization, and tilt correction, are performed on the image. Use a table recognition algorithm to convert tabular information into structured data.

[0020] Preferably, in step S2, the semantic feature extraction method based on the pre-trained language model adopts the BERT model.

[0021] Preferably, in step S2, the pre-trained convolutional neural network model is a ResNet model.

[0022] Preferably, in step S3, the attention-based fusion method specifically includes: Calculate the similarity between each pair of the text feature vector, image feature vector, and table feature vector, and assign attention weights to each feature vector based on the similarity. The comprehensive feature vector is obtained by weighting and summing the corresponding feature vectors according to the attention weights.

[0023] Preferably, in step S4, the rule parameters for qualification requirements include enterprise qualification level, project experience, and personnel certificates; the rule parameters for technical review include technological innovation, feasibility of construction technology, and reasonableness of construction period; the rule parameters for commercial review include the total quotation range, reasonableness of itemized quotations, and payment terms; and the rule parameters for reputation and risk include enterprise credit rating, litigation records, and performance capability.

[0024] Preferably, in step S4, the compliance assessment is used to determine whether the content of the tender document complies with the mandatory requirements of the tender document; the completeness assessment is used to determine whether the tender document contains all the required content items; and the accuracy assessment is used to determine whether the data in the tender document is accurate and free of logical contradictions.

[0025] Preferably, step S4 further includes: sorting the qualified tender documents according to a preset sorting rule, wherein the sorting rule includes sorting by total price from low to high or by technical solution score from high to low.

[0026] Preferably, in step S5, the abnormal information includes missing required documents, bid content that does not conform to the rules, logically contradictory data, and quotations below cost.

[0027] Preferably, a multimodal Transformer fusion method combined with an attention mechanism is used to fuse the three feature vectors together: An attention mechanism is used to dynamically fuse data from different modalities. By calculating the correlation between data from different modalities, different weights are assigned to each modality; the formula is as follows: Calculate attention weights: ; in It is attention weight. It is a similarity function. It is an eigenvector; Calculate the attention result: ; in It is a result of attention; The multi-head self-attention mechanism of the multimodal Transformer model is used to process three feature vectors simultaneously by establishing attentional connections between different modalities.

[0028] Example 2: This embodiment provides an implementation method for applying this method to a specific bid clearing process, as follows: Data preprocessing: Collect electronic tender documents and all bids submitted by bidders, and convert them into PDF format. Use OCR technology to extract text content and preprocess images.

[0029] Feature extraction: Feature vectors were extracted from text, images, and tables respectively. Text features were extracted using the TF-IDF method, image features using a pre-trained ResNet model, and table features were obtained through statistical methods. Specifically, during table feature extraction, field names and data types (numerical / categorical / time, etc.) were obtained using a data dictionary or automatic detection tools (such as Python's pandas.dtypes), and the minimum, maximum, and quantile values ​​of numeric fields were identified.

[0030] Example: Field Age: Numeric type (continuous); Field Gender: Type (Discrete); Minimum value: min(x1, x2, ..., x_n); Maximum value: max(x1, x2, ..., x_n); Quantiles (e.g., quartiles): Q1 = 25th percentile, Q3 = 75th percentile Multimodal information fusion: Three feature vectors are input into a pre-trained multimodal fusion neural network to obtain a comprehensive feature vector; in the multimodal fusion neural network model, three different types of feature vectors are received through the input layer, and the three feature vectors are fused together using the multimodal Transformer fusion method combined with an attention mechanism.

[0031] Attention mechanism: This mechanism dynamically fuses data from different modalities by calculating the correlations between them and assigning different weights to each modality. The formula is as follows: Calculate attention weights; Application of Bid Selection Rules. Based on the specific needs of the project, bid selection rules are set for qualifications, technology, and commercial aspects. The comprehensive feature vector is matched against these rules to filter out qualified bids. Table 1 below shows the classification and parameter examples of the bid selection rules: Table 1: Classification and Parameters of Bill Clearing Rules;

[0032] Comprehensive feature vector extraction: Transform the tender documents into structured data and extract key features.

[0033] Example: Use Natural Language Processing (NLP) techniques to extract keywords from a technical solution and calculate feature vectors.

[0034] Rule matching: Logical judgment is made on the feature vectors according to the clearing rules.

[0035] Example: If a bidding company's qualification level is Level 2, but the rules require Level 1, then it is determined that the qualification requirements are not met.

[0036] Screening and sorting: Remove bids that do not meet the bid clearing rules; sort the bids that meet the rules.

[0037] Results and feedback: Generate a bid evaluation report, including assessment results of the compliance, completeness, and accuracy of the bidding documents; based on the bidding documents, mark and highlight any anomalies found for manual review. Among these: Compliance assessment: Whether the tender documents comply with the requirements of the tender documents; for example: Supplier A: Compliant, Class A qualification for general contracting of engineering construction, meets the requirements; Supplier B: Non-compliant, its qualification level is Class B for general contracting of engineering construction, which does not meet the requirements; Completeness assessment: Does the document contain all necessary information such as qualification certificates and technical solutions? Supplier A: Completeness score 0.9, missing 1 required item; Supplier B: Completeness score 0.8, missing 2 required items; Accuracy assessment: Whether the data such as the quotation and construction period are accurate and logically consistent; for example: Supplier A: Accuracy score 0.95, outlier rate 5%; Supplier B: Accuracy score 0.80, outlier rate 20%; Anomaly information labeling: Content that does not conform to the rules is labeled and potential risks are highlighted; for example: Supplier A: The company's business license is missing and has been marked as abnormal. Bids priced below cost are marked as abnormal. The technical solution does not meet the starred requirements and is marked as abnormal.

[0038] The specific implementation steps include: Step 1: Data Preprocessing Collecting tender documents: Collect electronic tender documents from each bidder, including but not limited to PDF, Word, Excel and other formats.

[0039] Format conversion: Convert all tender documents into a unified PDF format for easier subsequent processing.

[0040] Text extraction: Use OCR technology to extract the text content from the tender documents, and then clean and standardize it.

[0041] Image processing: Preprocessing images in the tender documents, including noise reduction, binarization, and tilt correction, to improve image quality.

[0042] Table recognition: Use table recognition algorithms to extract table information from tender documents and convert it into structured data.

[0043] Step 2: Feature Extraction Text feature extraction: Text features are extracted using methods such as TF-IDF (term frequency-inverse document frequency) or Word2Vec, and semantic features of the text are extracted in combination with pre-trained language models (such as BERT).

[0044] Image feature extraction: Convolutional neural networks (CNNs) are used to extract feature vectors from images, and object detection models are combined to extract key graphic features from the images.

[0045] Table feature extraction: Perform statistical analysis on table data to extract key indicators and trend information, such as field names, numerical ranges, and logical relationships, as table features.

[0046] Step 3: Multimodal information fusion: Constructing a fusion model: Design a multimodal fusion model that concatenates or weights the feature vectors of text, images, and tables to form a comprehensive feature vector.

[0047] Training the model: Using historical bidding data and expert annotation results, supervised learning is performed on the fusion model to optimize the model parameters, enabling it to accurately judge the compliance of bidding documents.

[0048] Step 4: Application of Bill Clearing Rules: Compliance check: Based on the specific requirements in the bidding documents and laws and regulations, a series of bid clearing rules are set, such as qualification requirements, technical parameters, commercial terms, etc.

[0049] Rule matching: The fused feature vectors are matched with the bid clearing rules to automatically identify bids that do not meet the requirements.

[0050] Anomaly Detection: For tender documents suspected of being non-compliant or abnormal, further manual review will be conducted to ensure the accuracy of the tender clearing results.

[0051] Step 5: Results Output and Feedback Generate a bid evaluation report, including the assessment results of the compliance, completeness, and accuracy of the bidding documents.

[0052] Anomalies are marked and highlighted for manual review.

Claims

1. A bidding and procurement evaluation method based on multimodal information fusion, characterized in that, Includes the following steps: S1, Data Preprocessing: Obtain the tender documents and all bidders' tender documents, and convert the tender documents into electronic documents in a unified format; The text content in electronic documents is extracted using optical character recognition technology, and the images and tables in the electronic documents are preprocessed separately. S2, Multimodal Feature Extraction: Feature extraction is performed on the preprocessed text content, images, and tables to obtain text feature vectors, image feature vectors, and table feature vectors. Among them, text feature extraction adopts the term frequency-inverse document frequency method or the semantic feature extraction method based on a pre-trained language model; image feature extraction adopts a pre-trained convolutional neural network model; table feature extraction includes identifying the field names and data types in the table, and performing statistical analysis on numerical fields, including minimum value, maximum value, and quantiles. S3, Multimodal Information Fusion: Text feature vectors, image feature vectors, and table feature vectors are input into a multimodal fusion neural network model for fusion to obtain a comprehensive feature vector representing the content of the tender document; the multimodal fusion neural network model adopts a fusion method based on an attention mechanism or a multimodal Transformer model; S4, Bid Clearing Rule Matching and Evaluation: Based on the preset bid clearing rule set, the comprehensive feature vector is matched and logically judged; the bid clearing rule set includes rules under the dimensions of qualification requirements, technical review, commercial review, and reputation and risk; based on the matching and judgment results, the compliance, completeness and accuracy of the bid documents are evaluated, and the bid documents that meet the requirements are selected. S5, Result Generation and Output: Generate a bid clearing report, which includes at least the compliance assessment results, completeness assessment results, accuracy assessment results, and annotations and prompts for abnormal information for each bid document.

2. The bidding and procurement evaluation method based on multimodal information fusion according to claim 1, characterized in that, In step S1, the data preprocessing further includes: Convert tender documents of different formats to PDF format; The extracted text content is cleaned and standardized. Preprocessing operations, including denoising, binarization, and tilt correction, are performed on the image. Use a table recognition algorithm to convert tabular information into structured data.

3. The bidding and procurement evaluation method based on multimodal information fusion according to claim 1, characterized in that, In step S2, the semantic feature extraction method based on the pre-trained language model adopts the BERT model.

4. The bidding and procurement evaluation method based on multimodal information fusion according to claim 1, characterized in that, In step S2, the pre-trained convolutional neural network model is a ResNet model.

5. The bidding and procurement evaluation method based on multimodal information fusion according to claim 1, characterized in that, In step S3, the attention-based fusion method specifically includes: Calculate the similarity between each pair of the text feature vector, image feature vector, and table feature vector, and assign attention weights to each feature vector based on the similarity. The comprehensive feature vector is obtained by weighting and summing the corresponding feature vectors according to the attention weights.

6. The bidding and procurement evaluation method based on multimodal information fusion according to claim 5, characterized in that, In step S4, the rule parameters for qualification requirements include enterprise qualification level, project experience, and personnel certificates; the rule parameters for technical review include technological innovation, feasibility of construction technology, and reasonableness of construction period; the rule parameters for commercial review include total quotation range, reasonableness of itemized quotations, and payment terms; and the rule parameters for reputation and risk include enterprise credit rating, litigation records, and performance capability.

7. The bidding and procurement evaluation method based on multimodal information fusion according to claim 1, characterized in that, In step S4, the compliance assessment is used to determine whether the content of the tender document complies with the mandatory requirements of the tender document; the completeness assessment is used to determine whether the tender document contains all the required content items. The accuracy assessment is used to determine whether the data in the tender documents is accurate and free of logical contradictions.

8. The bidding and procurement evaluation method based on multimodal information fusion according to claim 1, characterized in that, Step S4 further includes: sorting the qualified tender documents according to preset sorting rules, including sorting by total price from low to high or by technical solution score from high to low.

9. The bidding and procurement evaluation method based on multimodal information fusion according to claim 1, characterized in that, In step S5, the abnormal information includes missing required documents, bid content that does not conform to the rules, logically contradictory data, and quotations below cost.

10. A bidding and procurement method based on multimodal information fusion according to claim 5, characterized in that, The three feature vectors are fused together using a multimodal Transformer fusion method combined with an attention mechanism: An attention mechanism is used to dynamically fuse data from different modalities. By calculating the correlation between data from different modalities, different weights are assigned to each modality; the formula is as follows: Calculate attention weights: ； in It is attention weight. It is a similarity function. It is an eigenvector; Calculate the attention result: ； in It is a result of attention; The multi-head self-attention mechanism of the multimodal Transformer model is used to process three feature vectors simultaneously by establishing attentional connections between different modalities.

Images