Image similarity detection method in a bid document based on a multi-modal large model

Abstract

The application discloses a kind of based on multi-modal big model's image similarity detection method in tender document, it is related to image processing field, comprising: S1, construct tender document special image dataset;S2, multidimensional fusion image feature extraction: to input the tender image to be detected is normalized and three categories of basic features are extracted in parallel;S3, multi-modal feature alignment and similarity measurement calculation;S4, certificate tampering trace auxiliary discrimination;S5, multi-modal big model explainability analysis report generation;S6, batch image detection output;Based on approximate nearest neighbor product quantization algorithm builds feature vector retrieval library, stores all tender document fusion feature vectors;Batch import multiple tenderer tender document image, retrieval library quickly matches similar image pair, outputs image similarity, certificate tampering, text duplication three types of risk results, forms visual risk review list.Effective identification existing bid evaluation system extremely easy to miss judgment Hidden string label image reuse behavior.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a multi-modal large model-based image similarity detection method in bidding documents. Background Art

[0002] As the domestic power industry's bidding and tendering supervision system becomes increasingly perfected and the standardization level continues to improve, the compliance, fairness and transparency of the bidding and tendering market have become the core cornerstone of high-quality industry development. However, with the continuous increase of participating entities in bidding and tendering and the increasingly fierce market competition, some bidding suppliers continue to iterate illegal bidding methods to evade supervision and increase the probability of winning bids, abandoning the traditional blatant bid rigging and collusive bidding operations, and turning to concealed, lightweight and modified document tampering and copying methods to carry out illegal bidding, among which the reuse and fine-tuning tampering of image materials in bidding documents is the most common form of violation at present.

[0003] Specifically, core modules such as technical solutions, implementation cases, construction deployment, and hardware architectures in power project bidding documents contain a large number of image materials such as project schematic diagrams, on-site scene photos, equipment structure diagrams, and process topology diagrams. Some suppliers produce homogenized bidding documents by directly reusing similar bidding images, or making minor tampering to original images such as cropping and scaling, fine color adjustment, adding occluding watermarks, and modifying local pixels, so as to piece together bidding schemes and engage in disguised collusive bidding. This type of illegal operation is extremely concealed with no obvious tampering traces, and has become the main inducement for the breeding of risks of collusive bidding and bid rigging in the current power bidding and tendering field. It not only seriously disrupts the fair competition order of the power procurement market, but also easily leads to problems such as substandard quality of winning bid projects and failure of procurement compliance, bringing great operational risks and compliance hidden dangers to power procurement units such as State Grid.

[0004] At present, in the bid evaluation work of the power industry, the verification of the text and image similarity of bidding documents still relies entirely on the manual work of bid evaluation experts, and an intelligent and standardized image similarity detection system has not yet been formed. In conventional power tendering projects, the bidding documents submitted by multiple bidding units often contain hundreds of various image materials. Bid evaluation experts need to complete cross-comparison, homogenization discrimination and violation screening of all bidding images within a limited evaluation period, which brings about industry pain points such as heavy verification workload, tight evaluation time, and strong limitations of manual identification.

[0005] The manual review model has several inherent flaws: First, the human eye has limited recognition capabilities, making it difficult to accurately identify subtle differences in similar images that have undergone cropping, color adjustment, watermarking, and partial modifications, easily leading to missed or incorrect judgments. Second, review results are greatly influenced by the subjective experience, focus, and work status of experts, and the inconsistent judgment standards among different experts result in a lack of standardized and regulated basis for image similarity determination, leading to highly subjective and inconsistent review results. Third, manual item-by-item comparison is extremely inefficient, unable to achieve full coverage and detailed verification of a large number of bidding images, and is difficult to adapt to the current high-frequency and large-scale power bidding review needs.

[0006] In summary, traditional manual verification methods suffer from numerous shortcomings, including low accuracy, poor efficiency, inconsistent standards, and difficulty in achieving full coverage. They are unable to effectively identify covert bidding violations such as image tampering and material reuse, and thus fail to meet the stringent requirements of risk control and compliance supervision in the current power industry. To address the deficiencies of existing technologies, there is an urgent need for an efficient, accurate, and standardized intelligent image similarity detection solution for bid documents. This solution should rely on intelligent technology to replace manual review, enabling pre-emptive warnings and precise in-process screening of bid image violations, thereby ensuring the fair and compliant operation of the power bidding market. Summary of the Invention

[0007] This invention provides an image similarity detection method for tender documents based on a multimodal large model, which effectively identifies hidden bid-rigging image reuse behavior that is easily missed by existing bid evaluation systems.

[0008] According to one aspect of this disclosure, an image similarity detection method for tender documents based on a multimodal large model is provided, comprising the following steps: S1. Construct a dedicated image dataset for tender documents: S2. Multi-dimensional fusion image feature extraction: Normalize the input bid image to be detected and extract three types of basic features in parallel; S3. Multimodal feature alignment and similarity measurement calculation; S4. Assisted identification of document tampering traces; S5. Generation of interpretability analysis report for multimodal large models; S6. Batch Image Detection Output: A feature vector retrieval library is built based on the approximate nearest neighbor product quantization algorithm to store the fused feature vectors of all bid documents; multiple bid documents images are imported in batches, and the retrieval library quickly matches similar image pairs, outputting three types of risk results: image similarity, document tampering, and text similarity, forming a visualized risk review list.

[0009] In one possible implementation, S2 includes: 2.1 Local Handcrafted Features: SIFT / ORB algorithm is used to detect key points in the image and extract local descriptor vectors. Mismatched feature pairs are removed by ratio detection and RANSAC algorithm. 2.2 Global hash feature: Perceptual hashing (pHash) is used to generate fixed-length binary hash codes for images, which are used for rapid coarse screening of massive images; 2.3 Deep Semantic Features: A lightweight CNN network is used to extract deep global semantic features of the image; We designed a feature fusion network with an embedded SE attention module, which aligns and concatenates local handcrafted features, global hash features, and CNN deep semantic features, assigns differentiated feature weights, and generates a unified multidimensional fusion feature vector. We introduced adversarial training, which randomly applied affine transformations, color shifts, and noise interference to the training images to optimize the robustness of the fusion features to image perturbations. We used Triplet Loss metric learning to optimize the feature space, making the feature vector space of similar images closer together.

[0010] In one possible implementation, S3 includes: 3.1 Modal Feature Extraction: Text within the tender image is recognized by OCR, and semantic vectors of the text are extracted using Sentence-BERT; layout features of document format, tables, and seal areas are extracted using LayoutLM; 3.2 Cross-modal unified representation: A cross-modal attention network is built to map the image fusion features, text semantic vectors, and layout features output by S2 to the same high-dimensional semantic space, thereby completing cross-modal semantic alignment of images and text; 3.3 Adaptive Weight Fusion: The similarity of three modal features (image, text, and layout) is dynamically learned through a lightweight neural network to determine the weights and generate a joint feature representation vector for the tender documents; 3.4 Similarity Measurement Calculation: Based on the Siamese network architecture, cosine similarity is used to calculate the similarity score of the joint feature representation vector of the two tender documents; a multi-task loss function integrating contrast loss and classification loss is constructed to train the measurement model and output a continuous similarity score in the range of 0 to 1; the similarity score is matched with the knowledge base of power bidding violations to generate three levels of risk labels: high, medium and low.

[0011] In one possible implementation, S4 includes: For images of three types of qualification certificates—patent certificates, safety production licenses, and high-tech enterprise certificates—a multimodal visual large model is used to identify image noise distribution, edge artifacts, JPEG compression block effects, and color banding tampering features. This model generates a tampering attention heatmap to annotate suspicious PS regions and outputs the certificate forgery judgment result.

[0012] In one possible implementation, S5 includes: Based on a multimodal large model of the power industry, a fine-tuning dataset containing "image pairs, similarity scores, tampering heatmaps, and risk labels" was constructed to complete domain fine-tuning. A thought chain prompt template was designed to guide the model to reason logically according to the following steps: "overall image similarity description - listing of local similarity evidence - explanation of suspected document tampering - comprehensive risk assessment of bid rigging". Image features, similarity measurement results, and tampering detection results were input into the fine-tuned multimodal large model to automatically generate a structured natural language risk analysis report and simultaneously output a list of suspected bid rigging clues for review by bid evaluation experts.

[0013] In one possible implementation, the fine-tuning process includes: Using desensitized real bidding images, corresponding OCR text descriptions, and document type labels as input conditions, multiple controllable modification instructions are set. These instructions include replacing company names, modifying qualification validity periods, adjusting performance project amounts, cropping partial images, overlaying watermarks, and adjusting image brightness and contrast. Simulated bidding document images with controllable tampering types are generated through a diffusion model to expand the small sample of bidding violation images.

[0014] In one possible implementation, the multi-dimensional fusion feature vector retrieval process in step S2 is as follows: All bid images are fused feature vectors and compressed through product quantization before being stored in a vector retrieval library; fused feature vectors are extracted from the images to be detected, the retrieval range is narrowed by coarse matching of cluster centers, Hamming distance / Euclidean distance of compressed vectors is calculated, and Top-K highly similar image pairs are quickly returned, achieving millisecond-level batch initial screening of bid images.

[0015] In one possible implementation, the cross-modal attention network described in step S3 can automatically focus on key areas such as seals in tender documents, performance charts, qualification certificates, and construction case drawings, thereby increasing the recognition weight of violations such as image reuse and qualification misuse.

[0016] In one possible implementation, the document tampering detection logic in step S4 includes: By comparing the global illumination consistency, text edge gradient, and seal pixel texture continuity of the document image, if there are abnormal local pixel distribution, blurred edges, or abrupt color gradient changes, it is determined to be a PS tampering, and the coordinates of the tampered pixel area are marked on the heat map.

[0017] In one possible implementation, the interpretability analysis report in step S5 includes: image similarity values, screenshots of similar parts, suspicious areas of document tampering, corresponding bidder information, violation type determination, review priority suggestions, and reference information on similar historical bid-rigging cases. The method also includes a model iterative optimization step: We collect the annotation feedback data after review by the evaluation experts, continuously expand the bidding image dataset, and regularly make incremental fine adjustments to the feature extraction network, multimodal attitude measurement model, and multimodal analysis large model to continuously reduce the false detection rate and false negative rate of image similarity detection, and steadily improve the image similarity detection accuracy to over 90%.

[0018] Compared with the prior art, the beneficial effects of the present invention are: I. Image Similarity Detection Performance 1. High anti-interference recognition capability, significantly reducing the false negative rate. This invention integrates local handcrafted features, perceptual hash global features, and CNN deep semantic features, and introduces adversarial training to optimize feature robustness. It has stable recognition capabilities against common image perturbations in bidding documents (cropping, scaling, rotation, color adjustment, watermarking, partial erasure, and slight compression distortion). It can accurately capture reused performance images, qualification drawings, and case illustrations after minor modifications. Compared with traditional single pHash and SIFT pixel matching schemes, it improves the recall rate of tampered and modified images by more than 12%, and can effectively identify hidden bid-rigging image reuse behavior that is easily missed by existing bidding evaluation systems.

[0019] 2. The detection accuracy meets the standards, satisfying the high risk control requirements of power bidding. By leveraging a diffusion model to expand the domain-specific labeled dataset and optimizing multimodal cross-semantic joint metric learning, the overall accuracy of image + text joint similarity detection has been steadily achieved to over 90%. Unlike traditional solutions that rely solely on image pixel comparison, this invention simultaneously links OCR text semantics and document layout information to complete cross-validation, which can distinguish between "normal similarity to industry standard templates" and "maliciously copied and applied images by suppliers," effectively reducing the false positive rate and decreasing the workload of manual review by more than 50%.

[0020] 3. High-speed batch testing, suitable for large-scale bidding and review scenarios. The underlying algorithm is equipped with a product quantization approximate vector retrieval algorithm. After the image features are vectorized, compressed, and stored in the database, the feature extraction speed of a single bidding image is ≥15 images / second. The batch parallel retrieval of multiple bidders' bids can output Top-K similar image pairs in milliseconds. The full initial screening of hundreds of bid documents and thousands of attached images in a single tender can be completed within 3 hours, solving the business pain points of traditional manual image-by-image comparison being time-consuming and having tight review cycles.

[0021] II. Detection of counterfeit qualification certificates For three types of frequently used qualification certificates in power bidding—patent certificates, safety production licenses, and high-tech enterprise certificates—a dedicated image tampering detection model was constructed. Through multi-dimensional anomaly identification based on pixel texture, illumination distribution, compression block effect, and edge gradient, the accuracy rate of detecting PS tampering traces on the certificates reached over 80%. The model outputs a visual tampering heat map to mark suspicious pixel areas, enabling automatic early warning of certificate forgery. This fills the technical gap in existing bidding tools that can only compare similar images but cannot identify the authenticity of certificates, thus blocking the risk of false qualification bidding at the source.

[0022] III. Multimodal Fusion Cross-Modal Verification 1. Achieve joint verification of text, images, and semantics to overcome the shortcomings of single-modality detection. Traditional solutions only perform text plagiarism checks or image comparisons independently, failing to identify covert bid-rigging tactics such as "reusing images but deliberately modifying text descriptions." This invention maps image visual features, text semantic vectors, and table / stamp layout features to a unified semantic space through a cross-modal attention network, simultaneously completing image-text consistency verification. It can accurately locate violations such as "the same performance image paired with different project texts," improving the full-dimensional identification capability of bid-rigging and collusion.

[0023] 2. Adaptive modal weight allocation, aligned with the business logic of power bidding. The model dynamically adjusts the weights of image, text, and layout features in similarity determination through self-learning. It automatically increases the weights of high-risk features such as seals, performance charts, and qualification certificates, while weakening the weights of risk-free general content such as generic templates and standard flowcharts. The recognition logic aligns with the power procurement review rules, and the detection results are highly consistent with the standards for expert manual judgment. Attached Figure Description

[0024] Figure 1 The flowchart illustrates an image similarity detection method in a tender document based on a multimodal large model, according to an embodiment of this disclosure. Detailed Implementation

[0025] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0026] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0027] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0028] An image similarity detection method for tender documents based on a multimodal large model includes the following steps: S1. Construct a dedicated image dataset for tender documents: Images of real power bidding documents after anonymization were collected. Standardized preprocessing was performed on the images, including resolution unification, image denoising, tilt correction, and random region masking. Based on a fine-tuned conditional diffusion model, simulated bidding images were generated using the bidding text, document type, and modification instructions extracted by OCR as conditional inputs. These images included cropping, color adjustment, local tampering, watermark addition, and text replacement perturbations. The real images were then fused with the generated simulated images. Industry experts completed pixel-level annotations, including overall image similarity scores, coordinates of tampered regions, and tampering type labels, resulting in a fully annotated bidding image dataset. S2. Multi-dimensional fusion image feature extraction: The input bid image to be detected is normalized, and three types of basic features are extracted in parallel: 2.1 Local Handcrafted Features: SIFT / ORB algorithm is used to detect key points in the image and extract local descriptor vectors. Mismatched feature pairs are removed by ratio detection and RANSAC algorithm. 2.2 Global hash feature: Perceptual hashing (pHash) is used to generate fixed-length binary hash codes for images, which are used for rapid coarse screening of massive images; 2.3 Deep Semantic Features: A lightweight CNN network is used to extract deep global semantic features of the image; We designed a feature fusion network with an embedded SE attention module, which aligns and concatenates local handcrafted features, global hash features, and CNN deep semantic features, assigns differentiated feature weights, and generates a unified multidimensional fusion feature vector. We introduced adversarial training, which randomly applied affine transformations, color shifts, and noise interference to the training images to optimize the robustness of the fusion features to image perturbations. We used Triplet Loss metric learning to optimize the feature space, making the feature vector space of similar images closer together. S3. Multimodal feature alignment and similarity measurement calculation: 3.1 Modal Feature Extraction: Text within the tender image is recognized by OCR, and semantic vectors of the text are extracted using Sentence-BERT; layout features of document format, tables, and seal areas are extracted using LayoutLM; 3.2 Cross-modal unified representation: A cross-modal attention network is built to map the image fusion features, text semantic vectors, and layout features output by S2 to the same high-dimensional semantic space, thereby completing cross-modal semantic alignment of images and text; 3.3 Adaptive Weight Fusion: The similarity of three modal features (image, text, and layout) is dynamically learned through a lightweight neural network to determine the weights and generate a joint feature representation vector for the tender documents; 3.4 Similarity Measurement Calculation: Based on the Siamese network architecture, cosine similarity is used to calculate the similarity score of the joint feature representation vector of the two bid documents; a multi-task loss function integrating contrast loss and classification loss is constructed to train the measurement model, outputting a continuous similarity score in the range of 0 to 1; the similarity score is matched with the knowledge base of violations in power bidding to generate three levels of risk labels: high, medium and low. S4. Assisted in identifying document tampering traces: For images of three types of qualification certificates—patent certificates, safety production licenses, and high-tech enterprise certificates—a multimodal visual large model is used to identify image noise distribution, edge artifacts, JPEG compression block effects, and color discontinuity tampering features. Tampering attention heatmaps are generated to annotate suspicious PS regions, and the certificate forgery judgment results are output. S5. Generation of interpretability analysis report for multimodal large models: Based on a multimodal large-scale model of the power industry, a fine-tuning dataset containing "image pairs, similarity scores, tampering heatmaps, and risk labels" was constructed to complete domain fine-tuning. A thought chain prompt template was designed to guide the model to reason logically according to the following steps: "overall image similarity description - listing of local similarity evidence - explanation of suspected document tampering - comprehensive risk assessment of bid rigging". Image features, similarity measurement results, and tampering detection results were input into the fine-tuned multimodal large-scale model to automatically generate a structured natural language risk analysis report and simultaneously output a list of suspected bid rigging clues for review by bid evaluation experts. S6. Batch Image Detection Output: A feature vector retrieval library is built based on the approximate nearest neighbor product quantization algorithm to store the fused feature vectors of all bid documents; multiple bid documents images are imported in batches, and the retrieval library quickly matches similar image pairs, outputting three types of risk results: image similarity, document tampering, and text similarity, forming a visualized risk review list.

[0029] In one possible implementation, the conditional diffusion model fine-tuning process in step S1 includes: Using desensitized real bidding images, corresponding OCR text descriptions, and document type labels as input conditions, multiple controllable modification instructions are set. These instructions include replacing company names, modifying qualification validity periods, adjusting performance project amounts, cropping partial images, overlaying watermarks, and adjusting image brightness and contrast. Simulated bidding document images with controllable tampering types are generated through a diffusion model to expand the small sample of bidding violation images.

[0030] In one possible implementation, the multi-dimensional fusion feature vector retrieval process in step S2 is as follows: All bid images are fused feature vectors and compressed through product quantization before being stored in a vector retrieval library; fused feature vectors are extracted from the images to be detected, the retrieval range is narrowed by coarse matching of cluster centers, Hamming distance / Euclidean distance of compressed vectors is calculated, and Top-K highly similar image pairs are quickly returned, achieving millisecond-level batch initial screening of bid images.

[0031] In one possible implementation, the cross-modal attention network described in step S3 can automatically focus on key areas such as seals in tender documents, performance charts, qualification certificates, and construction case drawings, thereby increasing the recognition weight of violations such as image reuse and qualification misuse.

[0032] In one possible implementation, the document tampering detection logic in step S4 includes: By comparing the global illumination consistency, text edge gradient, and seal pixel texture continuity of the document image, if there are abnormal local pixel distribution, blurred edges, or abrupt color gradient changes, it is determined to be a PS tampering, and the coordinates of the tampered pixel area are marked on the heat map.

[0033] In one possible implementation, the interpretability analysis report includes: image similarity values, screenshots of similar parts, suspicious areas of document tampering, corresponding bidder information, violation type determination, review priority suggestions, and reference information on similar historical bid-rigging cases.

[0034] In one possible implementation, the method further includes a model iterative optimization step: We collect the annotation feedback data after review by the evaluation experts, continuously expand the bidding image dataset, and regularly make incremental fine adjustments to the feature extraction network, multimodal attitude measurement model, and multimodal analysis large model to continuously reduce the false detection rate and false negative rate of image similarity detection, and steadily improve the image similarity detection accuracy to over 90%.

[0035] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for image similarity detection in tender documents based on a multimodal large model, characterized in that, Includes the following steps: S1. Construct a dedicated image dataset for tender documents: S2. Multi-dimensional fusion image feature extraction: Normalize the input bid image to be detected and extract three types of basic features in parallel; S3. Multimodal feature alignment and similarity measurement calculation: S4. Assisted identification of document tampering traces; S5. Generation of interpretability analysis report for multimodal large models; S6, Batch image detection output; A feature vector retrieval library is built based on the approximate nearest neighbor product quantization algorithm to store the fused feature vectors of all bid documents; multiple bid documents images are imported in batches, and the retrieval library quickly matches similar image pairs, outputting three types of risk results: image similarity, document tampering, and text similarity, forming a visualized risk review list.

2. The image similarity detection method in tender documents based on a multimodal large model according to claim 1, characterized in that, S2 include: 2.1 Local Handcrafted Features: SIFT / ORB algorithm is used to detect key points in the image and extract local descriptor vectors. Mismatched feature pairs are removed by ratio detection and RANSAC algorithm. 2.2 Global hash feature: Perceptual hashing (pHash) is used to generate fixed-length binary hash codes for images, which are used for rapid coarse screening of massive images; 2.3 Deep Semantic Features: A lightweight CNN network is used to extract deep global semantic features of the image; We designed a feature fusion network with an embedded SE attention module, which aligns and concatenates local handcrafted features, global hash features, and CNN deep semantic features, assigns differentiated feature weights, and generates a unified multidimensional fusion feature vector. We introduced adversarial training, which randomly applied affine transformations, color shifts, and noise interference to the training images to optimize the robustness of the fusion features to image perturbations. We used Triplet Loss metric learning to optimize the feature space, making the feature vector space of similar images closer together.

3. The image similarity detection method in tender documents based on a multimodal large model according to claim 1, characterized in that, S3 include: 3.1 Modal Feature Extraction: Text within the tender image is recognized by OCR, and semantic vectors of the text are extracted using Sentence-BERT; layout features of document format, tables, and seal areas are extracted using LayoutLM; 3.2 Cross-modal unified representation: A cross-modal attention network is built to map the image fusion features, text semantic vectors, and layout features output by S2 to the same high-dimensional semantic space, thereby completing cross-modal semantic alignment of images and text; 3.3 Adaptive Weight Fusion: The similarity of three modal features (image, text, and layout) is dynamically learned through a lightweight neural network to determine the weights and generate a joint feature representation vector for the tender documents; 3.4 Similarity metric calculation: Based on the Siamese network architecture, cosine similarity is used to calculate the similarity score of the joint feature representation vector of the two tender documents; A multi-task loss function integrating contrastive loss and classification loss is constructed to train the metric model, outputting a continuous similarity score in the range of 0 to 1; the similarity score is matched with a knowledge base of violations in power bidding to generate three levels of risk labels: high, medium, and low.

4. The image similarity detection method in tender documents based on a multimodal large model according to claim 1, characterized in that, S4 include: For images of three types of qualification certificates—patent certificates, safety production licenses, and high-tech enterprise certificates—a multimodal visual large model is used to identify image noise distribution, edge artifacts, JPEG compression block effects, and color banding tampering features. This model generates a tampering attention heatmap to annotate suspicious PS regions and outputs the certificate forgery judgment result.

5. The image similarity detection method in tender documents based on a multimodal large model according to claim 1, characterized in that, S5 include: Based on a multimodal large model of the power industry, a fine-tuning dataset containing "image pairs, similarity scores, tampering heatmaps, and risk labels" was constructed to complete domain fine-tuning. A thought chain prompt template was designed to guide the model to reason logically according to the following steps: "overall image similarity description - listing of local similarity evidence - explanation of suspected document tampering - comprehensive risk assessment of bid rigging". Image features, similarity measurement results, and tampering detection results were input into the fine-tuned multimodal large model to automatically generate a structured natural language risk analysis report and simultaneously output a list of suspected bid rigging clues for review by bid evaluation experts.

6. The image similarity detection method in tender documents based on a multimodal large model according to claim 5, characterized in that, The fine-tuning process includes: Using desensitized real bidding images, corresponding OCR text descriptions, and document type labels as input conditions, multiple controllable modification instructions are set. These instructions include replacing company names, modifying qualification validity periods, adjusting performance project amounts, cropping partial images, overlaying watermarks, and adjusting image brightness and contrast. Simulated bidding document images with controllable tampering types are generated through a diffusion model to expand the small sample of bidding violation images.

7. The image similarity detection method in tender documents based on a multimodal large model according to claim 1, characterized in that, The multi-dimensional fusion feature vector retrieval process described in step S2 is as follows: All bid images are fused feature vectors and compressed through product quantization before being stored in a vector retrieval library; fused feature vectors are extracted from the images to be detected, the retrieval range is narrowed by coarse matching of cluster centers, Hamming distance / Euclidean distance of compressed vectors is calculated, and Top-K highly similar image pairs are quickly returned, achieving millisecond-level batch initial screening of bid images.

8. The image similarity detection method in tender documents based on a multimodal large model according to claim 3, characterized in that, The cross-modal attention network described in step S3 can automatically focus on key areas such as seals in tender documents, performance charts, qualification certificates, and construction case drawings, thereby increasing the recognition weight of violations such as image reuse and qualification misuse.

9. The image similarity detection method in tender documents based on a multimodal large model according to claim 1, characterized in that, The document tampering detection logic in step S4 includes: By comparing the global illumination consistency, text edge gradient, and seal pixel texture continuity of the document image, if there are abnormal local pixel distribution, blurred edges, or abrupt color gradient changes, it is determined to be a PS tampering, and the coordinates of the tampered pixel area are marked on the heat map.

10. The image similarity detection method in tender documents based on a multimodal large model according to claim 1, characterized in that, The interpretability analysis report mentioned in step S5 includes: image similarity values, screenshots of similar parts, suspicious areas of document tampering, corresponding bidder information, violation type determination, review priority suggestions, and reference information on similar historical bid-rigging cases; The method also includes a model iterative optimization step: We collect the annotation feedback data after review by the evaluation experts, continuously expand the bidding image dataset, and regularly make incremental fine adjustments to the feature extraction network, multimodal attitude measurement model, and multimodal analysis large model to continuously reduce the false detection rate and false negative rate of image similarity detection, and steadily improve the image similarity detection accuracy to over 90%.

Images