Bidding document inspection method and device, computer equipment and storage medium

Abstract

The invention relates to a bidding document inspection method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a bidding file, and analyzing the bidding file to obtain multi-modal data; obtaining a query text, determining a qualification proof file type corresponding to the query text, and searching based on the multi-modal data according to a search strategy matched with the qualification proof file type to obtain a search result; carrying out existence check on the bidding file according to the search result and the qualification proof file type through a multi-modal large model to obtain an existence check result; and under the condition that the existence test result shows that the bidding file comprises the qualification certificate file belonging to the qualification certificate file type, carrying out accuracy test on the qualification certificate file through a multi-modal large model to obtain an accuracy test result. By adopting the method, the inspection processing efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a bidding document inspection method and device, computer equipment and storage medium. BACKGROUND

[0002] As a key link in the bidding process, the core purpose of bid auditing is crucial. It focuses on ensuring the integrity and compliance of the bidding document. Through a rigorous, detailed and systematic auditing process, each content in the bidding document, such as qualification certificate, technical scheme and price list, is verified one by one to accurately determine whether the bid document meets the requirements of the bidding party. However, in the face of a large number of complex and numerous bidding requirements and bidding documents, the current inspection and processing efficiency of the bidding document is low. SUMMARY

[0003] Therefore, it is necessary to provide a bidding document inspection method and device, computer equipment, computer readable storage medium and computer program product capable of improving the inspection and processing efficiency.

[0004] In a first aspect, the present application provides a bidding document inspection method, comprising:

[0005] Obtaining a bidding document, analyzing the bidding document, and obtaining multi-modal data;

[0006] Obtaining a query text, determining the qualification certificate file type corresponding to the query text, searching based on the multi-modal data according to the search strategy adapted to the qualification certificate file type, and obtaining a search result;

[0007] Performing existence inspection on the bidding document based on the search result and the qualification certificate file type by a multi-modal large model, and obtaining an existence inspection result;

[0008] In the case that the existence inspection result indicates that the bidding document includes a qualification certificate file belonging to the qualification certificate file type, performing accuracy inspection on the qualification certificate file by the multi-modal large model, and obtaining an accuracy inspection result.

[0009] In a second aspect, the present application further provides a bidding document inspection device, comprising:

[0010] A bidding document analysis module for obtaining a bidding document, analyzing the bidding document, and obtaining multi-modal data;

[0011] A search processing module for obtaining a query text, determining the qualification certificate file type corresponding to the query text, searching based on the multi-modal data according to the search strategy adapted to the qualification certificate file type, and obtaining a search result;

[0012] The existence verification module is configured to perform existence verification on the bid file according to the search result and the qualification certificate file type by using the multi-modal large model, and obtain an existence verification result.

[0013] The accuracy verification module is configured to perform accuracy verification on the qualification certificate file by using the multi-modal large model in a case where the existence verification result indicates that the bid file includes the qualification certificate file belonging to the qualification certificate file type, and obtain an accuracy verification result.

[0014] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the method provided in the first aspect when executing the computer program.

[0015] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in the method provided in the first aspect.

[0016] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the steps in the method provided in the first aspect.

[0017] The bid file verification method, device, computer device, computer readable storage medium and computer program product can analyze the obtained bid file, obtain multi-modal data, perform search based on the multi-modal data according to a search strategy suitable for the qualification certificate file type corresponding to the query text, obtain a search result, perform existence verification on the bid file according to the search result and the qualification certificate file type by using a multi-modal large model, and perform accuracy verification on the qualification certificate file by using the multi-modal large model in a case where the obtained existence verification result indicates that the bid file includes the qualification certificate file belonging to the qualification certificate file type, and obtain an accuracy verification result. In the bid file verification process, the qualification certificate file type corresponding to the query text is determined, search is performed based on the multi-modal data obtained by analyzing the bid file according to the search strategy suitable for the qualification certificate file type, and the relevant content of the file can be quickly located; existence verification and accuracy verification are sequentially performed on the qualification certificate file by using the multi-modal large model according to the search result and the qualification certificate file type, and based on the high processing capability of the multi-modal large model, the verification process is more smooth and rapid, and thus the verification processing efficiency of the bid file is improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a diagram illustrating the application environment of a tender document verification method in one embodiment.

[0020] Figure 2 This is a flowchart illustrating a tender document verification method in one embodiment;

[0021] Figure 3 This is a flowchart illustrating the multimodal data search and processing in one embodiment;

[0022] Figure 4 This is a flowchart illustrating the process of building a multi-dimensional searcher in one embodiment.

[0023] Figure 5 This is a flowchart illustrating the existence check process in one embodiment;

[0024] Figure 6 This is a flowchart illustrating the accuracy verification process in one embodiment;

[0025] Figure 7 This is a structural block diagram of a tender document inspection device in one embodiment;

[0026] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0029] The tender document inspection method provided in this application embodiment can be applied to, for example,Figure 1 The terminal 102 communicates with the server 104 through a network. The data storage system can store data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on a cloud or other network server. The user can send a bid file to the server 104 through the terminal 102. The server 104 can obtain the bid file sent by the terminal and parse the bid file to obtain multi-modal data. The server 104 can search based on the multi-modal data according to a search strategy adapted to the type of the qualification certificate file corresponding to the query text to obtain a search result. The server 104 can perform existence verification on the bid file based on the search result and the type of the qualification certificate file through a multi-modal large model. If the existence verification result indicates that the bid file includes a qualification certificate file belonging to the type of the qualification certificate file, the server 104 can perform accuracy verification on the qualification certificate file through the multi-modal large model to obtain an accuracy verification result. In some embodiments, the server 104 can obtain a file verification result for the bid file according to the existence verification result and the accuracy verification result, and return the file verification result to the terminal 102 for feedback to the user through the terminal 104.

[0030] The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, unmanned aerial vehicles, low-altitude flying vehicles, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0031] In an exemplary embodiment, as Figure 2 shown, a bid file verification method is provided, which is executed by a computer device, specifically by a terminal or a server, etc. The method can be executed by a terminal or a server alone, or by a terminal and a server together. In the embodiments of the present application, the method is applied to the server in Figure 1 the above environment. The method includes the following steps 202 to 208. Wherein:

[0032] Step 202, obtaining a bid file, and parsing the bid file to obtain multi-modal data.

[0033] The bid document is a complete set of application materials submitted by the bidder according to the requirements of the tendering party. The bid document can include the tender document, which can include the technical scheme, management scheme, and commercial offer of the bidder. The tender document can also include qualification certificate files for proving the strength of the company. The qualification certificate file is a supporting material for proving that the bidder has the necessary qualifications, ability, and reputation to perform the contract. The existence and accuracy of the qualification certificate file are important items for verifying the bid document. The qualification certificate file can be embedded in the text content of the tender document in the form of a picture or a scan. The qualification certificate file can include, but is not limited to, at least one of a business license, a financial report, an audit report, a qualification certificate, a proof of no violation record, an authorization, and an inspection and testing report. The multi-modal data can be obtained by text analysis of the bid document. The multi-modal data can include, but is not limited to, text, image, and other modal data.

[0034] Optionally, the server can obtain the bid document to be verified. When verifying the bid document, the server needs to locate and extract the scattered qualification certificate files from the bid document and verify the existence (whether submitted) and accuracy (whether the content is correct, whether the date is valid, and whether the seal is stamped) of the qualification certificate files. The server can analyze the bid document to obtain the multi-modal data in the bid document. In some embodiments, the server can perform text analysis on the bid document, such as image extraction and OCR (Optical Character Recognition) recognition, table data extraction, HTML (HyperText Markup Language) content analysis, content segmentation, and other processes to obtain the multi-modal data of the bid document.

[0035] In step 204, the query text is obtained, the type of the qualification certificate file corresponding to the query text is determined, the search strategy suitable for the type of the qualification certificate file is adopted, and the search is performed based on the multi-modal data to obtain the search result.

[0036] The query text can be provided by the user through the terminal to define the verification target, and the type of the qualification certificate file to be verified can be determined through the query text. The type of the qualification certificate file can be determined based on the query text, and is used to represent the type of the qualification certificate file to be verified. For example, the user can input "check if there is a business license", "find the financial audit report of the past three years", or "verify the authentication certificate" through the terminal. The search strategy is a strategy for searching data from the multi-modal data, and the search strategy can include at least one. Different search strategies can correspond to different search methods, such as different search strategies can correspond to different search dimensions. The search strategy can be adapted to the type of the qualification certificate file, and different types of qualification certificate files can have corresponding search strategies, and the search strategy can include one or more. The search result is obtained by searching data in the multi-modal data according to the search strategy adapted to the type of the qualification certificate file.

[0037] For example, the server can obtain the query text and determine the type of the qualification certificate file corresponding to the query text. For example, the server can parse the query text and determine the type of the qualification certificate file corresponding to the query text according to the preset mapping rule table. The type of the qualification certificate file can refer to the qualification certificate file that needs to be verified. For example, when the user inputs the query text such as "check the business license" or "find the certificate", the server can extract the core noun or phrase in the query text, such as the keywords "business license" and "authentication certificate"; the server can match the keywords in the query text in the mapping rule table to determine the type of the qualification certificate file corresponding to the query text.

[0038] The server can determine the search strategy adapted to the type of the qualification certificate file. The adaptation relationship between different types of qualification certificate files and search strategies can be preconfigured according to actual needs, and the server can query based on the type of the qualification certificate file and the adaptation relationship to determine the search strategy adapted to the type of the qualification certificate file. For example, the search strategy can include a text block search strategy, a title search strategy, and a paragraph search strategy. The server can search based on the multi-modal data according to the determined search strategy, such as searching for the qualification certificate file belonging to the type of the qualification certificate file in the multi-modal data according to the search strategy to obtain the search result.

[0039] In step 206, the existence verification result is obtained by performing existence verification on the bid file based on the search result and the type of the qualification certificate file through the multi-modal large model.

[0040] The multimodal large language model (MLLM) is a large language model (LLM) that can understand and process multiple different modal data simultaneously. For example, the multimodal large language model can process text, image, audio, video, and other modal data simultaneously. The purpose of the existence test is to verify whether the required qualification certificate file exists in the bid document. The existence test result can be the test result obtained by performing the existence test on the bid document.

[0041] Optionally, the server can obtain a pre-trained multimodal large language model, and perform the existence test on the bid document according to the search result and the qualification certificate file type through the multimodal large language model. For example, the server can input the search result and the qualification certificate file type into the multimodal large language model, and instruct the multimodal large language model to perform the existence test on the bid document based on the search result and the qualification certificate file type, thereby obtaining the existence test result. The existence test result can represent whether the qualification certificate file corresponding to the qualification certificate file type exists in the bid document.

[0042] In step 208, when the existence test result indicates that the bid document includes the qualification certificate file belonging to the qualification certificate file type, the server performs the accuracy test on the qualification certificate file through the multimodal large language model, and obtains the accuracy test result.

[0043] The accuracy test is a verification process on the correctness of the content of the qualification certificate file under the premise that the qualification certificate file corresponding to the qualification certificate file type exists. The accuracy test result can be the test result obtained by performing the accuracy test on the bid document.

[0044] For example, the server can determine whether the bid document includes the qualification certificate file belonging to the qualification certificate file type based on the existence test result. When the existence test result indicates that the bid document includes the qualification certificate file of the corresponding type, the server can perform further accuracy test on the qualification certificate file in the bid document to verify the correctness of the content in the qualification certificate file, and obtain the accuracy test result.

[0045] In the above bid document inspection method, the obtained bid document is parsed to obtain multi-modal data, a search strategy adapted to the qualification certificate file type corresponding to the query text is used to search based on the multi-modal data, a search result is obtained, and the existence of the qualification certificate file in the bid document is verified based on the search result and the qualification certificate file type by using the multi-modal large model. In the case where the existence verification result indicates that the bid document includes the qualification certificate file belonging to the qualification certificate file type, the accuracy of the qualification certificate file is verified by using the multi-modal large model, and an accuracy verification result is obtained. In the bid document inspection process, the qualification certificate file type corresponding to the query text is determined, and the multi-modal data obtained by parsing the bid document is searched based on the search strategy adapted to the qualification certificate file type. The relevant content of the file can be quickly located; by using the multi-modal large model, the existence of the qualification certificate file in the bid document is verified and the accuracy of the qualification certificate file is verified in sequence based on the search result and the qualification certificate file type. Based on the efficient processing capability of the multi-modal large model, the inspection process is more smooth and rapid, thereby improving the inspection processing efficiency of the bid document.

[0046] In one exemplary embodiment, as shown in Figure 3 The search processing for multi-modal data, i.e., determining the qualification certificate file type corresponding to the query text, searching based on the multi-modal data according to the search strategy adapted to the qualification certificate file type, and obtaining a search result, includes steps 302 to 308. Among them:

[0047] Step 302, extracting keywords from the query text, and determining the qualification certificate file type based on the keywords.

[0048] Among them, the keywords can be words related to the qualification certificate file type in the query text, such as including business license and ISO (International Organization for Standardization) 9001 certificate. Optionally, the server can parse the query text to extract keywords from the query text, and the server can determine the qualification certificate file type based on the keywords. For example, the server can maintain a mapping rule table, which can include a mapping relationship between different qualification certificate file types and keywords, and the server can determine the qualification certificate file type based on the mapping relationship based on the keywords.

[0049] Step 304, determining at least one search strategy adapted to the qualification certificate file type.

[0050] Exemplarily, the server can determine a search strategy adapted to the type of the qualification certificate, and the search strategy can include at least one. For example, the search strategy adapted to the type of the qualification certificate can include a text block retrieval strategy and a title search strategy.

[0051] In step 306, a target search set is determined from a candidate search set corresponding to the multi-modal data according to at least one search strategy, and the candidate search set is constructed based on the multi-modal data for different search strategies.

[0052] The candidate search set can be constructed based on the multi-modal data for different search strategies, and different search strategies can correspond to different candidate search sets. For example, the search strategy can include a text block retrieval strategy, and the candidate search set corresponding to the text block retrieval strategy can include various text blocks, which can be obtained by text block division based on the multi-modal data. For another example, the search strategy can include a paragraph search strategy, and the candidate search set corresponding to the paragraph search strategy can include various paragraphs, which can be obtained by paragraph division based on the multi-modal data. The target search set can be a candidate search set corresponding to a search strategy, so as to search according to the corresponding search strategy.

[0053] Optionally, the server can determine various candidate search sets constructed based on the multi-modal data for different search strategies, and the server can determine target search sets corresponding to various search strategies from the candidate search sets according to at least one search strategy adapted to the type of the qualification certificate. The number of target search sets can be the same as the number of types of search strategies, that is, at least one candidate search set can be included in the target search set.

[0054] In step 308, at least one search is performed in the target search set according to the search strategy, and a search result is obtained.

[0055] Exemplarily, for the obtained target search set, the server can perform at least one search in the target search set according to the corresponding search strategy to obtain a search result. For example, the target search set can include a search set 1 corresponding to a search strategy 1 and a search set 2 corresponding to a search strategy 2, the server can perform at least one search in the search set 1 according to the search strategy 1 to obtain a first search result, the server can perform at least one search in the search set 2 according to the search strategy 2 to obtain a second search result, and the server can obtain a final search result according to the first search result and the second search result.

[0056] In some embodiments, the search strategies include multiple types, and the target search sets can also include multiple types. For each search strategy, the server can perform a search in the corresponding target search set according to the search strategy to obtain a search result corresponding to the search strategy. The server can integrate the search results corresponding to the search strategies to obtain a final search result. In some embodiments, different search strategies can be assigned different weights for different types of qualification documents, so that the server can weight and integrate the search results corresponding to different search strategies according to the weights of the search strategies to obtain the final search result. In some embodiments, the number of searches performed according to each search strategy in the corresponding target search set can be related to the search strategy, and the number of searches can be the same or different between different search strategies. For example, for search strategy A, the server can determine that the number of searches for the search strategy A is M, and the server can perform a search in the target search set A corresponding to the search strategy A for M times to obtain the search result corresponding to the search strategy A.

[0057] In this embodiment, the server can determine the type of qualification document according to the keywords of the query text, determine the target search set from the candidate search set corresponding to the multi-modal data according to at least one search strategy suitable for the type of qualification document, and perform at least one search in the corresponding target search set according to the search strategy to obtain a search result. This can ensure the pertinence of the search strategy and the search set, thereby improving the search accuracy and the search efficiency.

[0058] In an exemplary embodiment, the bid document inspection method further includes: based on the multi-modal data, constructing a text block set, a title set, and a paragraph set respectively; the text block set corresponds to a text block search strategy, the title set corresponds to a title search strategy, and the paragraph set corresponds to a paragraph search strategy; and obtaining a candidate search set corresponding to the multi-modal data according to the text block set, the title set, and the paragraph set.

[0059] The text block set corresponds to a text block search strategy, the text block search strategy is to cut the entire document into "text blocks" of moderate size and semantic coherence, generate embedding vectors for each text block, and build a vector index, such as building a FAISS (Facebook AI Similarity Search) index, and when searching, the query text is also converted into a vector, and the most similar text blocks are searched in the index. The text block search strategy is suitable for searching for proof documents that are relatively independent in content and may appear anywhere in the document, such as authorization letters, no-violation record certificates, etc. The title set corresponds to a title search strategy, the title search strategy is to establish an index for the titles (such as "Chapter 1", "business license", etc.) in the document, and when searching, the most relevant title is matched or semantically searched with the query, and the entire chapter or paragraph content under the title is returned. The title search strategy is suitable for documents with standard and explicit titles, such as "business license", "financial audit report", "ISO9001 certification certificate", etc. The paragraph set corresponds to a paragraph search strategy, the paragraph search strategy is to index and search in units of natural paragraphs of the document, the paragraph search strategy is between the coarse granularity of "title search" and the fine granularity of "block search", and focuses on semantic coherence and integrity. The paragraph search strategy can be used as a bottom-up strategy for more generalized, semantic-based matching search processing.

[0060] Exemplarily, the server can construct corresponding search sets for various search strategies based on the multi-modal data. For example, for the text block search strategy, the server can construct a text block set; for the title search strategy, the server can construct a title set; for the paragraph search strategy, the server can construct a paragraph set. The server can obtain a candidate search set corresponding to the multi-modal data based on the text block set, the title set, and the paragraph set.

[0061] In this embodiment, the server can obtain a candidate search set corresponding to the multi-modal data based on the text block set corresponding to the text block search strategy, the title set corresponding to the title search strategy, and the paragraph set corresponding to the paragraph search strategy, so that corresponding search sets can be constructed for different strategies, and the accuracy and processing efficiency of the search can be improved.

[0062] In one exemplary embodiment, through the multi-modal large model, existence verification is performed on the bid document according to the search result and the type of the qualification proof document to obtain an existence verification result, including: determining a multi-modal large model, and generating an existence verification prompt word according to the search result and the type of the qualification proof document; through the existence verification prompt word, instructing the multi-modal large model to perform existence verification on the bid document based on the search result and the type of the qualification proof document to obtain an existence verification result.

[0063] The existence verification prompt word can be generated based on the search result and the qualification certificate file type, and is used to instruct the existence verification processing of the multi-modal large model. Optionally, the server can determine the pre-trained multi-modal large model, and generate the existence verification prompt word for the multi-modal large model based on the search result and the qualification certificate file type. The server can instruct the multi-modal large model to perform existence verification through the existence verification prompt word. For example, the server can input the existence verification prompt word into the multi-modal large model, so that the multi-modal large model performs existence verification on the bid document based on the search result and the qualification certificate file type, to determine whether there is a qualification certificate file belonging to the qualification certificate file type in the bid document, to obtain an existence verification result.

[0064] In this embodiment, the server generates an existence verification prompt word based on the search result and the qualification certificate file type, and instructs the multi-modal large model to perform existence verification through the existence verification prompt word. Based on the high processing capacity of the multi-modal large model, the verification process can be more smooth and rapid, thereby improving the verification processing efficiency of the bid document.

[0065] In one exemplary embodiment, the multi-modal large model is used to perform accuracy verification on the qualification certificate file, and an accuracy verification result is obtained, including: obtaining the qualification certificate file belonging to the qualification certificate file type in the bid document according to the search result; performing information consistency verification, content integrity verification, date information verification and qualification recognition mark verification on the qualification certificate file through the multi-modal large model, to obtain the accuracy verification result.

[0066] The information consistency verification is a verification process for verifying the consistency of the information in the qualification certificate file and the bidder information corresponding to the bid document; the content integrity verification is a verification process for checking whether the content in the qualification certificate file is complete; the date information verification is a verification process for whether the date information in the qualification certificate file meets the bidding requirements; and the qualification recognition mark verification is a verification process for detecting whether the qualification certificate file includes a recognition mark.

[0067] Exemplarily, the server can obtain the qualification certificate file belonging to the qualification certificate file type in the bid file based on the search result. For example, the server can take the search result as the qualification certificate file belonging to the qualification certificate file type. The server can perform accuracy verification on the qualification certificate file through the multi-modal large model, and the accuracy verification can include information consistency verification, content integrity verification, date information verification, and qualification certification mark verification, thereby obtaining an accuracy verification result. In some embodiments, after performing information consistency verification, content integrity verification, date information verification, and qualification certification mark verification on the qualification certificate file through the multi-modal large model respectively, the server can obtain the respective verification results of the information consistency verification, the content integrity verification, the date information verification, and the qualification certification mark verification, and the server can comprehensively obtain the accuracy verification result based on the respective verification results to determine whether the content in the qualification certificate file is correct.

[0068] In this embodiment, the server can obtain the qualification certificate file to be verified according to the search result, and perform information consistency verification, content integrity verification, date information verification, and qualification certification mark verification on the qualification certificate file through the multi-modal large model respectively. The multi-modal large model can be used to perform multi-dimensional accuracy verification based on the high processing capacity of the multi-modal large model, thereby improving the processing efficiency of the bid file verification.

[0069] In one exemplary embodiment, the bid file verification method further includes: determining an initial multi-modal large model; and performing at least one step of weight pruning on each layer of the layer structure included in the initial multi-modal large model in the order of the layer structure to obtain the multi-modal large model.

[0070] The initial multi-modal large model can be a multi-modal large model to be optimized, such as a large language model to be subjected to weight pruning processing. Weight pruning is a process of reducing the complexity of a model by removing "unimportant" weights in the model. The initial multi-modal large model can include a plurality of layer structures, and each layer structure can be used for feature processing. Exemplarily, the server can determine an initial multi-modal large model, and the initial multi-modal large model can include a plurality of layer structures. The plurality of layer structures can be stacked in order to combine to obtain the initial multi-modal large model. For each layer structure in the initial multi-modal large model, the server can perform at least one step of weight pruning on each layer in the order of the layer structure to perform lightweight processing on the initial multi-modal large model and obtain an optimized multi-modal large model.

[0071] In this embodiment, the server can perform at least one step of weight pruning on each layer in the order of the layer structure included in the initial multimodal large model to reduce the complexity of the multimodal large model and improve the processing efficiency of the multimodal large model for verifying tender documents.

[0072] In an exemplary embodiment, for the layer structure included in the initial multimodal large model, weight pruning is performed layer by layer in the order of the layer structure to obtain the multimodal large model, including: determining the target layer structure in the order of the layer structure included in the initial multimodal large model; performing weight pruning for the weights of the target layer structure in at least one step, and obtaining the target weights after pruning the target layer structure based on the output weights of the last weight pruning step; wherein, in each weight pruning step, the number of pruning weights in the current step is determined based on the sparsity of the current step; the pruning weight of the current step is determined from the initial weights of the current step based on the number of pruning weights; the output weight of the current step is determined based on the pruning weights, and if the current step is not the last step, the output weight of the current step is used as the initial weight for the next step of weight pruning; and the multimodal large model is obtained based on the target weights after pruning each of the layer structures included in the initial multimodal large model.

[0073] The server can perform weight pruning on each layer of the initial multimodal model, such as sequentially pruning from low to high layer order. When pruning weights for each layer, the server can perform at least one pruning step. After traversing all layers, the server can obtain a less complex multimodal model based on the initial model.

[0074] The target layer structure can be the layer structure currently traversed in the initial multimodal large model to which weight pruning is performed. The target weights can be the weights obtained after weight pruning of the target layer structure. Sparsity can characterize the proportion of weights pruned in the corresponding layer structure, and the number of pruned weights is the number of weights pruned in the corresponding layer structure. The initial weights are the weights that were not pruned in each step of the weight pruning process. The pruned weights are the weights that need to be pruned, determined from the initial weights. The output weights can be the weights of the corresponding layer structure re-determined after removing the pruned weights from the initial weights.

[0075] Optionally, the initial multi-modal large model can include a plurality of layer structures, and the server can determine a target layer structure currently requiring weight pruning processing in the order of the layer structures included in the initial multi-modal large model. For example, the server can traverse each layer structure in the initial multi-modal large model in the order of low-to-high layer structures to perform weight pruning processing on each layer structure. For the weights in the target layer structure, the server can perform at least one step of weight pruning processing, and obtain the target weights of the target layer structure after pruning according to the output weights of the last step of weight pruning. In each step of weight pruning processing, the server can determine the sparsity of the current step and determine the number of pruning weights of the current step according to the sparsity. The server can determine the initial weights of the current step, and determine the pruning weights of the current step requiring pruning from the initial weights based on the number of pruning weights of the current step.

[0076] In some embodiments, when the current step is the first step, the initial weights of the current step can be the initialization weights of the initial multi-modal large model; when the current step is not the first step, the initial weights of the current step can be obtained according to the output weights of the previous step, i.e., the output weights of the previous step can be used as the initial weights of the current step; when the current step is not the last step, the output weights of the current step can also be used as the initial weights of the next step; when the current step is the last step, the output weights of the current step can be used as the target weights of the target layer structure after pruning.

[0077] The server can determine the output weights of the current step based on the pruning weights, e.g., after removing the pruning weights from the initial weights of the current step, the server can re-determine the weights of the target layer structure to obtain the output weights of the current step. When the current step is not the last step, it indicates that the next step of weight pruning processing is performed, and the server can use the output weights of the current step as the initial weights of the next step to perform the next step of weight pruning, until the last step of weight pruning processing is performed, the server can obtain the target weights of the target layer structure after pruning according to the output weights of the last step, and perform weight pruning processing on the next layer structure. After traversing each layer structure in the initial multi-modal large model, the server can determine the target weights of each layer structure after pruning in the initial multi-modal large model, and the server can obtain the multi-modal large model based on the target weights of each layer structure after pruning.

[0078] In this embodiment, for each layer structure in the initial multi-modal large model, the server can perform at least one step of weight pruning processing to reduce the complexity of the initial multi-modal large model and obtain a lightweight multi-modal large model, which can improve the running efficiency of the multi-modal large model and improve the processing efficiency of the multi-modal large model for verifying the bid document.

[0079] The application also provides an application scenario of the bid document inspection method. Specifically, the bid document inspection method is applied in the application scenario as follows:

[0080] The core purpose of bid auditing is to ensure the integrity and compliance of the bid document, and to verify whether the bid document meets the bidding requirements through a systematic audit process. However, current enterprises mainly rely on manual verification methods for bid auditing. In the face of a large number of qualification certificate files and other picture contents in the bid document, manual auditing has obvious defects: first, manual positioning is difficult and time-consuming. Each bid document involves hundreds of qualification certificate files, and the picture contents of these files are scattered in various paragraphs of the bid document. Manual auditing needs to be positioned one by one, which consumes a lot of time. Second, the accuracy of manual auditing is low. The audit of qualification certificate files needs to verify the accuracy and validity of the content and the validity of the time. Manual auditing is easy to overlook details, resulting in inaccurate auditing.

[0081] In the field of natural language processing, text retrieval technology is the core link of information acquisition. Through semantic matching and relevance calculation, it quickly locates key paragraphs from massive documents, providing accurate data support for downstream tasks. At the same time, multi-modal large models integrate heterogeneous data such as text and images, and achieve deep understanding and generation ability for complex scenarios through large-scale pre-training and cross-modal alignment mechanism. In addition, the rapid development of model pruning technology provides a new path for lightweight deployment. Through structured sparsification and parameter importance evaluation, the model performance is maintained while the computational overhead is significantly reduced, bringing breakthrough efficiency improvement for real-time inference and edge computing scenarios. The three technologies evolve together and promote the development of natural language processing technology towards higher efficiency, intelligence and practicality.

[0082] Based on the problems existing in the auditing of qualification certificate files in the bidding document auditing process, the application provides a bidding document inspection method using a multi-dimensional retrieval system to improve retrieval accuracy. Specifically, for each qualification certificate file, the title and content information of the file are used to search in the bidding document, and then the existence of the file is judged by a large language model. If the initial judgment is not present, context expansion will be performed to prevent omission of content information. If there is a situation where the title and image position are inconsistent, a jump search will also be performed. Thus, the qualification certificate file is quickly and accurately located. The bidding document inspection method provided by the application uses multi-modal large model technology to audit the qualification certificate file. After the required qualification certificate file content is retrieved, the text content and image content are input into the multi-modal large model, and through the prompt word engineering, the accuracy of the file information, whether the date meets the requirements, and other key points are judged, thereby realizing the auditing of the qualification certificate file. In addition, the multi-modal large model is pruned using the layer-by-layer progressive pruning technology based on ADMM (Alternating Direction Method of Multipliers), which makes the model lightweight while maintaining its performance, improving the model's running efficiency. The bidding document inspection method provided by the application can achieve efficient and accurate bidding document qualification certificate file auditing, greatly saving labor costs and improving auditing accuracy.

[0083] For the qualification certificate file auditing task in the bidding document auditing process, the bidding document inspection method provided by the application combines the layer-by-layer progressive pruning technology based on ADMM to audit the bidding document qualification certificate file. Specifically, it includes:

[0084] (1) Multi-modal large model pruning

[0085] The layer-by-layer progressive pruning technology based on ADMM is used to prune the multi-modal large model, obtaining the pruned multi-modal large model. The pruning process is performed layer by layer, and an incremental strategy is used. Specifically: first, use the Wanda rule to select the weights to be pruned (i.e., generate pruning masks); then, use ADMM (Alternating Direction Method of Multipliers) to optimize the weight update after pruning to minimize the reconstruction error; finally, through layer-by-layer incremental pruning, gradually increase the sparsity, and make the model smoothly transition to the target sparsity. Specifically:

[0086] (1.1) Select pruning mask

[0087] The Wanda rule is used to select high-quality masks for pruning large language models (LLMs). Unlike selecting weights with the largest value (magnitude pruning), the Wanda rule selects weights with the highest product of weight absolute value and input neuron norm, i.e. , thereby obtaining high-quality masks where is an element in the weight matrix , specifically the weight in the th row and the th column, i.e., representing the connection weight between neurons; is the column index, corresponding to the number of input neurons (or input channel number), i.e., the output feature of the previous layer, denotes the source of input that the weight receives; is the row index, corresponding to the number of output neurons (or the number of neurons in the current layer), represents the target position of the output of the weight; is the vector of input neurons, i.e., the th input channel or feature; is the square of the L2 norm of (i.e., the square of the Euclidean norm), used to measure the importance or activation strength of the input neurons; is the pruning mask, denotes the layer index, i.e., is the pruning mask of the th layer. The pruning mask can be a binary matrix with the same dimension as the weight matrix.

[0088] (1.2) Weight update scheme based on ADMM

[0089] After the pruning mask is determined, the remaining weights (i.e., the weights that are not pruned) need to be updated to minimize the reconstruction error. Suppose a set of calibration inputs and the pruning mask are known, and is sought such that the reconstruction error is minimized. Here, denotes element-wise multiplication (Hadamard product); is the weight matrix before pruning in the th layer.

[0090] It can be found that when a set of weights is fixed to zero, the effective weight matrix forms a convex set . In other words, i.e., the following constrained optimization problem is being solved (the subscript is omitted for clarity):

[0091]

[0092] where is the calibration input, which can be a batch of training data, for example; is the original weight matrix; is the updated weight matrix. The reconstruction error measures the difference between the pruned output ( ) and the original output ( ) using the L2 norm square. The constraint of the objective function is that the pruned weights must be zero, i.e. the constraint of the objective function is:

[0093]

[0094] where is the mask matrix. This constraint is used to ensure that the positions in the mask matrix that are 0 (pruned weights) must be 0 in .

[0095] Since the objective function is also convex, ADMM (Alternating Direction Method of Multipliers) can be used to solve the optimization problem. The objective function is denoted as and an indicator function is used, which takes the value 0 when , otherwise . The following iterative process is obtained:

[0096]

[0097]

[0098]

[0099] where is the original objective function; is the augmented Lagrangian term of ADMM; is the multiplication parameter, and ; is the auxiliary variable; is the Lagrange multiplier; is the step number of the current iteration; represents the step number of the next iteration; denotes the projection to the constraint set , i.e. the positions in the mask that are 0 must be 0.

[0100] And updating is only a projection to the set of valid matrices, so:

[0101]

[0102] This means that will be forced to be 0 at the positions in the mask that are 0, and the other positions remain unchanged. That is, is forced to satisfy the constraint, ensuring that the pruned weights are zero.

[0103] renew The process is very similar to ridge regression and can be calculated as follows:

[0104]

[0105] in, It is the input correlation matrix; The identity matrix; the inverse matrix Ensure the issue is resolved. (Specific details will be updated later.) At that time, through and It balances minimizing reconstruction error with satisfying constraints.

[0106] For a fixed calibration input and mask The above calculation process will converge to the optimal solution with updated weights.

[0107] (1.3) Pruning layer by layer

[0108] This application also relates to an algorithm extension that, for each layer of weights, progressively prunes more and more weights while simultaneously calculating weight updates. This application employs a three-stage sparse scheduling method, where the first... The sparsity of the step is calculated as follows ,in, It is the first Sparsity of the step (i.e., the proportion of weights that are pruned). It is the final sparsity. This represents the number of sparsification steps. In each step, [the sparsification process is repeated]. Each weight is set to zero, and then ADMM weight updates continue until all layers are traversed to obtain a multimodal large model with weight pruning.

[0109] Specifically, at each layer, the pruning process is divided into... Proceed step by step, for each step :

[0110] 1. Calculate the current sparsity Sparsity from 0 ( hour ) began to gradually increase to ( hour cubic function This results in slow sparsity growth in the initial steps and rapid growth in the later stages, allowing the model to gradually adapt to pruning.

[0111] 2. Based on the current sparsity Use Wanda rules to select the weights (i.e., masks) to be pruned. The number of positions corresponding to the center 0 .

[0112] 3. Apply ADMM iterative updates to the weights to minimize reconstruction error.

[0113] Layer by layer: Repeat this process for each layer of the model independently, ensuring that the weights are optimized after pruning each layer before moving on to the next.

[0114] (2) Text Parsing

[0115] Step One: Image Extraction and OCR Recognition

[0116] Extract all images from the bid file (DOCX file) and perform OCR recognition on the images to obtain the text content within. Format the OCR results into a list, with each element containing detailed information about each image, to ensure that the image content (such as seals, certificate text, etc.) in the qualification certificate file can be recognized and utilized by the system.

[0117] Step Two: Table Data Extraction

[0118] Open the DOCX file and traverse all block elements to extract table data. Collect the row data of each table into a list, and finally format all table data into a list, with each element being a row of data from a table. By parsing the table structure in DOCX, extracting row and column data, and converting them into a structured list, key information such as financial data and qualification lists can be preserved.

[0119] Step Three: HTML Content Parsing

[0120] Use BeautifulSoup (a library for parsing HTML documents) to parse the HTML content generated from the DOCX file, extracting paragraphs and headings. Traverse the tags, collect paragraph, image, and table elements, and format them into a dictionary list. Create an index based on the content and update the element tags by embedding matches. By establishing the logical structure of the document (such as the association between headings and content), provide a basis for segmentation and indexing.

[0121] Step Four: Content Segmentation

[0122] Traverse the HTML element list, using headings (h tags) as delimiters to divide the content into multiple parts. Each part contains headings, content, paragraphs, tables, and images, and format all parts into a list, with each element being detailed information about a part. By dividing long documents into semantically coherent units, precise positioning and contextual association retrieval are facilitated.

[0123] (3) Building a Multi-Dimensional Searcher

[0124] For example Figure 4As shown, this application constructs a search tool that includes three dimensions: text block dimension, heading dimension, and paragraph dimension, as detailed below:

[0125] Dimension 1: Text Block Search

[0126] The core of text block search lies in how to efficiently find the correct text blocks from a collection of text blocks. First, an embedding vector of the text blocks is stored by creating a FAISS index for fast searching. When a user queries, the query text is first converted into an embedding vector, and then the FAISS index is used to search for the text blocks most relevant to the query vector. To improve search accuracy, results containing tables or images can be prioritized, depending on the user's needs.

[0127] During the search process, employing a contextual expansion strategy can provide further contextual information beyond the returned relevant results, helping users better understand the search results. This expansion can be achieved by expanding text blocks upwards and downwards, ensuring users receive more comprehensive information.

[0128] Dimension Two: Title Search

[0129] Title search primarily involves searching a collection of titles based on the user's query string to find the most relevant titles and return the related paragraphs. When performing a title search, the query text is first converted into an embedding vector, and then the FAISS index of the titles is loaded. By searching for the most relevant titles, the corresponding document segments can be quickly located, and the user can choose whether to prioritize returning results containing tables or images, depending on their needs. When returning results, in addition to the title, contextual information about the relevant paragraphs is also provided to help the user better understand the content of the title.

[0130] Dimension 3: Paragraph Search

[0131] Paragraph search focuses on searching within a collection of paragraphs to find those most relevant to the user's query. Similar to title search, paragraph search also requires converting the query text into an embedding vector and utilizing the FAISS index for searching. During the search process, options can be set to prioritize returning paragraphs containing tables or images, and contextual expansion can be used to enhance the search and ensure the user receives more comprehensive information.

[0132] (4) Verification of the existence of qualification certificates

[0133] like Figure 5 As shown, this application combines three search strategies and then utilizes a multimodal large model to determine whether the required qualification documents exist. Specifically:

[0134] First, a matching search is performed, a retrieval tool is called, and a preliminary search result is obtained using the provided query string. If the preliminary search does not find a match, a secondary extended search is performed, increasing the search range. In the existence verification scheme, according to the difference of file types, diversified search strategies are adopted to ensure the existence of qualification certificate files. The authorization is verified by block search, and the business license and financial report files use the title search strategy, and allow the use of the qualification commitment letter as a supplement. No violation record files also use block search strategy. For other proof materials not belonging to specific categories, use paragraph search as a bottom-up strategy.

[0135] Next, the multimodal large model is used to check the matching of the search result with the required qualification certificate file, and the multimodal large model is called to determine whether the searched qualification certificate file image and the required qualification certificate file are consistent. If they are consistent, it means that the qualification certificate file exists. The search result and the existence verification result are recorded.

[0136] If the preliminary search does not match, the method will perform a secondary extended search and check the matching again. If the secondary search result matches, the audit result is recorded.

[0137] Finally, if the secondary search still does not match, the language large model is used to determine whether there is a position jump, i.e., the position of the certificate file and the title is inconsistent, such as existing in the annex or appendix. If there is a position jump, a jump search is performed to update the material information. Then, the multimodal large model is used to determine whether it matches.

[0138] In some embodiments, when performing existence verification, it can include:

[0139] 1. Perform a matching search: use a multi-dimensional retriever (text block, title, or paragraph search) to perform an initial search according to the file type. For example: authorization uses block search; business license and the like use title search; other files use paragraph search.

[0140] 2. Check matching: use a multimodal large model to determine whether the search result matches the required qualification certificate file. If it matches, record the existence result (existence).

[0141] 3. Secondary extended search: If the initial search does not match, expand the search range (such as context expansion) to perform a secondary search.

[0142] 4. Check matching again: use a multimodal large model to determine whether the secondary search result matches. If it matches, record the existence result (existence).

[0143] 5. Position jump determination: If the secondary search still does not match, use a language large model to determine whether the file has a position jump (such as an annex or appendix).

[0144] 6. Jump search: If there is a location jump, perform a jump search, update the material information, and finally check the matching using the multimodal large model.

[0145] 7. Record results: Record the existence result (existence or non-existence) according to the final matching.

[0146] The multiple search mechanism improves the accuracy of matching, ensuring that the existence of the certificate can be effectively verified.

[0147] (5) Accuracy verification of qualification certificate

[0148] As shown in Figure 6 , the following four aspects of the audit plan are introduced in detail: information consistency, content integrity, date information, and whether to seal.

[0149] (5.1) Information consistency verification: The multimodal large model can process and analyze data of different modalities, ensuring the consistency of information between different modalities. Using the multimodal large model, extract the basic information in the qualification certificate, and judge whether it is consistent with the basic information of the bidder and the tenderer. Through the reasoning of the multimodal model, the basic information in the file can be effectively verified for accuracy.

[0150] (5.2) Content integrity verification: Use the multimodal large model to determine whether the content in the qualification certificate is complete, preventing content loss. This audit mechanism can check whether the file content is complete and avoid missing necessary information.

[0151] (5.3) Date information verification: The multimodal large model can analyze the date information in the file, and then compare the date requirements in the tender document with the date information in the file. Through analysis and verification of date information, it can be ensured that the date in the document meets the requirements in the tender document.

[0152] (5.4) Whether to seal verification (qualification certification mark verification): According to the qualification certification and evaluation criteria of inspection and testing agencies, inspection and testing reports or certificates should be marked with qualification certification marks, and special inspection and testing seals should be applied when applicable. The multimodal large model can be used to detect whether the image file has been sealed to ensure the formal and effective nature of the material.

[0153] Through these audit plans, combined with the technical advantages of the multimodal large model, the accuracy, integrity and compliance of the document can be fully ensured.

[0154] In some embodiments, the bid document verification method provided by the present application combines the layer-by-layer progressive pruning technique of ADMM to verify the bid qualification documents, which is suitable for bid auditing scenarios and specifically includes:

[0155] Bid document parsing: First, extract all image and table data from the DOCX file. For example, for a bid document containing multiple qualification documents, use OCR technology to recognize the text in the image and extract all table data.

[0156] Multi-dimensional search system construction: Next, construct a multi-dimensional search system. Input the extracted text and table data into the system, create an index to optimize the relevance of the search. The specific steps are as follows:

[0157] Break down large text into smaller paragraphs to optimize the relevance of retrieving content from the vector database.

[0158] Search strategy implementation: Implement chunk search, title search, and paragraph search strategies. For example, when verifying the existence of a qualification document, the system will search based on the title and content information of the file, and determine the existence of the file through a multi-modal large model. The specific steps are as follows:

[0159] Store the embedding vectors of the text blocks by creating a FAISS index for fast retrieval. When a user queries, convert the query text into an embedding vector, then use the FAISS index to search for the most relevant text blocks to the query vector. Use the context expansion strategy to further provide context information based on the relevant results. Then, combine the multi-modal large model to determine the existence of the qualification document

[0160] Qualification document accuracy verification: Finally, combine the multi-modal large model to audit the basic information consistency, content integrity, date compliance, and whether the document is stamped of the qualification document. For example, the system will analyze the date information in the file to ensure that the date in the qualification document meets the requirements in the bid document. The specific steps are as follows: Use the multi-modal large model to extract the basic information in the qualification document and determine whether it matches the basic information of the bidder and the tenderer; determine whether the content in the qualification document is complete to prevent content loss; analyze the date information in the file, then compare the date requirements in the bid document with the date information in the file; detect whether the document has been stamped to ensure the formality and effectiveness of the qualification material.

[0161] Through the above scheme, efficient and accurate bid qualification document auditing is achieved, greatly saving labor costs and improving auditing accuracy. Finally, the auditing results are returned to the user.

[0162] The bidding document inspection method provided in the application fuses a multi-modal large model and a multi-dimensional text retrieval technology for bid qualification certificate file review. The multi-modal large model comprehensively analyzes the text and image content of the file, accurately judges information accuracy, date compliance, and key points such as seals. At the same time, a three-dimensional retrieval tool for text blocks, titles, and paragraphs is constructed, and relevant text blocks are quickly located with the help of FAISS indexing. This scheme significantly improves review efficiency and accuracy and effectively avoids the omissions of manual review. In the bidding document inspection method provided in the application, the layer-by-layer progressive pruning technology based on ADMM realizes the lightweight of the multi-modal large model. The Wanda rule is used to select a pruning mask, the ADMM is used to iteratively update the weight, and a three-time sparsity scheduling method is used for step-by-step pruning. This technology effectively reduces the complexity of the model, improves the inference efficiency, ensures the performance of the model, and makes the multi-modal large model more efficient in bid qualification certificate file review.

[0163] In addition, in the bidding document inspection method provided in the application, the multi-modal large model and the multi-dimensional search strategy are fused in the existence and accuracy inspection of the qualification certificate file. The existence inspection is matched and searched first and then expanded, different strategies are used according to the file type, and if no match is found, the position is judged to jump and search. The accuracy inspection uses the multi-modal large model to review the consistency of the basic information and other four aspects. This method improves the matching accuracy and comprehensively ensures the existence and accuracy of the certificate.

[0164] The bid qualification certificate file review method provided in the application combined with the layer-by-layer progressive pruning technology based on ADMM has the following technical effects:

[0165] Improve the efficiency and accuracy of review: by using the multi-dimensional retrieval system and the multi-modal large model technology, the efficiency and accuracy of the bid qualification certificate file review are significantly improved. Compared with traditional manual review, this automatic review method greatly reduces the time for manual positioning and verification, and at the same time, through advanced algorithms, it ensures the high accuracy of the review and avoids the inaccuracy caused by neglecting details in manual review.

[0166] Improve the inference efficiency of the large model by combining the model pruning technology: by combining the layer-by-layer progressive pruning technology based on ADMM, the multi-modal large model is pruned, the inference efficiency of the multi-modal large model is improved, and the efficiency of the bid qualification certificate file review is improved.

[0167] Enhance search capability and content verification: by combining text retrieval and multi-modal large model technology, not only the speed and accuracy of the search are improved, but also the in-depth verification of the content of the qualification certificate file is enhanced, ensuring the accuracy and effectiveness of the certificate and the effectiveness of the time.

[0168] Reduction of labor costs and improvement of user experience: the automatic auditing scheme of the present application greatly saves labor costs and reduces the dependence on human resources. At the same time, by providing faster and more accurate auditing results, the user experience is improved. This method not only improves the fairness, impartiality and transparency of the bidding process, but also ensures the quality and efficiency of enterprise procurement activities.

[0169] It should be understood that although the steps in the flowchart involved in each of the above embodiments are displayed in sequence according to the direction of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0170] Based on the same inventive concept, the embodiments of the present application also provide a bid file inspection device for implementing the above-mentioned bid file inspection method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more bid file inspection device embodiments provided below can refer to the limitations of the bid file inspection method in the above text, which will not be repeated here.

[0171] In one exemplary embodiment, as shown in Figure 7 A bid file inspection device 700 is provided, comprising a bid file parsing module 702, a search processing module 704, an existence verification module 706, and an accuracy verification module 708, wherein:

[0172] The bid file parsing module 702 is configured to obtain a bid file, parse the bid file, and obtain multi-modal data;

[0173] The search processing module 704 is configured to obtain a query text, determine a qualification certificate file type corresponding to the query text, perform a search based on the multi-modal data according to a search strategy adapted to the qualification certificate file type, and obtain a search result;

[0174] The existence verification module 706 is configured to perform existence verification on the bid file according to the search result and the qualification certificate file type by using a multi-modal large model, and obtain an existence verification result.

[0175] The accuracy verification module 708 is configured to perform accuracy verification on the qualification certificate file by the multi-modal large model if the existence verification result indicates that the qualification certificate file is included in the bid document, and obtain an accuracy verification result.

[0176] In some embodiments, the search processing module 704 is further configured to extract keywords from the query text, determine the qualification certificate file type according to the keywords, determine at least one search strategy suitable for the qualification certificate file type, determine a target search set from a candidate search set corresponding to the multi-modal data according to the at least one search strategy, and perform at least one search in the target search set according to the search strategy to obtain a search result.

[0177] In some embodiments, the search set construction module is further configured to construct a text block set, a title set and a paragraph set based on the multi-modal data, the text block set corresponds to a text block search strategy, the title set corresponds to a title search strategy, and the paragraph set corresponds to a paragraph search strategy, and obtain the candidate search set corresponding to the multi-modal data according to the text block set, the title set and the paragraph set.

[0178] In some embodiments, the existence verification module 706 is further configured to determine the multi-modal large model, and generate an existence verification prompt word according to the search result and the qualification certificate file type; and instruct the multi-modal large model to perform existence verification on the bid document based on the search result and the qualification certificate file type through the existence verification prompt word, and obtain an existence verification result.

[0179] In some embodiments, the accuracy verification module 708 is further configured to obtain the qualification certificate file belonging to the qualification certificate file type in the bid document according to the search result, and perform information consistency verification, content integrity verification, date information verification and qualification recognition symbol verification on the qualification certificate file by the multi-modal large model respectively, and obtain an accuracy verification result.

[0180] In some embodiments, the weight pruning module is further configured to determine an initial multi-modal large model, and perform at least one step of weight pruning on the layer structure included in the initial multi-modal large model layer by layer in the order of the layer structure to obtain the multi-modal large model.

[0181] In some embodiments, the weight pruning module is further configured to determine a target layer structure in an order of the layer structures included in the initial multi-modal large model; perform at least one step of weight pruning on the weights of the target layer structure, and obtain target weights of the target layer structure after pruning according to output weights of the last step of weight pruning; wherein in each step of weight pruning, a number of pruned weights of the current step is determined according to a sparsity of the current step; the pruned weights of the current step are determined from the initial weights of the current step according to the number of pruned weights; the output weights of the current step are determined based on the pruned weights, and in a case that the current step is not the last step, the output weights of the current step are taken as the initial weights of the next step for the next step of weight pruning; and the multi-modal large model is obtained based on the target weights of the layer structures included in the initial multi-modal large model after pruning.

[0182] The modules in the bid document inspection apparatus can be implemented wholly or partially by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by a processor to perform operations corresponding to the modules.

[0183] In an exemplary embodiment, a computer device, which can be a server or a terminal, is provided, and an internal structure diagram of the computer device can be as shown in Figure 8 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store various data involved in the bid document inspection method. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a bid document inspection method.

[0184] Those skilled in the art can understand that Figure 8 The structure shown in the above is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0185] In an embodiment, a computer device is also provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0186] In an embodiment, a computer readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the steps in the above method embodiments.

[0187] In an embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps in the above method embodiments.

[0188] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0189] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0190] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0191] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for inspecting tender documents, characterized in that, The method includes: Obtain the tender documents, parse the tender documents, and obtain multimodal data; Obtain the query text, determine the type of qualification certificate document corresponding to the query text, and perform a search based on the multimodal data according to the search strategy adapted to the qualification certificate document type to obtain the search results; Using a multimodal large model, an existence test is performed on the tender document based on the search results and the type of qualification certificate, and the existence test result is obtained. If the existence test result indicates that the tender document includes a qualification certificate of the type mentioned above, the accuracy of the qualification certificate is tested using the multimodal large model to obtain the accuracy test result.

2. The method according to claim 1, characterized in that, The step involves determining the type of qualification certificate document corresponding to the query text, performing a search based on the multimodal data according to a search strategy suitable for that qualification certificate document type, and obtaining search results, including: Extract keywords from the query text, and determine the type of qualification certificate document based on the keywords; Determine at least one search strategy that is compatible with the type of qualification certificate document; According to the at least one search strategy, a target search set is determined from the candidate search set corresponding to the multimodal data, wherein the candidate search set is constructed based on the multimodal data for different search strategies; According to the search strategy, at least one search is performed in the target search set to obtain the search results.

3. The method according to claim 2, characterized in that, The method further includes: Based on the multimodal data, a text block set, a title set, and a paragraph set are constructed respectively; the text block set corresponds to a text block search strategy, the title set corresponds to a title search strategy, and the paragraph set corresponds to a paragraph search strategy. Based on the set of text blocks, the set of titles, and the set of paragraphs, a candidate search set corresponding to the multimodal data is obtained.

4. The method according to claim 1, characterized in that, The existence check of the tender documents is performed using a multimodal large model based on the search results and the type of qualification certificate, and the existence check results are obtained, including: Identify the multimodal large model and generate existence verification prompts based on the search results and the type of qualification certificate. The existence check prompts instruct the multimodal large model to perform an existence check on the tender document based on the search results and the qualification document type, and obtain the existence check results.

5. The method according to claim 1, characterized in that, The accuracy verification of the qualification certificate document using the multimodal large model, and the resulting accuracy verification results, include: Based on the search results, the qualification documents belonging to the qualification document type in the tender documents are obtained; Using the multimodal large model, the information consistency check, content completeness check, date information check, and qualification mark check are performed on the qualification certificate documents to obtain the accuracy check results.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Determine the initial multimodal large model; For the layer structure included in the initial multimodal large model, weight pruning is performed layer by layer in the order of the layer structure, with at least one step, to obtain the multimodal large model.

7. The method according to claim 6, characterized in that, The step of performing at least one weight pruning step on each layer in the order of the layer structure included in the initial multimodal large model to obtain the multimodal large model includes: The target layer structure is determined according to the order of the layer structures included in the initial multimodal large model; The weights of the target layer structure are pruned at least once. The target weights of the target layer structure after pruning are obtained based on the output weights of the last pruning step. In each pruning step, the number of pruning weights for the current step is determined based on the sparsity of the current step. The pruning weights for the current step are determined from the initial weights of the current step based on the number of pruning weights. The output weights for the current step are determined based on the pruning weights. If the current step is not the last step, the output weights of the current step are used as the initial weights for the next step of weight pruning. The multimodal large model is obtained based on the target weights of each pruned layer structure included in the initial multimodal large model.

8. A tender document inspection device, characterized in that, The device includes: The bid document parsing module is used to obtain bid documents, parse the bid documents, and obtain multimodal data. The search processing module is used to acquire query text, determine the type of qualification certificate document corresponding to the query text, perform a search based on the multimodal data according to the search strategy adapted to the qualification certificate document type, and obtain search results. The existence verification module is used to perform an existence verification on the tender document based on the search results and the type of qualification certificate document using a multimodal large model, and obtain the existence verification result. The accuracy verification module is used to perform an accuracy verification on the qualification certificate document using the multimodal large model when the existence verification result indicates that the tender document includes a qualification certificate document of the type mentioned above, and to obtain an accuracy verification result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images