Bidding scene-oriented cross-document semantic consistency verification system

Abstract

The invention discloses a cross-document semantic consistency verification system and method oriented to bidding and tendering scenes, and relates to the technical field of data processing, and the method comprises the steps: collecting bidding and tendering document data, and carrying out the preprocessing; analyzing the key degree of a semantic unit in the bidding document data to construct a term key index; analyzing the risk degree of the key semantic unit of the bidding document based on the term key index, and constructing an anchor point risk entropy; analyzing risk sensitivity degrees of public key semantic units of the bidding and tendering document data and the comparison document data based on an anchor point risk entropy, and constructing a cross-document risk factor; and improving a comparison loss function of a twin neural network based on the cross-document risk factor, and realizing semantic consistency verification of different bidding document data. According to the method, the problem that the identification precision of a system on high-risk errors is affected due to the fact that the distinguishing degree of the model on key inconsistency and non-key inconsistency in the training process is low is solved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a cross-document semantic consistency verification system and method for bidding scenarios. Background Technology

[0002] Cross-document semantic consistency verification in bidding scenarios refers to the automated comparison and analysis of key information, technical requirements, business terms, or functional descriptions across multiple documents involved in the project bidding process. This ensures that there are no semantic contradictions, ambiguities, or omissions between different documents. Its core functions are: significantly improving the efficiency of bidding document preparation and review; reducing the error rate caused by negligence in manual verification; avoiding legal or commercial disputes arising from semantic inconsistencies in clauses; and ensuring that bidding activities meet predetermined requirements and standards. It has irreplaceable value in improving project quality and reducing project risks.

[0003] In existing technologies, document semantic consistency verification typically uses Siamese neural networks as the basic framework for contrastive learning, combined with standard contrastive loss functions to train models and analyze cross-document semantic consistency. However, bidding documents are characterized by high industry-specificity, numerous technical terms, and the coexistence of high-risk inconsistencies in key information and low-risk inconsistencies in non-key information. Directly using existing standard loss functions to verify the consistency of bidding documents fails to effectively distinguish and weight key and non-key semantic information. This results in existing algorithms' loss functions being unable to effectively prioritize and weight high-risk semantic inconsistencies in bidding scenarios, leading to low model differentiation between key and non-key inconsistencies during training, thus affecting the system's accuracy in identifying high-risk errors.

[0004] Therefore, this invention provides a cross-document semantic consistency verification system and method for bidding scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide a cross-document semantic consistency verification system and method for bidding scenarios, so as to solve the existing problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a cross-document semantic consistency verification system for bidding scenarios, comprising: The data acquisition and preprocessing module is used to collect bidding document data and perform preprocessing. The Terminology Key Index Construction Module is used to analyze the keyness of semantic units in bidding document data and construct terminology key indices. The semantic risk analysis module is used to analyze the risk level of key semantic units in bidding documents based on terminology key indices and to construct anchor risk entropy. The cross-document risk factor acquisition module is used to analyze the risk sensitivity of common key semantic units in the bidding document data and the comparison document data based on anchor risk entropy and to construct cross-document risk factors. The consistency verification module is used to improve the contrastive loss function of the Siamese neural network based on the cross-document risk factor to realize semantic consistency verification of different bidding document data.

[0007] A further improvement of the present invention is that the terminology key index construction module includes a corpus construction submodule and a terminology key index adaptive calculation submodule; The corpus construction submodule is used to construct the corpus based on the i-th semantic unit in the bidding document. The TF-IDF value of the frequency of appearance in the bidding documents is calculated, and news, reports and articles are collected through relevant API interfaces. After preprocessing according to the above steps, a general domain corpus is formed. The terminology key index adaptive calculation submodule is used to sort semantic units according to the order of the bidding document content. The set of the n nearest semantic units is called the nearest neighbor set. The frequency of each semantic unit in the bidding document data is counted. The nearest neighbor set is treated as a whole, and its frequency in the bidding document data is counted to obtain the term key index.

[0008] A further improvement of this invention is that the calculation formula for the terminology key index is expressed as follows: ; in This represents the terminology key index of the i-th semantic unit in the bidding document. and Representing semantic units respectively The TF-IDF values ​​in the bidding documents and the corpus in the general domain, where n represents the number of semantic units in the nearest neighbor set. , and Representing semantic units respectively The first, second, and nth semantic units in the nearest neighbor unit set, This indicates the frequency of the nearest neighbor set in the bidding document data. Represents semantic units The frequency of the j-th semantic unit in the nearest neighbor set in the bidding document data.

[0009] A further improvement of this invention is that the construction process of the anchor point risk entropy includes: for the bidding document data, calculating the term key index of each semantic unit, and using it as the input of the Otsu threshold segmentation method, outputting the segmentation threshold of the term key index; the set of semantic units that are greater than or equal to the segmentation threshold and deduplicated is taken as the key semantic unit set, and the elements in the set are denoted as key semantic units; for the a-th key semantic unit, whenever it appears in the bidding document data, the sequence of the last M semantic units at any time it appears is denoted as the key value sequence, and the information entropy is calculated according to the occurrence probability of all key value sequences corresponding to the a-th key semantic unit.

[0010] A further improvement of this invention is that the method for calculating the anchor point risk entropy includes: ; in Let represent the anchor risk entropy of the a-th key semantic unit in the bidding document data, and let ln() represent the logarithmic function with the natural constant as the base, used to smooth the data within the parentheses. This indicates the number of occurrences of the a-th key semantic unit in the bidding document data. This represents the information entropy of all key value sequences corresponding to the a-th key semantic unit in the bidding document data.

[0011] A further improvement of this invention is that the cross-document risk factor acquisition module includes a public key semantic unit acquisition submodule and a cross-document risk factor calculation submodule; the public key semantic unit acquisition submodule is used to extract public key semantic units by comparing document data; The cross-document risk factor calculation submodule is used to extract the key value sequence of all common key semantic units in the bidding document data through the terminology key index construction module and the semantic risk analysis module. The set formed is denoted as the key value set. The key value set of the qth common key semantic unit in the comparison document data is extracted in the same way as described above to obtain the cross-document risk factor.

[0012] A further improvement of the present invention is that the public key semantic unit acquisition submodule records the bidding document data used for comparison with the bidding document data as comparison document data, and obtains the key semantic sets of the bidding document data and the comparison document data through the terminology key index construction module and the semantic risk analysis module respectively, and takes the key semantic units with completely identical text content in the two key semantic sets as public key semantic units.

[0013] A further improvement of this invention is that the calculation formula for the cross-document risk factor is expressed as follows: ; in This represents the cross-document risk factor of the q-th common key semantic unit in the bidding document data and the comparison document data, where Sig() represents the sigmoid function. and Let Jac() represent the anchor risk entropy of the q-th common key semantic unit in the bidding document data and the comparison document data, respectively, and let Jac() represent the Jaccard distance function used to calculate the Jaccard distance between the two sets. and These represent the set of key values ​​for the q-th common key semantic unit in the bidding document data and the comparison document data, respectively.

[0014] A further improvement of this invention is that the consistency verification module includes: for non-public key semantic units, the dynamic adjustment factor is preset to a base value; for public key semantic units, the cross-document risk factor of the public key semantic units in the bidding document data and the comparison document data is used as a dynamic adjustment factor to adjust the contrast loss function in the neural network; the product of the dynamic adjustment factor and the contrast loss function is used as the adjusted contrast loss function; the bidding document data and the comparison document data are used as the input of the Siamese neural network; the adjusted contrast loss function is used as the loss function of the Siamese neural network; the semantic difference distance between the bidding document data and the comparison document data is output; and the cross-document semantic consistency verification result is obtained through a preset distance threshold.

[0015] On the other hand, this invention provides a cross-document semantic consistency verification method for bidding scenarios, the specific steps of which include: S1. Collect bidding document data and preprocess it; S2. Analyze the criticality of semantic units in bidding document data to construct a terminology criticality index; S3. Analyze the risk level of key semantic units in bidding documents based on key terminology indices and construct anchor risk entropy; S4. Analyze the risk sensitivity of common key semantic units in the bidding document data and the comparison document data based on anchor point risk entropy, and construct cross-document risk factors; S5. Based on the cross-document risk factor, improve the contrastive loss function of the Siamese neural network to realize semantic consistency verification of different bidding document data.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention first assesses the criticality of semantic units in bidding document data through a terminology key index construction module, and assesses the risk level of key semantic units in bidding documents through a semantic risk analysis module. It then calculates the cross-document risk factor of common key semantic units and dynamically improves the contrastive loss function of the Siamese neural network. This allows for a stronger penalty for high-risk inconsistencies during the training process of the Siamese neural network, ultimately enabling the trained verification system to effectively distinguish significant risk differences and improve the accuracy of cross-document semantic consistency verification in bidding scenarios. Attached Figure Description

[0017] Figure 1 This is a framework diagram of a cross-document semantic consistency verification system for bidding scenarios according to the present invention; Figure 2 This is a flowchart of a cross-document semantic consistency verification method for bidding scenarios according to the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0019] The term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone.

[0020] Example 1 Figure 1 This embodiment illustrates a framework diagram of a cross-document semantic consistency verification system for bidding scenarios, including: The data acquisition and preprocessing module is used to collect bidding document data and perform preprocessing. In the verification system, relevant data is collected through a data acquisition module deployed on a local server or cloud server. This includes: automatically collecting data such as bidding documents and winning bid announcements from data sources such as bidding websites and enterprise procurement platforms, and saving the collected data to plain text txt files; and accessing the enterprise's internal document management system through authorized application API interfaces to obtain document data such as technical specifications, tender documents, and draft contracts from historical projects.

[0021] After collecting the raw document data, relevant preprocessing is required to obtain clean plain text data. Taking one document as an example: OCR technology is used to uniformly recognize documents of different formats such as PDF, DOCX, or XML as plain text txt format; regular expressions are used to identify and remove irrelevant information such as headers, footers, page numbers, and watermarks from the document; using punctuation marks as delimiters, the plain text is segmented into independent sentence units; for each sentence unit, the jieba word segmentation library in Python is used to divide it into several semantic units; all sentence units in the document are processed in the same way as above to obtain the semantic unit set of the document, which is used as the preprocessed bidding document data.

[0022] The Terminology Key Index Construction Module is used to analyze the keyness of semantic units in bidding document data and construct terminology key indices. The unique characteristic that distinguishes bidding documents from general texts lies in the highly uneven distribution of their value and risk. These documents contain numerous key semantic units that are crucial to the success or failure of the project, such as price, schedule, and core technical indicators. These terms typically exist in structured or semi-structured forms as keywords, numerical values, or specific phrases. When comparing documents across different documents, subtle changes in these key semantic units can pose significant business risks. For example, bidding document 1 might state a delivery period of 90 days, while bidding document 2 states a delivery period of 90 working days. Differences in other descriptive or modifying statements have a smaller impact; for instance, bidding document 1 might use the phrase "advanced technology," while bidding document 2 might use "leading technology." Currently, the common practice of directly comparing the semantic units of two documents globally is insufficient to effectively distinguish between these two types of differences.

[0023] Based on the above analysis, this application constructs a term key index through a corpus construction submodule and a term key index adaptive calculation submodule. This index is used to reflect the criticality of semantic units in bidding document data. The construction process of the term key index is as follows: For ease of description, this application uses any bidding document as an example to describe the corpus construction submodule; the i-th semantic unit in the bidding document... For example, based on semantic units The frequency of occurrences in the bidding document was calculated to obtain TF-IDF values. News, reports and articles were collected through relevant API interfaces and other means. After preprocessing according to the above steps, a general domain corpus was formed.

[0024] Following the order of the bidding documents, run the terminology key index adaptive calculation submodule to calculate semantic units. The set of the n nearest semantic units is denoted as the nearest neighbor set. The frequency of each semantic unit in the nearest neighbor set within the bidding document data is counted. The nearest neighbor set is then treated as a whole, and its frequency within the bidding document data is also counted. In this application, n is taken as 3, but the implementer can choose a value based on the actual situation. For example, if n is 3 in this application, and the nearest neighbor set is {delivery, cycle, one day}, and there are 100 semantic units in the bidding document data, the frequency of {delivery, cycle, one day} is 3. Additionally, {delivery, cycle, two days} appears twice, and {delivery, cycle, ten days} appears once. Therefore, the frequency of the nearest neighbor set {delivery, cycle, one day} in the bidding document data is: .

[0025] Based on the above processing steps, the calculation method for the terminology key index in this application is as follows: in This represents the terminology key index of the i-th semantic unit in the bidding document. and Representing semantic units respectively In the TF-IDF values ​​of the bidding documents and general domain corpora, the 1 in both the numerator and denominator is to improve robustness and avoid extreme values, and n represents the number of semantic units in the nearest neighbor set. , and Representing semantic units respectively The first, second, and nth semantic units in the nearest neighbor unit set, This indicates the frequency of the nearest neighbor set in the bidding document data. Represents semantic units The frequency of the j-th semantic unit in the nearest neighbor set in the bidding document data.

[0026] In the formula Indicating key weights: In bidding document data, if semantic units... The larger the TF-IDF value in the bidding document data compared to the TF-IDF value in the general domain corpus, the better. The larger the value, the more likely the semantic unit is to belong to the professional or commonly used words in the bidding and tendering scenario. Therefore, it is more likely to be a keyword in the bidding and tendering scenario, and the corresponding key weight is larger. In the formula Indicating the solidification degree of semantic units: In bidding document data, if semantic units... The greater the probability of a set of nearest neighbor units appearing as a whole, the more likely a word combination formed by that set of nearest neighbor units is a professional term in the bidding scenario, and the greater its cohesion. Therefore, the calculated semantic unit cohesion is greater. In summary, if semantic units The greater the key weight and the greater the semantic unit solidification, the more likely the semantic unit is to be a semantic unit in the core position of the key information structure. In the bidding scenario, the more critical it is, the greater the term key index calculated by multiplying the key weight and the semantic unit solidification.

[0027] The semantic risk analysis module is used to analyze the risk level of key semantic units in bidding documents based on terminology key indices and to construct anchor risk entropy. In bidding and tendering scenarios, key semantic units in bidding and tendering documents possess both an anchoring effect for crucial information and variability in their value content. For example, key semantic units such as "quotation," "cycle," or "warranty" exhibit an anchoring effect in bidding and tendering documents, meaning the content following these key semantic units is variable and important. For instance, the value following the semantic unit "bid quotation" could be 5 million yuan, 8 million yuan, or any other arbitrary data. This "anchor-value" structure, along with the high degree of uncertainty in the "value" component, represents the core verification item in bidding and tendering documents most prone to high risk.

[0028] Therefore, this application constructs an anchor risk entropy based on the terminology key index to reflect the risk level of key semantic units in bidding documents. The construction process of the anchor risk entropy is as follows: For the bidding document data, the term key index of each semantic unit is calculated according to the above steps, and used as the input of the Otsu threshold segmentation method. The output is the segmentation threshold of the term key index. The set of semantic units that are greater than or equal to the segmentation threshold and are deduplicated is taken as the key semantic unit set, and the elements in this set are denoted as key semantic units. For the a-th key semantic unit, whenever it appears in the bidding document data, the sequence of the next M semantic units at any given time is denoted as the key value sequence, and the information entropy is calculated based on the occurrence probability of all key value sequences corresponding to the a-th key semantic unit. The value of M is taken as 3 in this application, but can be selected according to the actual situation.

[0029] Based on the above processing steps, the anchor point risk entropy is calculated as follows: in Let represent the anchor risk entropy of the a-th key semantic unit in the bidding document data, and let ln() represent the logarithmic function with the natural constant as the base, used to smooth the data within the parentheses. This indicates the number of occurrences of the a-th key semantic unit in the bidding document data. This represents the information entropy of all key value sequences corresponding to the a-th key semantic unit in the bidding document data.

[0030] In the formula This represents the harmonizing term, reflecting the effectiveness of the a-th key semantic unit. The larger the value, the more times the key semantic unit appears in the bidding documents, the greater its weight when risks arise, and the larger the corresponding harmonic term is calculated. After identifying all key semantic units through the terminology key index of bidding document data, these key semantic units are used as important anchors in the bidding document data. The content following an important anchor is more likely to be a core risk item in the bidding document data. For the *a*th key semantic unit, if the subsequent core risk items are the same, the calculated information entropy is smaller. The smaller the value, the lower the corresponding risk level, resulting in a smaller calculated anchor point risk entropy; if the core risk items following important anchor points are different, the corresponding calculated information entropy will be larger, i.e. The larger the value, the greater the corresponding level of risk, resulting in a larger calculated anchor point risk entropy.

[0031] The cross-document risk factor acquisition module is used to analyze the risk sensitivity of common key semantic units in the bidding document data and the comparison document data based on anchor risk entropy and to construct cross-document risk factors; it includes a common key semantic unit acquisition submodule and a cross-document risk factor calculation submodule. When performing semantic consistency checks on different bidding document data, key semantic units with higher risk levels in the two bidding document data need to be given special attention. That is, when key semantic units with higher risk levels are inconsistent, such as the content in the bidding document data being "delivery cycle 90 days" and the content in the bidding document data to be compared being "delivery cycle 90 working days", a larger loss function value should be applied. During the training of the neural network, a stronger penalty signal is passed to the model, and the internal weight parameters are updated based on the gradient calculated by the loss function through the backpropagation algorithm. The attention to key semantic units with lower risk levels can be appropriately reduced. That is, when key semantic units with lower risk levels are inconsistent, such as the content in the bidding document data being "advanced technology" and the content in the bidding document data to be compared being "leading technology", a smaller loss function value should be applied. During the training of the neural network, a weaker penalty signal is sufficient to pass to the model.

[0032] Based on the above analysis, this application constructs a cross-document risk factor based on anchor risk entropy to reflect the risk sensitivity of common key semantic units in different bidding document data. The construction process of the cross-document risk factor is as follows: The public key semantic unit acquisition submodule records the bidding document data used for comparison with the bidding document data as the comparison document data, and obtains the key semantic sets of the bidding document data and the comparison document data respectively according to the above steps. The key semantic units with completely identical text content in the two key semantic sets are taken as the public key semantic units. The cross-document risk factor calculation submodule takes the q-th common key semantic unit as an example, extracts the key value sequence of all such common key semantic units in the bidding document data according to the above steps, and the set formed is denoted as the key value set. Then, it extracts the key value set of the q-th common key semantic unit in the comparison document data in the above manner.

[0033] Based on the above processing steps, the calculation method for the cross-document risk factor in this application is as follows: in This represents the cross-document risk factor of the q-th common key semantic unit in the bidding document data and the comparison document data. Sig() represents the sigmoid function, used to normalize the data within the parentheses. and Let Jac() represent the anchor risk entropy of the q-th common key semantic unit in the bidding document data and the comparison document data, respectively, and let Jac() represent the Jaccard distance function used to calculate the Jaccard distance between the two sets. and These represent the set of key values ​​for the q-th common key semantic unit in the bidding document data and the comparison document data, respectively.

[0034] In the formula This reflects the average risk level of the common key semantic unit in the two document data. The larger this item is, the more concentrated the risk of the common key semantic unit is, and the more likely it is to belong to the core verification item. Using it as the basic weight makes the model pay more attention to the common key semantic unit. In the formula This reflects the specific differences in the key value set corresponding to the q-th common key semantic unit in the bidding document data and the comparison document data. For example, for the "delivery period" semantic unit in the bidding document data, the key value set is {100,000 yuan, 200,000 yuan}, while in the comparison document data, the key value set for the "delivery period" semantic unit is {500,000 yuan, 800,000 yuan, 200,000 yuan}. This indicates that although there is some overlap in the specific value content of the two documents, other value content has significant differences, and the corresponding calculated... The larger.

[0035] In summary, based on and The calculated cross-document risk factor can reflect the risk sensitivity of common key semantic units in different bidding document data.

[0036] The consistency verification module is used to improve the contrastive loss function of the Siamese neural network based on the cross-document risk factor to realize semantic consistency verification of different bidding document data.

[0037] The cross-document risk factors of common key semantic units in the bidding document data and comparison document data are obtained through the above modules, and are used as dynamic adjustment factors to adjust the contrastive loss function in the neural network. For non-common key semantic units, the dynamic adjustment factor is preset to a base value, which is 0.1 in this application, but can be selected according to the actual situation.

[0038] The process of adjusting the contrastive loss function in the neural network through a dynamic adjustment factor is as follows: the product of the dynamic adjustment factor and the contrastive loss function is used as the adjusted contrastive loss function.

[0039] The bidding document data and the comparison document data are used as inputs to the Siamese neural network, and the adjusted comparison loss function is used as the loss function of the Siamese neural network. The output is the semantic consistency verification result of the bidding document data and the comparison document data, which is the semantic difference distance between the bidding document data and the comparison document data.

[0040] A preset distance threshold is used. If the semantic difference distance is greater than or equal to the threshold, it indicates that the semantics of the bidding document data and the comparison document data are inconsistent; if the semantic difference distance is less than the threshold, it indicates that the semantics of the bidding document data and the comparison document data are consistent. This is used as the verification result of cross-document semantic consistency. The preset distance threshold is set to 0.5 in this application, but can be selected according to actual conditions.

[0041] The threshold and weight settings can be set by default according to the present invention, or they can be set by those skilled in the art.

[0042] Example 2 Figure 2 This invention presents a flowchart of a cross-document semantic consistency verification method for bidding scenarios, based on the same inventive concept as Embodiment 1. The specific steps of this invention include: S1. Collect bidding document data and preprocess it; S2. Analyze the criticality of semantic units in bidding document data to construct a terminology criticality index; S3. Analyze the risk level of key semantic units in bidding documents based on key terminology indices and construct anchor risk entropy; S4. Analyze the risk sensitivity of common key semantic units in the bidding document data and the comparison document data based on anchor point risk entropy, and construct cross-document risk factors; S5. Based on the cross-document risk factor, improve the contrastive loss function of the Siamese neural network to realize semantic consistency verification of different bidding document data.

[0043] In summary, this invention constructs a terminology key index by analyzing the specificity and semantic solidification of semantic units in bidding scenarios, assessing the keyness of semantic units in bidding document data, which facilitates accurate extraction of anchor points in subsequent steps. The terminology key index is clustered to extract key semantic units from the bidding document data. Anchor point risk entropy is constructed based on the changes in value content following the key semantic units to assess the risk level of key semantic units in bidding documents. The average risk level of common key semantic units in different bidding document data is combined with the Jaccard distance of common key semantic units to their respective key value sets in the bidding document data to calculate the cross-document risk factor of common key semantic units. This dynamically improves the contrastive loss function of the Siamese neural network, imposing a stronger penalty on high-risk inconsistencies during the training process of the Siamese neural network. Ultimately, the trained verification system can effectively distinguish significant risk differences, improving the accuracy of cross-document semantic consistency verification in bidding scenarios.

[0044] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0045] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0046] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0047] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0048] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A cross-document semantic consistency verification system for bidding and tendering scenarios, characterized in that: The system includes: The data acquisition and preprocessing module is used to collect bidding document data and perform preprocessing. The Terminology Key Index Construction Module is used to analyze the keyness of semantic units in bidding document data and construct terminology key indices. The semantic risk analysis module is used to analyze the risk level of key semantic units in bidding documents based on terminology key indices and to construct anchor risk entropy. The cross-document risk factor acquisition module is used to analyze the risk sensitivity of common key semantic units in the bidding document data and the comparison document data based on anchor risk entropy and to construct cross-document risk factors. The consistency verification module is used to improve the contrastive loss function of the Siamese neural network based on the cross-document risk factor to realize semantic consistency verification of different bidding document data.

2. The cross-document semantic consistency verification system for bidding scenarios according to claim 1, characterized in that: The terminology key index construction module includes a corpus construction submodule and a terminology key index adaptive calculation submodule. The corpus construction submodule is used to construct the corpus based on the i-th semantic unit in the bidding document. The TF-IDF value of the frequency of appearance in the bidding documents is calculated, and news, reports and articles are collected through relevant API interfaces. After preprocessing according to the above steps, a general domain corpus is formed. The terminology key index adaptive calculation submodule is used to sort semantic units according to the order of the bidding document content. The set of the n nearest semantic units is called the nearest neighbor set. The frequency of each semantic unit in the bidding document data is counted. The nearest neighbor set is treated as a whole, and its frequency in the bidding document data is counted to obtain the term key index.

3. The cross-document semantic consistency verification system for bidding scenarios according to claim 2, characterized in that: The formula for calculating the key index of the term is as follows: ； in This represents the terminology key index of the i-th semantic unit in the bidding document. and Representing semantic units respectively The TF-IDF values ​​in the bidding documents and the corpus in the general domain, where n represents the number of semantic units in the nearest neighbor set. , and Representing semantic units respectively The first, second, and nth semantic units in the nearest neighbor unit set, This indicates the frequency of the nearest neighbor set in the bidding document data. Represents semantic units The frequency of the j-th semantic unit in the nearest neighbor set in the bidding document data.

4. A cross-document semantic consistency verification system for bidding scenarios according to claim 1, characterized in that: The process of constructing the anchor risk entropy includes: for the bidding document data, calculating the term key index of each semantic unit and using it as the input of the Otsu threshold segmentation method, outputting the segmentation threshold of the term key index, and using the set of semantic units that are greater than or equal to the segmentation threshold and deduplicated as the key semantic unit set, with the elements in the set being denoted as key semantic units; for the a-th key semantic unit, whenever it appears in the bidding document data, the sequence of the last M semantic units at any given time is denoted as the key value sequence, and the information entropy is calculated based on the occurrence probability of all key value sequences corresponding to the a-th key semantic unit.

5. A cross-document semantic consistency verification system for bidding scenarios according to claim 4, characterized in that: The method for calculating the anchor point risk entropy includes: ； in Let represent the anchor risk entropy of the a-th key semantic unit in the bidding document data, and let ln() represent the logarithmic function with the natural constant as the base, used to smooth the data within the parentheses. This indicates the number of occurrences of the a-th key semantic unit in the bidding document data. This represents the information entropy of all key value sequences corresponding to the a-th key semantic unit in the bidding document data.

6. A cross-document semantic consistency verification system for bidding scenarios according to claim 1, characterized in that: The cross-document risk factor acquisition module includes a public key semantic unit acquisition submodule and a cross-document risk factor calculation submodule; the public key semantic unit acquisition submodule is used to extract public key semantic units by comparing document data. The cross-document risk factor calculation submodule is used to extract the key value sequence of all common key semantic units in the bidding document data through the terminology key index construction module and the semantic risk analysis module. The set formed is denoted as the key value set. The key value set of the qth common key semantic unit in the comparison document data is extracted in the same way as described above to obtain the cross-document risk factor.

7. A cross-document semantic consistency verification system for bidding scenarios according to claim 6, characterized in that: The public key semantic unit acquisition submodule records the bidding document data used for comparison with the bidding document data as comparison document data, and obtains the key semantic sets of the bidding document data and comparison document data through the terminology key index construction module and semantic risk analysis module, respectively. Key semantic units with completely identical text content in the two key semantic sets are taken as public key semantic units.

8. A cross-document semantic consistency verification system for bidding scenarios according to claim 6, characterized in that: The formula for calculating the cross-document risk factor is as follows: ； in This represents the cross-document risk factor of the q-th common key semantic unit in the bidding document data and the comparison document data, where Sig() represents the sigmoid function. and Let Jac() represent the anchor risk entropy of the q-th common key semantic unit in the bidding document data and the comparison document data, respectively, and let Jac() represent the Jaccard distance function used to calculate the Jaccard distance between the two sets. and These represent the set of key values ​​for the q-th common key semantic unit in the bidding document data and the comparison document data, respectively.

9. A cross-document semantic consistency verification system for bidding scenarios according to claim 1, characterized in that: The consistency verification module includes: for non-public key semantic units, a dynamic adjustment factor is preset to a base value; for public key semantic units, the cross-document risk factor of the public key semantic units in the bidding document data and the comparison document data is used as a dynamic adjustment factor to adjust the contrast loss function in the neural network; the product of the dynamic adjustment factor and the contrast loss function is used as the adjusted contrast loss function; the bidding document data and the comparison document data are used as the input of the Siamese neural network; the adjusted contrast loss function is used as the loss function of the Siamese neural network; the semantic difference distance between the bidding document data and the comparison document data is output; and the cross-document semantic consistency verification result is obtained through a preset distance threshold.

10. A method for cross-document semantic consistency verification in bidding scenarios, used to execute a cross-document semantic consistency verification system for bidding scenarios as described in any one of claims 1-9, characterized in that: The specific steps include: S1. Collect bidding document data and preprocess it; S2. Analyze the criticality of semantic units in bidding document data to construct a terminology criticality index; S3. Analyze the risk level of key semantic units in bidding documents based on key terminology indices and construct anchor risk entropy; S4. Analyze the risk sensitivity of common key semantic units in the bidding document data and the comparison document data based on anchor point risk entropy, and construct cross-document risk factors; S5. Based on the cross-document risk factor, improve the contrastive loss function of the Siamese neural network to realize semantic consistency verification of different bidding document data.

Images