A cross-document semantic consistency checking system for a bidding scene

Abstract

The application discloses a kind of cross-document semantic consistency checking systems and methods for bidding scene, it is related to data processing technical field, including: collecting bidding document data and pre-processing;Analysis of the key degree of semantic unit in bidding document data constructs term key index;Based on term key index, the risk degree of key semantic unit of bidding document is analyzed and anchor risk entropy is constructed;Based on anchor risk entropy, the risk sensitivity of common key semantic unit of the bidding document data and comparison document data is analyzed and cross-document risk factor is constructed;Based on the cross-document risk factor, the comparison loss function of twin neural network is improved, the semantic consistency checking of different bidding document data is realized.The application solves the problem that the model has low degree of distinction between key inconsistency and non-key inconsistency during training, thereby affecting the identification accuracy of high-risk errors by the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a cross-document semantic consistency verification system and method for a bidding scene. BACKGROUND

[0002] Cross-document semantic consistency verification in the bidding scene refers to automatically comparing and analyzing key information, technical requirements, commercial terms or function descriptions and other contents in multiple documents involved in the project bidding process to ensure that there is no contradiction, ambiguity or omission in the semantic level between different documents. The core role is to significantly improve the preparation and review efficiency of bidding documents, reduce the error rate caused by negligence in manual checking, avoid legal or commercial disputes caused by inconsistent terms, and ensure that the bidding activities meet the predetermined requirements and standards, which has irreplaceable value for improving project quality and reducing project risk.

[0003] In the prior art, when verifying the consistency of document semantics, a twin neural network is usually used as the basic framework for contrast learning, and a standard contrast loss function is used to train the model to analyze the consistency of cross-document semantics. However, bidding documents have high industry and field specialization, a large number of terms, and the coexistence of high-risk and low-risk inconsistencies in key information. When directly using existing standard loss functions to verify the consistency of bidding documents, the key and non-key semantic information cannot be effectively distinguished and weighted, resulting in the loss function of the existing algorithm being unable to effectively prioritize the high-risk semantic inconsistency in the bidding scene, so that the model has low discrimination between key inconsistencies and non-key inconsistencies during the training process, thereby affecting the recognition accuracy of the system for high-risk errors.

[0004] Therefore, the present application provides a cross-document semantic consistency verification system and method for a bidding scene. SUMMARY

[0005] The purpose of the present application is to provide a cross-document semantic consistency verification system and method for a bidding scene to solve the existing problems raised in the background art.

[0006] To achieve the above purpose, the present application provides the following technical solution: a cross-document semantic consistency verification system for a bidding scene, comprising:

[0007] A data acquisition and preprocessing module is configured to acquire and preprocess bidding document data.

[0008] A term key index construction module is configured to analyze the key degree of semantic units in the bidding document data and construct a term key index.

[0009] The semantic risk analysis module is configured to analyze the risk degree of the key semantic units of the bidding document based on the term key index and construct an anchor risk entropy.

[0010] The cross-document risk factor acquisition module is configured to analyze the risk sensitivity degree of the common key semantic units of the bidding document data and the comparative document data based on the anchor risk entropy and construct a cross-document risk factor.

[0011] The consistency verification module is configured to improve the comparative loss function of the twin neural network based on the cross-document risk factor, and realize the semantic consistency verification of different bidding document data.

[0012] The application further improves that the term key index construction module comprises a corpus construction submodule and a term key index adaptive calculation submodule.

[0013] The corpus construction submodule is configured to calculate the TF-IDF value of the i-th semantic unit in the bidding document according to the frequency of the i-th semantic unit in the bidding document. The TF-IDF value of the i-th semantic unit in the bidding document is calculated according to the frequency of the i-th semantic unit in the bidding document, and news, reports and articles are collected through a related API interface, and after preprocessing according to the above steps, a general field corpus is constructed.

[0014] The term key index adaptive calculation submodule is configured to calculate the term key index of the i-th semantic unit in the bidding document according to the order of the content of the bidding document. The set of the last n semantic units is denoted as a neighbor unit set, the frequency of each semantic unit in the neighbor unit set in the bidding document data is counted respectively, and the neighbor unit set is taken as a whole, the frequency of the neighbor unit set in the bidding document data is counted, and the term key index is obtained.

[0015] The application further improves that the calculation formula of the term key index is represented as:

[0016] ;

[0017] Wherein represents the term key index of the i-th semantic unit in the bidding document, and respectively represent the first, second and n-th semantic units in the neighbor unit set of the i-th semantic unit, the TF-IDF values of the i-th semantic unit in the bidding document and the corpus of the general field, , and respectively represent the first, second and n-th semantic units in the neighbor unit set of the i-th semantic unit, and represents the frequency of the neighbor unit set in the bidding document data, Representing semantic units The frequency of the jth semantic unit in the neighbor unit set in the bidding document data.

[0018] The application further improves that the construction process of the anchor point risk entropy includes: calculating the term key index of each semantic unit as the input of the Otsu threshold segmentation method, and outputting the segmentation threshold of the term key index, and taking the set of the semantic units greater than or equal to the segmentation threshold and after deduplication as the key semantic unit set, and the elements in the set are recorded as key semantic units; for the a th key semantic unit, whenever it appears in the bidding document data, the sequence of the last M semantic units in any one appearance is recorded as a key value sequence, and the information entropy is calculated according to the appearance probability of all key value sequences corresponding to the a th key semantic unit.

[0019] The application further improves that the calculation method of the anchor point risk entropy includes:

[0020] ;

[0021] Wherein The anchor point risk entropy of the a th key semantic unit in the bidding document data, ln() represents the logarithmic function with natural constant as the base, which is used for smoothing the data in the parentheses, The number of the a th key semantic unit appearing in the bidding document data, The information entropy of all key value sequences corresponding to the a th key semantic unit in the bidding document data.

[0022] The application further improves that the cross-document risk factor acquisition module includes a public key semantic unit acquisition submodule and a cross-document risk factor calculation submodule; the public key semantic unit acquisition submodule is used for extracting public key semantic units by comparing document data;

[0023] The cross-document risk factor calculation submodule is used for extracting the key value sequence of all the public key semantic units in the bidding document data through the term key index construction module and the semantic risk analysis module, and the set formed is recorded as a key value set, and the key value set of the q th public key semantic unit in the comparison document data is extracted in the above-mentioned manner to obtain the cross-document risk factor.

[0024] The application further improves that the public key semantic unit acquisition submodule records the bidding document data used for comparison as comparison document data, obtains the key semantic set of the bidding document data and the comparison document data through the term key index construction module and the semantic risk analysis module respectively, and takes the key semantic unit with the same text content in the two obtained key semantic sets as the public key semantic unit.

[0025] The application further improves that the calculation formula of the cross-document risk factor is represented as:

[0026] ;

[0027] Wherein represents the cross-document risk factor of the qth public key semantic unit in the bidding document data and the comparison document data, Sig() represents the sigmoid function, and respectively represent the anchor point risk entropy of the qth public key semantic unit in the bidding document data and the comparison document data, Jac() represents the Jaccard distance function, and is used for calculating the Jaccard distance of two sets, and respectively represent the key value set of the qth public key semantic unit in the bidding document data and the comparison document data.

[0028] The application further improves that the consistency verification module comprises that for the non-public key semantic unit, the dynamic adjustment factor is preset as a basic value; for the public key semantic unit, the cross-document risk factor of the public key semantic unit in the bidding document data and the comparison document data is taken as the dynamic adjustment factor, the comparison loss function in the neural network is adjusted, the product of the dynamic adjustment factor and the comparison loss function is taken as the adjusted comparison loss function, the bidding document data and the comparison document data are taken as the input of the twin neural network, the adjusted comparison loss function is taken as the loss function of the twin neural network, the semantic difference distance of the bidding document data and the comparison document data is output, and the cross-document semantic consistency verification result is obtained through the preset distance threshold.

[0029] On the other hand, the application provides a cross-document semantic consistency verification method for a bidding scene, and the specific steps comprise:

[0030] S1, collecting bidding document data and performing pretreatment;

[0031] S2, analyzing the key degree of the semantic unit in the bidding document data to construct a term key index;

[0032] S3, analyze the risk degree of the key semantic units of the bidding document based on the term key index, and construct an anchor point risk entropy;

[0033] S4, analyze the risk sensitivity degree of the common key semantic units of the bidding document data and the comparison document data based on the anchor point risk entropy, and construct a cross-document risk factor;

[0034] S5, improve the comparison loss function of the twin neural network based on the cross-document risk factor, and realize semantic consistency verification of different bidding document data.

[0035] Compared with the prior art, the beneficial effects of the present application are:

[0036] Firstly, the semantic unit is evaluated in the bidding document data by the term key index construction module, and the risk degree of the key semantic unit of the bidding document is evaluated by the semantic risk analysis module. The cross-document risk factor of the common key semantic unit is calculated, and the comparison loss function of the twin neural network is dynamically improved, so that in the training process of the twin neural network, stronger punishment is applied to high-risk inconsistency, and finally the trained verification system can effectively distinguish significant risk differences, and improve the accuracy of cross-document semantic consistency verification in the bidding scene. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A cross-document semantic consistency verification system framework for the bidding scene of the present application;

[0038] Figure 2 A cross-document semantic consistency verification method flow chart for the bidding scene of the present application. DETAILED DESCRIPTION

[0039] The technical solutions of the present application will be described in detail below with the help of the drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solutions of the present application, and are not limitations of the technical solutions of the present application. In the case of no conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0040] The term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent three cases of A alone, A and B together, and B alone.

[0041] Embodiment 1

[0042] Figure 1 A cross-document semantic consistency verification system framework for the bidding scene disclosed by the present embodiment is shown, which comprises:

[0043] A data collection and preprocessing module is configured to collect and preprocess the bidding document data;

[0044] In the verification system, the data collection module deployed on the local server or the cloud server collects relevant data, including automatically collecting data such as bidding documents and bid-winning announcements from data sources such as bidding websites and enterprise procurement platforms, and saving the collected data into a txt file in a pure text format; through an authorized application API interface, the document management system in the enterprise is accessed to obtain document data such as technical specifications, bidding documents, and contract drafts in historical projects.

[0045] After the original document data is collected, relevant preprocessing is required to obtain pure text data. Taking one document as an example: different formats of documents such as PDF, DOCX, or XML are uniformly recognized as pure text txt format through OCR technology, and irrelevant information such as headers, footers, page numbers, and watermarks in the document is identified and removed using regular expressions; the pure text is divided into independent sentence units using punctuation marks as delimiters; for a sentence unit, it is divided into several semantic units through the jieba word segmentation library of python; all sentence units in the document are processed according to the above steps to obtain a set of semantic units of the document, which are used as the preprocessed bidding document data.

[0046] A term key index construction module is configured to analyze the key degree of the semantic units in the bidding document data to construct a term key index;

[0047] The unique feature of bidding documents that distinguishes them from general texts is the highly uneven distribution of value and risk, that is, the documents contain a large number of key semantic units that play a decisive role in the success or failure of the project, such as price, construction period, and core technical indicators. These words usually exist in a structured or semi-structured form of keywords, numerical values, or specific phrases. When comparing across documents, subtle changes in these key semantic units can cause significant commercial risks, such as the delivery period written in bidding document 1 being 90 days and the delivery period written in bidding document 2 being 90 working days; the difference in other descriptive or modifying statements has less impact, such as the content written in bidding document 1 being “advanced technology” and the content written in bidding document 2 being “leading technology”. In the prior art, the semantic units of two documents are usually compared globally, which cannot effectively distinguish the above two differences.

[0048] Based on the above analysis, the term key index is constructed by a corpus construction submodule and a term key index adaptive calculation submodule, which is used to reflect the key degree of the semantic units in the bidding document data. The construction process of the term key index is as follows:

[0049] For the convenience of description, the application is described by taking any one of the bidding documents as an example to construct the corpus database; the i-th semantic unit in the bidding document is denoted as For example, according to the semantic unit The TF-IDF value of the frequency of occurrence in the bidding document is calculated, and news, reports and articles are collected through a related API interface or the like, and after being preprocessed according to the above steps, a general field corpus is formed.

[0050] According to the order of the content of the bidding document, the term key index adaptive calculation submodule is run, and the semantic units The set of the last n semantic units is denoted as a neighbor unit set, the frequency of occurrence of each semantic unit in the neighbor unit set in the bidding document data is counted, and the neighbor unit set is taken as a whole, and the frequency of occurrence of the neighbor unit set in the bidding document data is counted. The size of n is 3 in the application, and the implementer can select it according to the actual situation. For example, when n is 3 in the application, the neighbor unit set is {delivery, period, one day}, and there are a total of 100 semantic units in the bidding document data, among which {delivery, period, one day} appears 3 times, in addition to {delivery, period, two days} which appears 2 times and {delivery, period, ten days} which appears 1 time, then the frequency of occurrence of the neighbor unit set {delivery, period, one day} in the bidding document data is .

[0051] Based on the above processing steps, the calculation method of the term key index in the application is as follows:

[0052]

[0053] Among them denotes the term key index of the i-th semantic unit in the bidding document, and denote the TF-IDF value of the semantic unit in the bidding document and the general field corpus, 1 in the numerator and the denominator is to improve the robustness and avoid extreme values, and n denotes the number of semantic units in the neighbor unit set, , and denote the first, second and n-th semantic units in the neighbor unit set of the semantic unit , denotes the frequency of occurrence of the neighbor unit set in the bidding document data, denotes the frequency of occurrence of the j-th semantic unit in the neighbor unit set of the semantic unit in the bidding document data.

[0054] In the formula Key weight: in the bidding document data, if the semantic unit The greater the TF-IDF value in the bidding document data relative to the TF-IDF value in the general field corpus, the greater the possibility that the semantic unit belongs to the professional or commonly used vocabulary in the bidding scenario, and thus the greater the possibility that it is a key vocabulary in the bidding scenario, and the greater the calculated key weight;

[0055] wherein Solidification degree of semantic unit: in the bidding document data, if the semantic unit The greater the probability that the neighbor unit set as a whole appears, the greater the possibility that the word combination formed by the neighbor unit set is a professional term in the bidding scenario, and the greater the solidification degree, and thus the greater the calculated solidification degree of the semantic unit;

[0056] In summary, the greater the key weight of the semantic unit The greater the possibility that the semantic unit is a semantic unit in the core position of the key information structure, and the greater the key degree in the bidding scenario, and thus the greater the calculated term key index.

[0057] The semantic risk analysis module is configured to analyze the risk degree of the key semantic unit of the bidding document based on the term key index and construct an anchor point risk entropy.

[0058] In the bidding scenario, the key semantic unit in the bidding document data has an anchor point effect of key information and variability of value content. For example, the key semantic units such as “bid price”, “period” or “warranty” have an anchor point effect in the bidding document data, that is, the content after these key semantic units is variable and important, such as the value after the semantic unit “bid price” can be 5 million yuan, 8 million yuan or any other data. This “anchor-value” structure and the high uncertainty of the “value” part are the core verification items that are most likely to have high risk in the bidding document data.

[0059] Therefore, the present application constructs an anchor point risk entropy based on the term key index, which is used to reflect the risk degree of the key semantic unit of the bidding document. The construction process of the anchor point risk entropy is as follows:

[0060] ​For the bidding document data, the term key index of each semantic unit is calculated according to the above steps, and is taken as the input of the Otsu threshold segmentation method, the output is the segmentation threshold of the term key index, and the set composed of the semantic units greater than or equal to the segmentation threshold and after deduplication is taken as the key semantic unit set, and the elements in the set are recorded as key semantic units; for the a-th key semantic unit, whenever it appears in the bidding document data, the sequence composed of the last M semantic units at any one time is recorded as a key value sequence, and the information entropy is calculated according to the appearance probability of all key value sequences corresponding to the a-th key semantic unit. The size of M is 3 in this application, which can be selected according to the actual situation.

[0061] Based on the above processing steps, the calculation method of the anchor point risk entropy is as follows:

[0062]

[0063] Wherein represents the anchor point risk entropy of the a-th key semantic unit in the bidding document data, and ln() represents the logarithmic function with natural constant as the base, which is used for smoothing the data in the parentheses, represents the number of occurrences of the a-th key semantic unit in the bidding document data, represents the information entropy of all key value sequences corresponding to the a-th key semantic unit in the bidding document data.

[0064] In the formula represents the harmonic term, reflecting the effectiveness of the a-th key semantic unit, The greater, the more times the key semantic unit appears in the bidding document, the greater the weight when the risk appears, and the greater the corresponding harmonic term calculated;

[0065] After identifying all the key semantic units through the term key index of the bidding document data, the key semantic unit is taken as an important anchor point in the bidding document data, and the content after the important anchor point is more likely to be the core risk item of the bidding document data. For the a-th key semantic unit, if the core risk item after it is the same, the information entropy calculated is smaller, that is, The smaller, the smaller the corresponding risk degree, so that the anchor point risk entropy calculated is smaller; if the core risk item after the important anchor point is different, the information entropy calculated is greater, that is, The greater, the greater the corresponding risk degree, so that the anchor point risk entropy calculated is greater.

[0066] The cross-document risk factor acquisition module is configured to analyze risk sensitivity of common key semantic units of the bidding document data and the comparison document data based on the anchor point risk entropy, and construct a cross-document risk factor; and includes a common key semantic unit acquisition submodule and a cross-document risk factor calculation submodule.

[0067] When performing semantic consistency verification on different bidding document data, key semantic units with greater risk degree in the two bidding document data need to be focused on, that is, when the key semantic units with greater risk degree are inconsistent, such as the content "delivery cycle 90 days" in the bidding document data and the content "delivery cycle 90 working days" in the comparison document data, a greater loss function value should be generated, a stronger punishment signal is transmitted to the model in the training process of the neural network, and the internal weight parameters are updated according to the gradient calculated by the loss function through the back propagation algorithm; the attention degree of the key semantic units with smaller risk degree can be appropriately reduced, that is, when the key semantic units with smaller risk degree are inconsistent, such as the content "advanced technology" in the bidding document data and the content "leading technology" in the comparison document data, a smaller loss function value should be generated, and a weaker punishment signal is transmitted to the model in the training process of the neural network.

[0068] Based on the above analysis, the cross-document risk factor is constructed based on the anchor point risk entropy in the application, and the risk sensitivity of the common key semantic units in different bidding document data is reflected. The construction process of the cross-document risk factor is as follows:

[0069] The common key semantic unit acquisition submodule records the bidding document data used for comparison with the bidding document data as comparison document data, obtains the key semantic sets of the bidding document data and the comparison document data according to the above steps, and takes the key semantic units with the same text content in the two key semantic sets as the common key semantic units.

[0070] The cross-document risk factor calculation submodule takes the qth common key semantic unit as an example, extracts the key value sequence of all the common key semantic units in the bidding document data according to the above steps to form a set, and extracts the key value set of the qth common key semantic unit in the comparison document data according to the above method.

[0071] Based on the above processing steps, the calculation method of the cross-document risk factor in the application is as follows:

[0072]

[0073] wherein Sig() represents a sigmoid function, used for normalizing the data in the parentheses, and respectively represent the anchor point risk entropy of the qth common key semantic unit in the bidding document data and the comparison document data, Jac() represents a Jaccard distance function, used for calculating the Jaccard distance of two sets, and respectively represent the key value set of the qth common key semantic unit in the bidding document data and the comparison document data.

[0074] In the formula, reflects the average risk degree of the common key semantic unit in the two document data. The greater the value is, the more concentrated the risk of the common key semantic unit is, and the more likely it is to belong to the core verification item. Taking the common key semantic unit as the basic weight makes the model pay more attention to the common key semantic unit;

[0075] In the formula, reflects the specific expression difference of the key value set corresponding to the qth common key semantic unit in the bidding document data and the comparison document data. For example, for the "delivery period" semantic unit in the bidding document data, the key value set is {100,000 yuan, 200,000 yuan}, while the key value set of the "delivery period" semantic unit in the comparison document data is {500,000 yuan, 800,000 yuan, 200,000 yuan}, which indicates that although there is some overlap in the specific value content of the two documents, other value contents have significant differences. The greater the is, the greater the difference is.

[0076] In summary, based on and the cross-document risk factors calculated can reflect the risk sensitivity of the common key semantic units in different bidding document data.

[0077] The consistency verification module is configured to improve the comparison loss function of the twin neural network based on the cross-document risk factor, and realize semantic consistency verification of different bidding document data.

[0078] Through the above modules, the cross-document risk factor of the common key semantic unit in the bidding document data and the comparison document data is obtained, which is used as a dynamic adjustment factor to adjust the comparison loss function in the neural network. For non-common key semantic units, the dynamic adjustment factor is preset to a basic value, which is 0.1 in this application, and can be selected according to actual conditions.

[0079] The process of adjusting the contrast loss function in the neural network by the dynamic adjustment factor is: taking the product of the dynamic adjustment factor and the contrast loss function as the adjusted contrast loss function.

[0080] The bidding document data and the contrast document data are taken as the input of the twin neural network, the adjusted contrast loss function is taken as the loss function of the twin neural network, and the output is the semantic consistency verification result of the bidding document data and the contrast document data. The semantic consistency verification result is the semantic difference distance of the bidding document data and the contrast document data.

[0081] A distance threshold is preset. If the semantic difference distance is greater than or equal to the distance threshold, it indicates that the semantics of the bidding document data and the contrast document data are not consistent. If the semantic difference distance is less than the distance threshold, it indicates that the semantics of the bidding document data and the contrast document data are consistent. It is taken as the verification result of the cross-document semantic consistency. The preset distance threshold is 0.5 in this application, which can be selected according to the actual situation.

[0082] The threshold and weight setting values can be set by default according to the application, or can be set by the person skilled in the art.

[0083] Embodiment 2

[0084] Figure 2 A flow chart of a cross-document semantic consistency verification method for a bidding scenario is shown. Based on the same inventive concept as embodiment 1, the application provides a cross-document semantic consistency verification method for a bidding scenario. The specific steps include:

[0085] S1, collecting bidding document data and preprocessing;

[0086] S2, analyzing the key degree of the semantic unit in the bidding document data to construct a term key index;

[0087] S3, analyzing the risk degree of the key semantic unit of the bidding document based on the term key index and constructing an anchor risk entropy;

[0088] S4, analyzing the risk sensitivity degree of the common key semantic unit of the bidding document data and the contrast document data based on the anchor risk entropy and constructing a cross-document risk factor;

[0089] S5, improving the contrast loss function of the twin neural network based on the cross-document risk factor, and realizing the semantic consistency verification of different bidding document data.

[0090] To sum up, the application constructs a term key index by analyzing the specificity and semantic cohesiveness of semantic units in the bidding scene, evaluates the key degree of the semantic units in the bidding document data, and facilitates the accurate extraction of anchor points of the bidding document data in the subsequent steps; the term key index is subjected to clustering processing, the key semantic units in the bidding document data are extracted, the anchor point risk entropy is constructed according to the change of the value content after the key semantic units, and the risk degree of the key semantic units of the bidding document is evaluated; the average risk degree of the public key semantic units in different bidding document data and the Jaccard distance of the key value set of the public key semantic units in the respective bidding document data are combined, the cross-document risk factor of the public key semantic units is calculated, the comparison loss function of the twin neural network is dynamically improved, stronger punishment is applied to the high-risk inconsistency in the training process of the twin neural network, and finally the trained verification system can effectively distinguish the major risk difference, and the accuracy of the cross-document semantic consistency verification in the bidding scene is improved.

[0091] Those skilled in the art will appreciate that embodiments of the application can be supplied as methods, systems, or computer program products. Accordingly, the application can be embodied in the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware aspects. Furthermore, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) having computer usable program code embodied thereon.

[0092] The present application is described in reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device that implements the flowcharts and / or block diagrams. Figure 1 The flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in the flow or multiple flows and / or blocks.

[0093] These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including instruction devices that implement the flowcharts and / or block diagrams. Figure 1 The flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in the flow or multiple flows and / or blocks.

[0094] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate computer-implemented processes, so that the instructions executed on the computer or other programmable devices provide operational steps for implementing the functions specified in the flowchart Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0095] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and those of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which are all within the protection of the present application.

Claims

1. A cross-document semantic consistency verification system for bidding and tendering scenarios, characterized in that: The system includes: The data acquisition and preprocessing module is used to collect bidding document data and perform preprocessing. The Terminology Key Index Construction Module is used to analyze the keyness of semantic units in bidding document data and construct terminology key indices. The semantic risk analysis module is used to analyze the risk level of key semantic units in bidding documents based on terminology key indices and to construct anchor risk entropy. The cross-document risk factor acquisition module is used to analyze the risk sensitivity of common key semantic units in the bidding document data and the comparison document data based on anchor risk entropy and to construct cross-document risk factors. The consistency verification module is used to improve the contrastive loss function of the Siamese neural network based on the cross-document risk factor to realize semantic consistency verification of different bidding document data. The terminology key index construction module includes a corpus construction submodule and a terminology key index adaptive calculation submodule. The corpus construction submodule is used to construct the corpus based on the i-th semantic unit in the bidding document. The TF-IDF value of the frequency of appearance in the bidding documents is calculated, and news, reports and articles are collected through relevant API interfaces. After preprocessing according to the above steps, a general domain corpus is formed. The terminology key index adaptive calculation submodule is used to sort semantic units according to the order of the bidding document content. The set of the n nearest semantic units is called the nearest neighbor unit set. The frequency of each semantic unit in the bidding document data is counted. The nearest neighbor unit set is treated as a whole, and its frequency in the bidding document data is counted to obtain the term key index. The formula for calculating the key index of the term is as follows: ； in This represents the terminology key index of the i-th semantic unit in the bidding document. and Representing semantic units respectively The TF-IDF values ​​in the bidding documents and the corpus in the general domain, where n represents the number of semantic units in the nearest neighbor set. , and Representing semantic units respectively The first, second, and nth semantic units in the nearest neighbor unit set, This indicates the frequency of the nearest neighbor set in the bidding document data. Represents semantic units The frequency of the j-th semantic unit in the nearest neighbor set in the bidding document data.

2. The cross-document semantic consistency verification system for bidding scenarios according to claim 1, characterized in that: The process of constructing the anchor risk entropy includes: for the bidding document data, calculating the term key index of each semantic unit and using it as the input of the Otsu threshold segmentation method, outputting the segmentation threshold of the term key index, and using the set of semantic units that are greater than or equal to the segmentation threshold and deduplicated as the key semantic unit set, with the elements in the set being denoted as key semantic units; for the a-th key semantic unit, whenever it appears in the bidding document data, the sequence of the last M semantic units at any given time is denoted as the key value sequence, and the information entropy is calculated based on the occurrence probability of all key value sequences corresponding to the a-th key semantic unit.

3. The cross-document semantic consistency verification system for bidding scenarios according to claim 2, characterized in that: The method for calculating the anchor point risk entropy includes: ； in Let represent the anchor risk entropy of the a-th key semantic unit in the bidding document data, and let ln() represent the logarithmic function with the natural constant as the base, used to smooth the data within the parentheses. This indicates the number of occurrences of the a-th key semantic unit in the bidding document data. This represents the information entropy of all key value sequences corresponding to the a-th key semantic unit in the bidding document data.

4. A cross-document semantic consistency verification system for bidding scenarios according to claim 1, characterized in that: The cross-document risk factor acquisition module includes a public key semantic unit acquisition submodule and a cross-document risk factor calculation submodule; the public key semantic unit acquisition submodule is used to extract public key semantic units by comparing document data. The cross-document risk factor calculation submodule is used to extract the key value sequence of all common key semantic units in the bidding document data through the terminology key index construction module and the semantic risk analysis module. The set formed is denoted as the key value set. The key value set of the qth common key semantic unit in the comparison document data is extracted in the same way as described above to obtain the cross-document risk factor.

5. A cross-document semantic consistency verification system for bidding scenarios according to claim 4, characterized in that: The public key semantic unit acquisition submodule records the bidding document data used for comparison with the bidding document data as comparison document data, and obtains the key semantic sets of the bidding document data and comparison document data through the terminology key index construction module and semantic risk analysis module, respectively. Key semantic units with completely identical text content in the two key semantic sets are taken as public key semantic units.

6. A cross-document semantic consistency verification system for bidding scenarios according to claim 4, characterized in that: The formula for calculating the cross-document risk factor is as follows: ； in This represents the cross-document risk factor of the q-th common key semantic unit in the bidding document data and the comparison document data, where Sig() represents the sigmoid function. and Let Jac() represent the anchor risk entropy of the q-th common key semantic unit in the bidding document data and the comparison document data, respectively, and let Jac() represent the Jaccard distance function used to calculate the Jaccard distance between the two sets. and These represent the set of key values ​​for the q-th common key semantic unit in the bidding document data and the comparison document data, respectively.

7. A cross-document semantic consistency verification system for bidding scenarios according to claim 1, characterized in that: The consistency verification module includes: for non-public key semantic units, a dynamic adjustment factor is preset to a base value; for public key semantic units, the cross-document risk factor of the public key semantic units in the bidding document data and the comparison document data is used as a dynamic adjustment factor to adjust the contrast loss function in the neural network; the product of the dynamic adjustment factor and the contrast loss function is used as the adjusted contrast loss function; the bidding document data and the comparison document data are used as the input of the Siamese neural network; the adjusted contrast loss function is used as the loss function of the Siamese neural network; the semantic difference distance between the bidding document data and the comparison document data is output; and the cross-document semantic consistency verification result is obtained through a preset distance threshold.

8. A method for cross-document semantic consistency verification in bidding scenarios, used to execute a cross-document semantic consistency verification system for bidding scenarios as described in any one of claims 1-7, characterized in that: The specific steps include: S1. Collect bidding document data and preprocess it; S2. Analyze the criticality of semantic units in bidding document data to construct a terminology criticality index; S3. Analyze the risk level of key semantic units in bidding documents based on key terminology indices and construct anchor risk entropy; S4. Analyze the risk sensitivity of common key semantic units in the bidding document data and the comparison document data based on anchor point risk entropy, and construct cross-document risk factors; S5. Based on the cross-document risk factor, improve the contrastive loss function of the Siamese neural network to realize semantic consistency verification of different bidding document data.

Images