Green energy power industry purchase file compliance inspection method and system

Through natural language processing technology, the green energy power industry procurement documents are analyzed using the converter bidirectional encoder model and neural network, solving the problem of traditional manual review time-consuming and inability to fully detect compliance, and achieving efficient and accurate compliance Inspection and risk analysis.

CN119067457AInactive Publication Date: 2024-12-03SICHUAN ENRISING INFORMATION TECH CO LTD

Patent Information

Application Number
CN202411562424.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-05
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The compliance inspection of procurement documents in the traditional green energy and power industry relies on manual review, which is time-consuming and labor-intensive and is prone to omit key details due to human negligence or errors. The existing automation platform cannot fully ensure the compliance of procurement documents through completeness inspection only. , in particular, it is impossible to identify compliance with regulations and policies, industry regulations and market risks.

Method used

The bidirectional encoder model based on the transformer is used to distribute the vectorization of the procurement file text data, and the word vector is weighted by combining word frequency and inverse document frequency methods. The two-way long and short-term memory network and conditional random field model are used to extract entity information and pass the neural network. Analyze whether laws, policies and industry regulations are violated and a detailed review report is generated.

Benefits of technology

It significantly improves the efficiency and accuracy of compliance testing of procurement documents, reduces human misjudgment, can identify current and potential compliance and market risks, and provides scientific and reasonable risk management strategies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067457A_ABST
    Figure CN119067457A_ABST
Patent Text Reader

Abstract

The invention discloses a method and a system for checking the compliance of purchasing files in the green energy power industry, and relates to the technical field of data analysis, the method and the system are mainly applied to checking the compliance of the purchasing files in the green energy power industry, and the purchasing files in the green energy power industry have high requirements on compliance. In order to solve the problem that the existing purchasing files are difficult to check due to various contents, when the compliance of the purchasing files is checked, whether the purchasing files are complete or not is checked item by item, then the purchasing files are checked item by item according to preset rules of a user, relevant laws and regulations, policies and industry regulations are checked, and whether potential risks exist or not is analyzed. Finally, the review results are sorted, review items are generated and marked, and through the automatic review process, the time and labor intensity of manual review are remarkably reduced. The system can quickly process a large number of purchase files, and the review efficiency is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and more specifically, to a method and system for checking compliance of procurement documents in the green energy and power industry. Background Art

[0002] Traditional compliance checks on procurement documents in the green energy and power industry mainly rely on manual review, which is time-consuming and labor-intensive, and is prone to missing key details due to human negligence or errors. At present, some platforms also provide automated compliance checks on procurement documents, but these platforms usually only review the content of procurement documents through integrity checks. However, the integrity check method alone has limitations and cannot fully ensure whether the procurement documents comply with preset rules, whether they comply with relevant laws and policies, industry regulations, and whether there are market risks. Therefore, the data resource platform needs to continuously improve and optimize the compliance detection of procurement documents. How to accurately detect non-compliance in procurement documents and generate detailed review reports has become a new challenge.

[0003] With the rapid development of natural language processing technology, there have been more and more innovations in document compliance detection technology. However, current innovations are mostly focused on compliance detection of general documents, and less attention has been paid to procurement documents with extremely strict compliance requirements, especially procurement documents in the green energy and power industry. Therefore, it is particularly urgent to develop a method that can automatically identify and evaluate the compliance of procurement documents in the green energy and power industry. Summary of the invention

[0004] In order to address the deficiencies in the prior art, the purpose of the present invention is to provide a method and system for compliance checking of procurement documents in the green energy and power industry, which can significantly improve the efficiency and accuracy of compliance checking of procurement documents in the green energy and power industry, and provide strong technical support and solutions for users in related fields.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions: In a first aspect, a method for checking compliance of procurement documents in the green energy power industry is provided, comprising the following steps: Perform natural language processing on the text data of the procurement document to obtain initial text data; A transformer-based bidirectional encoder model is used to perform distributed vectorization on the initial text data to obtain a first word vector set consisting of word vectors; The word vectors in the first word vector set are weighted by using the word frequency and inverse document frequency method to obtain a second word vector set consisting of the weighted word vectors; A model combining a bidirectional long short-term memory network and a conditional random field is used to extract entity information from the second word vector set, the integrity rules and user preset rules are loaded into the rule engine, and the entity information is matched with the rule engine to check whether the rule conditions are met; The regulatory policy and industry regulations inspection problem and potential risk analysis problem are modeled as multi-classification problems respectively. A classifier network is established by using a neural network to analyze whether the procurement document text data violates regulatory policies and industry regulations and whether there are potential risks. The detected items that do not comply with integrity rules, user preset rules, regulations, policies and industry provisions, or have potential risks are sorted and marked in the green energy and power industry procurement documents, and review items are generated.

[0006] Furthermore, the process of using the transformer-based bidirectional encoder model to perform distributed vectorization on the initial text data is specifically as follows: Input the initial text data into the bidirectional encoder for processing to obtain a vector composed of the sum of the word vector, the text vector and the position vector; The vector of the bidirectional encoder is passed into the transformer encoding block, and the output is a word vector containing semantic and contextual information.

[0007] Furthermore, the process of inputting the initial text data into the bidirectional encoder for processing is specifically as follows: The initial text data is segmented into a series of symbols through a bidirectional encoder, and the symbols are mapped to a fixed word vector through a word embedding model; When there are multiple sentences input, the bidirectional encoder uses paragraph or sentence embedding to distinguish the two sentences. All the symbols of the first sentence are added with a set paragraph embedding vector, and all the symbols of the second sentence are added with another different paragraph embedding vector to form a text vector. The relationship between positions is learned based on sine and cosine functions with periodicity and frequency characteristics to obtain a position vector.

[0008] Furthermore, each of the converter encoding blocks is composed of two sub-layers of a multi-head self-attention mechanism and a feed-forward neural network, and residual connections and layer normalization techniques are used between the two sub-layers and afterwards.

[0009] Furthermore, the process of weighting the word vectors in the first word vector set by using the word frequency and inverse document frequency method is specifically as follows: Calculate the word frequency and inverse document probability of the word vector respectively, and calculate the word frequency and inverse document frequency weights of the corresponding word vector by the product of the word frequency and the inverse document probability; Multiply the word vector by the corresponding word frequency-inverse document frequency weight to obtain the weighted word vector.

[0010] Furthermore, the classifier network is constructed by fusing the bidirectional encoder model and the text convolutional neural network model, and the fusion process is specifically as follows: The bidirectional encoder model removes the first input layer and retains 12 encoding block layers. The first symbol vector of each encoding block layer is regarded as the sentence vector. And, the sentence vectors from the 1st to the 12th layer are used as the input of the text convolutional neural network and then classified.

[0011] Furthermore, the method further comprises: Generate virtual training data using a large model, including a word vectorization virtual data set, an entity information virtual data set, and a procurement contract regulations virtual data set; The training data of the transformer-based bidirectional encoder model is padded with word vectorization virtual datasets; The training data of the model combining the bidirectional long short-term memory network and the conditional random field is filled with data through the entity information virtual data set; And, the training data of the classifier network is filled with data volume through the virtual data set of procurement contract regulations.

[0012] Furthermore, the method further comprises: Users customize data sets according to template formats; The virtual training data and the user-defined data set are merged to generate the final virtual data set.

[0013] Furthermore, the method further comprises: When the integrity rules and user preset rules check meet the rule conditions, the regulatory policy and industry regulations check is initiated; Also, when the regulatory policies and industry regulations check shows no violation of regulatory policies and industry regulations, initiate potential risk analysis.

[0014] In a second aspect, a green energy power industry procurement document compliance checking system is provided, and the system is used to implement a green energy power industry procurement document compliance checking method as described in any one of the first aspects, including: A text processing module is used to perform natural language processing on the text data of the procurement document to obtain initial text data; A text vectorization module, used for performing distributed vectorization on the initial text data by using a transformer-based bidirectional encoder model to obtain a first word vector set consisting of word vectors; A vector weighting module, used for weighting the word vectors in the first word vector set by using a word frequency and inverse document frequency method to obtain a second word vector set consisting of weighted word vectors; An entity extraction module is used to extract entity information from the second word vector set using a model combining a bidirectional long short-term memory network and a conditional random field, load the integrity rules and user preset rules into the rule engine, and match the entity information with the rule engine to check whether the rule conditions are met; The inspection and analysis module is used to model the inspection of laws, policies and industry regulations and the analysis of potential risks as multi-classification problems. A classifier network is established by using a neural network to analyze whether the text data of procurement documents violates laws, policies and industry regulations and whether there are potential risks. The inspection and sorting module is used to sort and mark the items that are detected to be non-compliant with integrity rules, user preset rules, regulations, policies and industry regulations, or have potential risks in the green energy and power industry procurement documents, and generate review items.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention is mainly used for compliance inspection of procurement documents in the green energy and power industry. Procurement documents in the green energy and power industry have high compliance requirements and are difficult to inspect due to their numerous contents. When inspecting the compliance of procurement documents, first check each item to see if they are complete, then review each item according to user preset rules, then check relevant laws, policies and industry regulations, and analyze whether there are potential risks. Finally, organize the review results, generate review items and mark them. By automating the review process, the time and labor intensity of manual review are significantly reduced. The system can quickly process a large number of procurement documents, greatly improving the review efficiency; 2. The present invention uses advanced natural language processing technology to deeply understand and analyze texts, which can accurately identify and classify risks and reduce the possibility of human misjudgment; 3. The present invention can not only identify current compliance issues, but also analyze potential market risks and compliance risks, and predict the probability of their occurrence. Users can formulate more scientific and reasonable risk management strategies based on the risk analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings: Figure 1 is a flow chart of Embodiment 1 of the present invention; Figure 2 Schematic diagram of a method for generating training data based on a large model in Embodiment 1 of the present invention; Figure 3 is a schematic diagram of a bidirectional encoder model in Embodiment 1 of the present invention; Figure 4is a schematic diagram of a long short-term memory network model in Example 1 of the present invention; Figure 5 Schematic diagram of a bidirectional long short-term memory network model in Embodiment 1 of the present invention; Figure 6 is a schematic diagram of a fusion model in Example 1 of the present invention; Figure 7 It is a system block diagram in Example 2 of the present invention. DETAILED DESCRIPTION

[0017] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0018] Example 1: A method for checking compliance of procurement documents in the green energy power industry, such as Figure 1 As shown, it is specifically implemented by steps.

[0019] Step 1: Perform natural language processing on the procurement document text data to obtain initial text data.

[0020] Process the green energy and power industry procurement document text uploaded by users, perform word segmentation and semantic extraction on the text, and filter out useless spaces, punctuation marks, auxiliary words, and digital titles based on parts of speech to prevent these useless words from affecting subsequent compliance checks.

[0021] Step 2: Since there are relatively few data sets on green power industry procurement documents, a large model can be used to generate virtual training data to fill the data volume and improve the model training effect.

[0022] The virtual training data includes word vectorization virtual dataset, entity information virtual dataset and procurement contract regulations virtual dataset.

[0023] like Figure 2 As shown, first specify the input template of the data set. Different models require different data sets, so different input templates need to be specified: (1) Word vectorization dataset input template: { "Green Power Industry Examples": [ "The seller promises that the electricity provided will be sourced from renewable energy and will meet the green electricity certification standards specified by the buyer, such as green certificates or carbon emission reduction projects.", "This contract is valid for X years from the date of signing. After the expiration of the contract, the two parties may negotiate to renew or sign a new contract based on actual circumstances." ] }.

[0024] (2) Entity information dataset input template: { "Entity Information Example": [ "The green energy project is expected to be completed in 45 days with a total contract value of 6 million yuan." "The construction site of the green power project is in Dujiangyan." ] }.

[0025] (3) Procurement contract regulations data set input template: like Figure 2 As shown, firstly, a template instruction for the procurement contract regulations is provided, for example: { "Contract Terms": [ "If Party B finds any quality problems when delivering the product, Party A has the right to require Party B to replace it free of charge within three days and compensate Party A for its losses.", "This tender is limited to suppliers with XYZ model equipment manufactured by ABC Company." ] }.

[0026] Then specify the structured output template: (1) Structured output template of word vectorization dataset: Since the text is vectorized using a transformer-based bidirectional encoder (Bert) in step 3, the bidirectional encoder uses unsupervised learning. The bidirectional encoder randomly masks some words and tries to predict these masked words. In this way, the model can better learn word vectors in the context. Therefore, no specific labels are required to supervise the bidirectional encoder, and no structured output template is required.

[0027] (2) Entity information dataset structured output template: We use the standard entity start-inside-end-outside (BIEO, BIEO) annotation format for data annotation, where B (Begin) represents the starting position of the entity, I (Inside) represents the internal position of the entity, E (End) represents the end position of the entity, and O (Outside) represents the non-entity position. In addition, in order to distinguish between noun entities and numeric entities and facilitate subsequent integrity checks and rule checks, we divide entities into noun entities N and numeric unit entities C. In more detail, BN represents the beginning of the noun entity, IN represents the middle part of the noun entity, and EN represents the end part of the noun entity. The same applies to BC, IC, and EC. Therefore, the structured output template is: { "Entity Information Label": [ { "Entity Information Number": 0, "Entity code": "BN,IN,IN,EN,BN,EN,O,O,BC,IC,EC,O,O,BN,IN,BN,EN,BC,IC,IC,IC,EC" }, { "Entity Information Number": 1, "Entity encoding": "BN,IN,IN,EN,BN,EN,O,BN,IN,IN,EN,O,BN,IN,EN" } ] }.

[0028] Finally, generate virtual training data: (1) Word vectorization virtual dataset The seller promises that the electricity provided will come from renewable energy and meet the green electricity certification standards specified by the buyer, such as green certificates or carbon emission reduction projects. The validity period of this contract is X years from the date of signing. After the expiration of the contract, both parties may negotiate to renew the contract or sign a new contract based on actual circumstances. Table 1 Virtual dataset for word vectorization (2) Entity Information Virtual Dataset green color able source work Procedure Pre count 4 5 sky over become combine same total price 6 0 0 Ten thousand Yuan BN IN IN EN BN EN O O BC IC EC O O BN IN BN EN BI IC IC IC EC Table 2 Entity information virtual dataset In addition to generating virtual data sets through large models, users can also customize data sets according to template formats to make the model more suitable for personal use. The training data set generated by the large model and the user-defined data set are merged to generate the final virtual data set.

[0029] Step 3: Use the transformer-based bidirectional encoder model to distribute vectorize the initial text data to obtain the first word vector set consisting of word vectors.

[0030] Vectorize the text data after natural language processing. Traditional word vector representation methods, such as one-hot encoding, represent each word as a vector with a length equal to the size of the vocabulary, where only one dimension has a value of 1 and the rest are 0. However, this method loses the semantic relationship between words, and when there are too many words, there will be a dimensionality explosion. Distributed word vector representation methods have made improvements to these problems. By using a bidirectional encoder based on transformers (Bert, hereinafter referred to as a bidirectional encoder) to distribute and vectorize text data, it uses semantic knowledge learned from a large amount of text corpus to encode words, and can map words to a relatively low-dimensional continuous space. There is no dimensionality disaster, and the semantic information between words and sentences is retained, which helps to improve the efficiency of subsequent model training. At the same time, the bidirectional encoder model is built based on the Transformer model. At present, there are a large number of Chinese pre-trained models that have been pre-trained with general Chinese corpus data. Among them, the Chinese-BERT-wwm model performs better. Then use the virtual data in step 2 to fine-tune the bidirectional encoder to make the model more in line with the characteristics of green energy and power industry procurement documents. More importantly, the bidirectional encoder used in word vectorization is unsupervised learning and does not require specific data labels. Therefore, the data sets used in steps 5 and 6 can be removed from the labels as word vectorization data sets. A large amount of training data can make the bidirectional encoding model closer to the use scenarios of green power industry procurement documents, and the vectorization effect is better, so that the subsequent inspection effect will also be better. In addition, the unique masked language model of the bidirectional encoder will randomly mask some words in the input sequence and require the model to predict these masked words so that it can better handle unregistered words.

[0031] like Figure 3 As shown in the figure, the bidirectional encoder divides the text into a series of symbols, and then maps the symbols to a fixed word vector through a word embedding model. When there are multiple sentences input, the bidirectional encoder uses paragraph or sentence embedding to distinguish the two sentences. Usually all the symbols of the first sentence are added with a set paragraph embedding vector, and all the symbols of the second sentence are added with another different paragraph embedding vector to form a text vector, which helps the model understand the boundaries and relationships between sentences. The last step is to generate the position vector, which is the most important step of the bidirectional encoder. The calculation of the position vector is based on the sine and cosine functions of the periodicity and frequency characteristics, allowing the model to learn the relationship between positions and generalize well to sequences of unknown length.

[0032] For each location and each dimension , the calculation formula of the position vector is as follows: For even dimensions : For odd dimensions : in, represents the position vector; is the position index in the sequence, starting from 0; is the dimension index of the embedding vector; is the total dimension of the embedding vector, e.g. in a bidirectional encoder Usually 768.

[0033] The word vector, text vector, and position vector calculated through the above process are added together to get the final vector: in, is the vector of the corresponding symbol in the word vector matrix, is the corresponding position in the text vector matrix The vector of is the corresponding text in the text vector The bidirectional encoder then passes the resulting vector into the transformer encoding block. Each transformer encoding block consists of two main sublayers: a multi-head self-attention mechanism and a feedforward neural network, with residual connections and layer normalization techniques used between and after these two layers.

[0034] The multi-head self-attention mechanism first transforms the obtained vector Convert to query vectors , key vector Sum value vector : in, It is the transformation matrix obtained by continuous adjustment during the training process.

[0035] Then the attention weight is calculated: in, is the dimension of the key vector; is the function of the attention mechanism. In order to allow the model to learn information in different representation subspaces, the attention mechanism is replicated multiple times, and each replication is called a "head". Each head has an independent linear transformation matrix, and then the outputs of all heads are concatenated and passed through a final linear transformation to obtain the output of the multi-head self-attention.

[0036] The bidirectional encoder then uses a feedforward neural network to further process and transform the output of the multi-head self-attention layer. It usually consists of two linear transformation layers with an activation function (usually a rectified linear unit (ReLU) function) in between. The specific structure is as follows: in, Represents the output vector of multi-head self-attention; are the weight matrix and bias vector of the first linear transformation layer respectively; are the weight matrix and bias vector of the second linear transformation layer respectively; It is the function that processes and transforms the output of the multi-head self-attention layer. After using residual connections and layer normalization, the transformer encoding block in the bidirectional encoder is formed. Multiple such encoder layers are stacked together to form a deep network structure. The output of each layer is used as the input of the next layer. The final output vector will contain rich semantic and contextual information, which can be used for various downstream natural language processing tasks.

[0037] Step 4: Use the word frequency and inverse document frequency method to weight the word vectors in the first word vector set to obtain a second word vector set consisting of weighted word vectors.

[0038] Weight the word vectors obtained in step 3. The word vectors themselves have captured the semantic relationship between words, and weighting further emphasizes those words that are more important for understanding the meaning of the text. This can significantly improve the performance of the model for subsequent compliance checks, and the model can learn more valuable information from it. The word vectors are weighted using the term frequency-inverse document frequency (TF-IDF) method, which is a commonly used text statistics method used to evaluate the importance of a word in a document. TF-IDF is composed of term frequency (TF) and inverse document probability (IDF), which are calculated as follows: in, Representative words, Represents a document, Representative words In the documentation The number of times it appears in the denominator Represents the total number of all words in document d; Expressive words In the documentation The word frequency probability in .

[0039] in, is the total number of documents in the document set, Refers to the word containing The number of documents; Expressive words The inverse document probability of .

[0040] in, Represents word frequency-inverse document frequency weight After calculating the word frequency-inverse document frequency weight of the word, multiply the word vector calculated in step 2 by the word frequency-inverse document frequency weight to obtain the weighted word vector.

[0041] Step 5: Use a model combining bidirectional long short-term memory network and conditional random field to extract entity information from the second word vector set, load the integrity rules and user preset rules into the rule engine, and match the entity information with the rule engine to check whether the rule conditions are met.

[0042] The entity information in the procurement document text is extracted by combining the BiLSTM and the Conditional Random Field (CRF) model. First, the real training data is processed. The training uses the People's Daily Entity Information Dataset 1998 Edition, which provides about 23,000 sentences from the official news corpus of the People's Daily. Each sentence is segmented and detailed part-of-speech tagged.

[0043] Preprocess the data and convert the data set into the standard BIEO format. BN represents the starting point of entity nouns, IN represents the middle part of entity nouns, EN represents the end of entity nouns, BC represents the starting point of quantity nouns, and O represents non-entity nouns. In addition, IC can be used to represent the middle part of quantity nouns, and EC represents the end of quantity nouns. However, there is relatively little entity information about the procurement documents of the green energy and power industry in this data set. It is necessary to add the virtual data set generated in step 2 to fill in the entity information of the procurement documents of the green energy and power industry so that the entity extraction is more in line with the characteristics of the green energy and power industry. After the data processing is completed, input the model for training. The bidirectional long short-term memory network is an extended long short-term memory network (LSTM) network. It processes sequence data by combining two forward and backward long short-term memory networks, and designs a gate mechanism to overcome the gradient vanishing problem of the recurrent neural network. It can capture the bidirectional dependency information of each time step in the input sequence, which enables the model to consider the contextual information of the entire sentence when predicting the label of each word, thereby improving the accuracy of entity recognition. In the sequence labeling task, the conditional random field ensures the consistency of the output label sequence by considering the relationship between labels. Since the bidirectional long short-term memory network can handle variable-length sequences and complex contextual dependencies, and combined with the conditional random field, it can globally optimize the labels, it has a significant effect on the recognition of irregular and complex entities.

[0044] like Figure 4 As shown, is the input at the previous moment, is the input at the current moment, It is the hidden state of the previous moment. is the hidden state at the current moment. The first step of the long short-term memory network is to pass the forget gate Discarding some information, the data state after passing through the forget gate is calculated as follows: in, and is the weight matrix and bias term, It is the hidden state of the previous moment. is the input at the current moment, Represents the state of the data after passing through the forget gate.

[0045] The second step is to determine what information to add. and Through the input gate To determine what information needs to be updated: in, and Represents the input gate The weight matrix and bias term of ; Represents the state of the data after passing through the input gate; Then and New information is obtained through the tanh activation layer:

[0046] in, and Represents the weight matrix and bias term of the tanh activation layer.

[0047] The third step is to forget the information state obtained by the gate , the information state obtained by the input gate and new information Weighted update information :

[0048] in, Indicates the data status of the final output at the previous moment; Finally, and Filter the information through the output gate and then Weighted output results : in, and Represents the weight matrix and bias term of the output gate.

[0049] Figure 5 The bidirectional long short-term memory network model in uses a bidirectional long short-term memory network. One network processes the input sequence from left to right, and the other network processes the input sequence from right to left, so that it can capture the bidirectional information of the input sequence. The output of each time step contains richer features, which is better for capturing the features of long texts. For a text input sequence: , the state after the bidirectional long short-term memory network model is: in, is the long short-term memory network model, is the forward state, is the backward state, is the output, which is the connection between the forward and backward states. Passing the output through a linear layer, we can get the predicted label scores for each word, for example: 1.5 (BN), 0.9 (IN), 0.1 (EN), 0.005 (BC), 0.005 (IC), 0.005 (EC) and 0.08 (O). These scores will be the input of the conditional random field. The conditional random field defines a set of feature functions that describe the relationship between the features and labels at each position in the input sequence. These feature functions are used to calculate the transfer score and the emission score. The emission score indicates the score of a position being labeled with a certain label given the input features. This score is obtained by the output of the bidirectional long short-term memory network model. The transfer score indicates the score of transferring from one label to another. This score is a parameter learned during the model training process. The corresponding text label is , the conditional random field calculates the total score: in, is the total score of the conditional random field, T is the number of texts and labels, is the emission fraction, is the transfer score, and the total score is the sum of the launch score and the transfer score.

[0050] The training objective of the conditional random field is to maximize the log-likelihood of the true label sequence in the training data: in, yes As observation data As the probability distribution of the model parameters, is the set of all possible label sequences, For any possible tag sequence, For the The probability distribution of training samples, is the number of training samples, It is training samples and their corresponding label sequences, is the calculated log-likelihood.

[0051] After training, the Viterbi algorithm is used to calculate the optimal label sequence and quickly find the label sequence with the highest score: To calculate the label sequence with the highest score, after passing through the bidirectional long short-term memory network model and conditional random field, the entity information of the text is obtained.

[0052] In order to complete the integrity check and preset rule check, two rule engines are defined to load the integrity rules and user preset rules respectively: (1) Integrity rules Use the rule engine to load integrity check rules. These rules describe logical operations under specific conditions, usually in a rule language, such as: Rule 1: If the tender notice is not in the procurement document, then it is not approved. Rule 2: If the contract terms are not in the procurement document, then it will not be approved. . . .

[0053] Rule n: if signature not in purchase document then fail.

[0054] (2) User preset rules Before checking, the user defines the rules to be checked, such as: "[Total contract price] [>5 million yuan]", "[Construction period] [<1 year]", "[Number of construction workers] [<500 people]". The rule language is as follows: Rule 1: If the total contract value is less than or equal to 5 million, then fail Rule 2: If the duration is >= 1 year then fail . . .

[0055] Rule n: if number of construction workers >= 500 then fail.

[0056] Use entity information to match fact attributes (tender notice, contract terms, total contract price, construction period, etc.) in the rule engine. In order to improve the accuracy of matching, we extract synonyms of several fact attributes from the open source Chinese vocabulary and match them with entity information at the same time, such as [contract----agreement, contract, agreement, agreement, contract]. After the match is successful, the quantity information after the entity is extracted and compared with the condition value to obtain the matching status of the integrity rule and the preset rule. The integrity rule and the preset rule are the basic conditions for the compliance check of the procurement document. Therefore, only after passing the integrity rule and the preset rule can we proceed to the subsequent steps for a deeper check.

[0057] Step 6: Model the regulatory policy and industry regulations inspection problem and potential risk analysis problem as multi-classification problems respectively, and use a neural network to establish a classifier network to analyze whether the procurement document text data violates regulatory policies and industry regulations and whether there are potential risks.

[0058] After the procurement document has been checked for integrity rules and preset rules, the regulations and policies, industry regulations, and potential risk analysis are performed. The regulations and policies, industry regulations, and potential risk analysis problems are modeled as multi-classification problems. The classification problem has the following characteristics: (1) There are many categories of regulations and policies, industry-specific inspection questions, and potential risk analysis, and some categories may appear less frequently, while others may be very common, which may affect the performance of the model.

[0059] (2) There may be certain correlations between different regulations, policies, industry-specified inspection questions, and categories of potential risk analysis, and dealing with these correlations may require a more complex model.

[0060] (3) The model needs to be highly accurate; in regulation and risk analysis, misclassification can lead to serious consequences.

[0061] (4) Different categories may rely on different features. Issues related to regulations and policies may focus more on text analysis of legal provisions, while potential risk analysis may involve more data-driven features.

[0062] Therefore, it is necessary to establish a multi-classification model suitable for regulatory policies, industry regulations inspections, and potential risk analysis. At present, a better way to achieve text classification tasks is to use a bidirectional encoder to generate word vectors, and then use the first symbol position of the last layer output as the sentence identifier, followed by a fully connected layer to output the probability of each category for classification. Before the advent of the bidirectional encoder, the text convolutional neural network (TextCNN) occupied an important position in the text classification model. This is because the convolutional network (CNN) can effectively capture information in text sequences, and the classification task is essentially to capture permutation and combination features. Bidirectional encoders can often better classify some obscure sentences, and text convolutional neural networks are often more sensitive to keywords. It is possible to consider fusing the two to form a fusion model that combines the advantages of the two models. The basic bidirectional encoder model has 12 encoding block layers except for the first input layer. The first symbol vector of each encoding block layer can be used as a sentence vector. The sentence vectors from the 1st to the 12th layers are used as the input of the text convolutional neural network and then classified. The model is as follows Figure 6 shown.

[0063] For laws and regulations, industry regulations inspection and potential risk analysis, collect laws and regulations, industry regulations data, and collect relevant laws and regulations and industry regulations data from official legal texts, case law, professional legal websites and green power industry websites, and organize them into training data. Then use the virtual data generated in step 2 to fill the classified training data and train the fusion model to obtain good laws and regulations, industry regulations inspection and potential risk analysis capabilities in green energy power industry procurement documents.

[0064] Step 7: Organize and mark the items that are detected to be non-compliant with integrity rules, user preset rules, laws, policies and industry regulations, or have potential risks in the green energy and power industry procurement documents, and generate review items.

[0065] First, design the data structure of the review items, such as a key-value pair structure, where the key consists of "non-compliance point", "non-compliance type", "probability of occurrence", etc. The non-compliance of the green energy and power industry procurement documents detected is sorted and graded according to the probability of occurrence, so as to reflect the severity when the review report is generated later. Then design the template of the review report, including the format and structure of the report, including the cover, directory, review results, analysis conclusions, suggestions and other parts, insert all the review items into the pre-designed report template, give different labels according to the severity, and generate the final review report.

[0066] Step 8: Annotate and organize the review results into corpus data and update the training database. Use the new data to update the text vectorization model, text entity extraction model, regulations, policies and industry regulations inspection model and potential risk analysis model. Conduct a feedback loop, use the annotated data to continuously improve the model and review process, and apply the improved model in new reviews.

[0067] Through the feedback loop mechanism, the system of the present invention continuously uses the newly labeled data to train and optimize the model, continuously improving the accuracy and effectiveness of the review. The results of each review and labeling will be used to improve the model, making it more accurate and efficient when dealing with new review tasks.

[0068] Embodiment 2: A green energy power industry procurement document compliance checking system, the system is used to implement a green energy power industry procurement document compliance checking method as described in Embodiment 1, such as Figure 7 As shown, it includes a text processing module, a text vectorization module, a vector weighting module, an entity extraction module, an inspection and analysis module, and an inspection and sorting module.

[0069] Among them, the text processing module is used to perform natural language processing on the text data of the procurement document to obtain the initial text data; the text vectorization module is used to use a transformer-based bidirectional encoder model to perform distributed vectorization on the initial text data to obtain a first word vector set composed of word vectors; the vector weighting module is used to weight the word vectors in the first word vector set by using the word frequency and inverse document frequency method to obtain a second word vector set composed of weighted word vectors; the entity extraction module is used to use a model combining a bidirectional long short-term memory network and a conditional random field to extract the entity information in the second word vector set, load the integrity rules and user preset rules into the rule engine, and match the entity information with the rule engine to check whether the rule conditions are met; the inspection and analysis module is used to model the inspection problems of laws, policies and industry regulations and the potential risk analysis problems as multi-classification problems, and use a neural network to establish a classifier network to analyze whether the text data of the procurement document violates laws, policies and industry regulations and whether there are potential risks; the inspection and sorting module is used to sort and mark the items that are detected to be inconsistent with the integrity rules, inconsistent with the user preset rules, inconsistent with laws, policies and industry regulations or have potential risks in the procurement documents of the green energy and power industry, and generate review items.

[0070] In addition, the system may also include a virtual generation module, which uses a large model to generate virtual training data, and the virtual training data includes a word vectorization virtual data set, an entity information virtual data set, and a procurement contract regulations virtual data set; the word vectorization virtual data set is used to fill the data volume of the training data of the transformer-based bidirectional encoder model; the entity information virtual data set is used to fill the data volume of the training data of the model using a bidirectional long short-term memory network and a conditional random field; and the procurement contract regulations virtual data set is used to fill the data volume of the training data of the classifier network.

[0071] In addition, the system may also include a result feedback module, which annotates the review results and organizes them into corpus data and updates the training database, uses the new data to update the text vectorization model, text entity extraction model, regulatory policy and industry regulations inspection model and potential risk analysis model, conducts a feedback loop, uses the annotated data to continuously improve the model and review process, and applies the improved model in new reviews.

[0072] Working principle: The present invention is mainly used for compliance inspection of procurement documents in the green energy and power industry. Procurement documents in the green energy and power industry have high compliance requirements and are difficult to inspect due to their numerous contents. When inspecting the compliance of procurement documents, first check whether the procurement documents are complete item by item, then review the procurement documents item by item according to the user's preset rules, then check the relevant laws, policies and industry regulations, and analyze whether there are potential risks. Finally, organize the review results, generate and annotate review items, and significantly reduce the time and labor intensity of manual review through the automated review process. The system can quickly process a large number of procurement documents, greatly improving the review efficiency.

[0073] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0074] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0077] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for checking compliance of procurement documents in the green energy power industry, characterized in that: The following steps are involved: Perform natural language processing on the text data of the procurement document to obtain initial text data; A transformer-based bidirectional encoder model is used to perform distributed vectorization on the initial text data to obtain a first word vector set consisting of word vectors; The word vectors in the first word vector set are weighted by using the word frequency and inverse document frequency method to obtain a second word vector set consisting of the weighted word vectors; A model combining a bidirectional long short-term memory network and a conditional random field is used to extract entity information from the second word vector set, the integrity rules and user preset rules are loaded into the rule engine, and the entity information is matched with the rule engine to check whether the rule conditions are met; The regulatory policy and industry regulations inspection problem and potential risk analysis problem are modeled as multi-classification problems respectively. A classifier network is established by using a neural network to analyze whether the procurement document text data violates regulatory policies and industry regulations and whether there are potential risks. The detected items that do not comply with integrity rules, user preset rules, regulations, policies and industry provisions, or have potential risks are sorted and marked in the green energy and power industry procurement documents, and review items are generated.

2. According to claim 1, a green energy power industry procurement document compliance inspection method is characterized by: The process of using the transformer-based bidirectional encoder model to perform distributed vectorization on the initial text data is specifically as follows: Input the initial text data into the bidirectional encoder for processing to obtain a vector composed of the sum of the word vector, the text vector and the position vector; The vector of the bidirectional encoder is passed into the transformer encoding block, and the output is a word vector containing semantic and contextual information.

3. A green energy power industry procurement document compliance inspection method according to claim 2, characterized in that: The process of inputting the initial text data into the bidirectional encoder for processing is specifically as follows: The initial text data is segmented into a series of symbols through a bidirectional encoder, and the symbols are mapped to a fixed word vector through a word embedding model; When there are multiple sentences input, the bidirectional encoder uses paragraph or sentence embedding to distinguish the two sentences. All the symbols of the first sentence are added with a set paragraph embedding vector, and all the symbols of the second sentence are added with another different paragraph embedding vector to form a text vector. The relationship between positions is learned based on sine and cosine functions with periodicity and frequency characteristics to obtain a position vector.

4. According to claim 2, a green energy power industry procurement document compliance inspection method is characterized in that each The transformer encoding blocks are composed of two sub-layers of a multi-head self-attention mechanism and a feed-forward neural network, and residual connections and layer normalization techniques are used between the two sub-layers and afterwards.

5. According to claim 1, a green energy power industry procurement document compliance inspection method is characterized by: The process of weighting the word vectors in the first word vector set by using the word frequency and inverse document frequency method is specifically as follows: Calculate the word frequency and inverse document probability of the word vector respectively, and calculate the word frequency and inverse document frequency weights of the corresponding word vector by the product of the word frequency and the inverse document probability; Multiply the word vector by the corresponding word frequency-inverse document frequency weight to obtain the weighted word vector.

6. According to claim 1, a green energy power industry procurement document compliance inspection method is characterized by: The classifier network is constructed by fusing the bidirectional encoder model and the text convolutional neural network model. The fusion process is as follows: The bidirectional encoder model removes the first input layer and retains 12 encoding block layers. The first symbol vector of each encoding block layer is regarded as the sentence vector. And, the sentence vectors from the 1st to the 12th layer are used as the input of the text convolutional neural network and then classified.

7. According to claim 1, a green energy power industry procurement document compliance inspection method is characterized by: The method further includes: Generate virtual training data using a large model, including a word vectorization virtual data set, an entity information virtual data set, and a procurement contract regulations virtual data set; The training data of the transformer-based bidirectional encoder model is padded with word vectorization virtual datasets; The training data of the model combining the bidirectional long short-term memory network and the conditional random field is filled with data through the entity information virtual data set; And, the training data of the classifier network is filled with data volume through the virtual data set of procurement contract regulations.

8. A green energy power industry procurement document compliance inspection method according to claim 7, characterized in that: The method further includes: Users customize data sets according to template formats; The virtual training data and the user-defined data set are merged to generate the final virtual data set.

9. A green energy power industry procurement document compliance inspection method according to claim 1, characterized in that: The method further includes: When the integrity rules and user preset rules check meet the rule conditions, the regulatory policy and industry regulations check is initiated; Also, when the regulatory policies and industry regulations check shows no violation of regulatory policies and industry regulations, initiate potential risk analysis.

10. A green energy power industry procurement document compliance inspection system, characterized by: The system is used to implement a green energy power industry procurement document compliance checking method as described in any one of claims 1 to 9, comprising: A text processing module is used to perform natural language processing on the text data of the procurement document to obtain initial text data; A text vectorization module, used for performing distributed vectorization on the initial text data by using a transformer-based bidirectional encoder model to obtain a first word vector set consisting of word vectors; A vector weighting module, used for weighting the word vectors in the first word vector set by using a word frequency and inverse document frequency method to obtain a second word vector set consisting of weighted word vectors; An entity extraction module is used to extract entity information from the second word vector set using a model combining a bidirectional long short-term memory network and a conditional random field, load the integrity rules and user preset rules into the rule engine, and match the entity information with the rule engine to check whether the rule conditions are met; The inspection and analysis module is used to model the inspection of laws, policies and industry regulations and the analysis of potential risks as multi-classification problems. A classifier network is established by using a neural network to analyze whether the text data of procurement documents violates laws, policies and industry regulations and whether there are potential risks. The inspection and sorting module is used to sort and mark the items that are detected to be non-compliant with integrity rules, user preset rules, regulations, policies and industry regulations, or have potential risks in the green energy and power industry procurement documents, and generate review items.

Citation Information

Patent Citations

  • Method, device and equipment for extracting information

    CN107783960A

  • Title text processing method and device, electronic equipment and storage medium

    CN111507097A

  • Inspection service basic rule extraction method and device

    CN113901218A

  • Text semantic retrieval method and device for science and technology resource information of experts and scholars

    CN114840645A

  • Text auditing method, device and equipment and storage medium

    CN117033626A

Cited By

  • Informatization project compliance inspection method, device, equipment, medium and product

    CN119623453A

  • Purchase file qualification fairness intelligent review method based on multi-technology fusion

    CN120278685A

  • Intelligent Review Method for Eligibility and Fairness of Procurement Documents Based on Multi-Technology Integration

    CN120278685B