Electronic contract auditing method and device, nonvolatile storage medium and electronic equipment
Through the capsule neural network, the image and text features of electronic contracts are extracted and fused, combined with the abnormal detection model, the problem of low audit accuracy caused by the failure to utilize multimodal data in the existing technology is solved, and more efficient electronic contract audits and Risk management.
Patent Information
- Application Number
- CN202411027177.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-08
AI Technical Summary
The existing technology fails to make full use of multimodal data in electronic contract audits, resulting in low accuracy of audit results.
The capsule neural network is used to extract target features from the images and text of the electronic contract, fuse it into a multimodal vector, and detect it through an abnormality detection model, and use multiple decision trees to detect preset detection terms on the multimodal vector.
It improves the accuracy of electronic contract audits, can effectively identify and prevent forgery and tampering, and enhances the standardization and effectiveness of contracts.
Smart Images

Figure CN118918602A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of electronic digital data processing, and more specifically, to an electronic contract auditing method, device, non-volatile storage medium and electronic device. Background Art
[0002] When auditing electronic contracts in related technologies, usually only rule-based methods or statistical-based methods can be used to audit electronic contracts. However, the rich information of multimodal data in electronic contracts cannot be fully utilized to audit electronic contracts, resulting in low accuracy of the audit results of electronic contracts in related technologies.
[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0004] The embodiments of the present application provide an electronic contract auditing method, apparatus, non-volatile storage medium and electronic device to at least solve the technical problem of low accuracy of electronic contract auditing results caused by the failure to utilize multimodal data of electronic contracts when auditing electronic contracts in related technologies.
[0005] According to one aspect of an embodiment of the present application, an electronic contract auditing method is provided, comprising: extracting a contract text and a contract image of an electronic contract to be audited, wherein the contract image includes a signature image and a seal image in the electronic contract; extracting target image features from the contract image by a first capsule neural network, and extracting target text features from the contract text by a second capsule neural network; fusing the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract; detecting the multimodal vector by an anomaly detection model, and determining whether the electronic contract is abnormal based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects a preset detection item of the multimodal vector.
[0006] Optionally, the multimodal vector is detected through an anomaly detection model to determine whether the electronic contract is abnormal, including: obtaining anomaly evaluation results output by each decision tree in the anomaly detection model; determining an anomaly evaluation score of the multimodal vector based on the anomaly evaluation results output by each decision tree; and determining that the electronic contract is abnormal when the anomaly evaluation score is within a preset value range.
[0007] Optionally, after detecting the multimodal vector through an anomaly detection model to determine whether the electronic contract is abnormal, the electronic contract audit method also includes: when the evaluation result is that the electronic contract is abnormal, determining the anomaly detection items of the electronic contract based on the detection results of each decision tree; based on the anomaly detection items, generating and displaying recommended modification plans corresponding to the electronic contract.
[0008] Optionally, the first capsule network includes a first capsule network layer and a second capsule network layer; extracting target image features from the contract image through the first capsule neural network includes: extracting initial image features from the contract image through the first capsule network layer, wherein the first capsule network layer includes a convolutional layer; iteratively processing the initial image features through the first capsule network layer and the second capsule network layer to obtain target image features.
[0009] Optionally, iteratively processing the initial image features through the first capsule network layer and the second capsule network layer to obtain the target image features includes: determining a weight vector of the first capsule network layer; extracting initial target image features from the initial image features through the first capsule network layer; and iteratively processing the initial target image features through the first capsule network layer and the second capsule network layer according to the weight vector of the first capsule network layer, thereby obtaining the target image features.
[0010] Optionally, the second capsule neural network includes an encoder and a decoder, wherein the encoder includes a convolutional layer, a main capsule layer and a digital capsule layer connected in sequence, the main capsule layer includes a lower-level capsule layer and an upper-level capsule layer, wherein the convolutional layer is used to extract text features from the contract text, the main capsule layer is used to process the text features using a dynamic routing algorithm, and output digital capsule vectors corresponding to the text features through the digital capsule layer; the decoder is used to select a target digital capsule vector from multiple digital capsule vectors, and generate a reconstructed document based on the digital capsule vector, wherein the length of the reconstructed document is equal to the length of the contract text.
[0011] Optionally, extracting target text features from the contract text by using the second capsule neural network includes: after generating a reconstructed document, generating target text features based on the reconstructed document.
[0012] According to another aspect of an embodiment of the present application, an electronic contract auditing device is also provided, including: a first processing module, used to extract a contract text and a contract image of an electronic contract to be audited, wherein the contract image includes a signature image and a seal image in the electronic contract; a second processing module, used to extract target image features from the contract image through a first capsule neural network, and to extract target text features from the contract text through a second capsule neural network; a third processing module, used to fuse the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract; a fourth processing module, used to detect the multimodal vector through an anomaly detection model, and determine whether the electronic contract is abnormal based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects a preset detection item of the multimodal vector.
[0013] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored, wherein when the program is running, a device where the non-volatile storage medium is located is controlled to execute an electronic contract audit method.
[0014] According to another aspect of an embodiment of the present application, there is also provided an electronic device, comprising: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the electronic contract audit method is executed when the program is running.
[0015] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a computer program, which implements an electronic contract auditing method when executed by a processor.
[0016] In an embodiment of the present application, a contract text and a contract image of an electronic contract to be audited are extracted, wherein the contract image includes a signature image and a seal image in the electronic contract; target image features are extracted from the contract image by a first capsule neural network, and target text features are extracted from the contract text by a second capsule neural network; target image features and target text features are fused to obtain a multimodal vector corresponding to the electronic contract; the multimodal vector is detected by an anomaly detection model, and whether the electronic contract is abnormal is determined based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects a preset detection item of the multimodal vector. By using multiple capsule networks to extract target text features and target image features of the electronic contract respectively, and fusing them to obtain a multimodal vector, and then detecting the multimodal vector to determine whether the electronic contract is abnormal, the purpose of auditing the electronic contract based on data of various modalities of the electronic contract is achieved, thereby achieving the technical effect of improving the accuracy of the audit results, and further solving the technical problem of low accuracy of the audit results of the electronic contract caused by not utilizing the multimodal data of the electronic contract when auditing the electronic contract in the related art. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0018] Figure 1 It is a structural schematic diagram of a computer terminal (mobile terminal) provided according to an embodiment of the present application;
[0019] Figure 2 It is a flowchart of an electronic contract audit method provided according to an embodiment of the present application;
[0020] Figure 3It is a flowchart of a routing algorithm provided according to an embodiment of the present application;
[0021] Figure 4 It is a schematic diagram of the structural flow of an encoder provided according to an embodiment of the present application;
[0022] Figure 5 It is a structural schematic diagram of an electronic contract auditing device provided according to an embodiment of the present application. DETAILED DESCRIPTION
[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0025] The use of electronic contracts is becoming more and more common, and these electronic contracts provide convenience and efficiency for commercial transactions. However, electronic contracts also face the risk of forgery and fraud, which may lead to contract disputes and economic losses. At present, although there are some electronic contract auditing methods, they are often limited to a single technical field, such as rule-based methods or statistical-based methods, which cannot fully utilize the rich information of multimodal data.
[0026] In the field of electronic contract auditing, natural language processing and deep learning models can effectively improve auditing efficiency and accuracy, help enterprises better manage contract-related risks, promptly identify potential problems and take measures to reduce risks and losses during contract execution. Natural language processing and deep learning models can automatically identify, extract and analyze key information in contracts, such as terms, conditions, responsibilities and obligations, and detect electronic contract forgery and fraud. By building an intelligent auditing system, the entire process from contract import, data extraction, analysis to report generation can be automated, reducing manual intervention and improving efficiency and accuracy. However, existing electronic contract auditing methods have failed to fully utilize natural language processing and deep learning models to audit electronic contracts.
[0027] In order to solve the above problems, an electronic contract auditing method based on a capsule network is provided in an embodiment of the present application. By utilizing deep learning and natural language processing technology, combined with a multimodal learning method, the automated analysis and auditing of electronic contracts is realized, thereby effectively preventing forgery and tampering and improving the standardization and effectiveness of electronic contracts. The details are described below.
[0028] According to an embodiment of the present application, a method embodiment of an electronic contract audit method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing the electronic contract audit method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1Different configurations are shown.
[0030] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).
[0031] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the electronic contract audit method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned electronic contract audit method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0032] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0033] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).
[0034] In the above operating environment, the present application embodiment provides an electronic contract audit method, such as Figure 2 As shown, the method comprises the following steps:
[0035] Step S202, extracting the contract text and contract image of the electronic contract to be audited, wherein the contract image includes the signature image and the seal image in the electronic contract;
[0036] As an optional implementation, before extracting the contract text and contract image of the electronic contract to be audited, a large number of approved electronic contracts as positive samples and forged contracts as negative samples can be obtained in advance and included in the electronic contract database together for subsequent model training.
[0037] In some embodiments of the present application, forged contracts can be collected through network channels, and approved contracts can be obtained through the MSS (Management Support System) provided by the communication operator. Among them, the scope supported by the MSS system corresponds to the enterprise management domain in eTOM, including all non-core business processes required to support the enterprise, and there are a large number of approved electronic contracts in the MSS system. After obtaining these contract data, these contract data can be pre-processed, including normal compliance and abnormalities, and the text and image in each electronic contract data are segmented. The text is formatted, unnecessary tags and formats are removed, the plain text information is retained, and further operations such as word segmentation and word vectorization are performed. Preprocess the image information, including image enhancement and image denoising.
[0038] As an optional implementation, when obtaining the contract text of the electronic contract to be audited, it is necessary to extract text information from the electronic contract, such as terms, conditions, responsibilities and obligations, and format it to remove unnecessary tags and formats and retain the plain text information. In addition, the text information can also be preprocessed, including word segmentation, word vectorization, etc.
[0039] The extraction of contract images includes extracting image information of important subjects such as signatures and seals from electronic contracts. The extracted contract images can be further preprocessed, including image enhancement and image denoising.
[0040] Step S204, extracting target image features from the contract image by using the first capsule neural network, and extracting target text features from the contract text by using the second capsule neural network;
[0041] In the technical solution provided in step S204, the first capsule network includes a first capsule network layer and a second capsule network layer; the step of extracting target image features from the contract image through the first capsule neural network includes: extracting initial image features from the contract image through the first capsule network layer, wherein the first capsule network layer includes a convolutional layer; iteratively processing the initial image features through the first capsule network layer and the second capsule network layer to obtain target image features.
[0042] The first capsule network layer can also be called the bottom capsule layer, and the second capsule network layer can also be called the high-level capsule layer or the routing capsule layer. The first capsule network layer includes a convolutional layer for identifying the contract body text information in the contract image, the arc shape around the contract body text information, etc. The arc shape around the contract body text information is used to determine the signature information in the contract image. The second capsule network layer uses a routing by agreement algorithm to detect larger and more complex initial image feature information, such as the contour shape and overall features of the contract seal.
[0043] In some embodiments of the present application, the step of iteratively processing initial image features through the first capsule network layer and the second capsule network layer to obtain target image features includes: determining a weight vector of the first capsule network layer; extracting initial target image features from the initial image features through the first capsule network layer; and iteratively processing the initial target image features through the first capsule network layer and the second capsule network layer according to the weight vector of the first capsule network layer, thereby obtaining the target image features.
[0044] As an optional implementation, in the present application, the first capsule network layer and the second capsule network layer use a routing algorithm to iteratively process the initial image features. Figure 3 As shown. Figure 3 The i in represents the first capsule layer, that is, the low-level capsule, and j represents the second capsule layer, that is, the high-level capsule.
[0045] like Figure 3 As shown in the figure, in the process of processing the initial image features, the low-level capsule i needs to decide how to send its output vector to the high-level capsule j. The low-level capsule changes the scalar weight cij, and after the output vector is multiplied by the weight, it is sent to the high-level capsule as the input of the high-level capsule. The low-level capsule sends its output to the high-level capsule that "agrees" with this. Among them, the high-level capsule "agrees" means that the output of the low-level capsule should be in the preset value range. When the output of the low-level capsule is not in the preset value range, the iterative process will end.
[0046] Figure 3The σ in it represents the squash function, ⊙ is the dot product of the vector, r is the number of iterations, vi is the input vector, Wij is the weight, cij is the coupling coefficient, bij is a temporary variable, the initial value is 0, and it corresponds one-to-one with cij. After the iteration, this variable will be stored in cij. Updating the vector bij is the most critical step in the routing protocol algorithm. In the routing algorithm provided in the implementation of this application, the output vj of the high-level capsule and the output uij of the low-level capsule are dot-producted to expand the vector with the same direction as the high-level capsule output. The initial value of the above-mentioned weight Wij can be randomly sampled from the standardized normal distribution, and the parameters are updated using the optimizer Adam, and the loss functions of the first capsule neural network and the second capsule neural network can be set to the cross-entropy loss function.
[0047] The specific calculation formula is as follows:
[0048] u i j =W ij v i
[0049]
[0050] b ij =b ij +u ij v j
[0051] The k in the above formula is the index of the high-level capsule, which is used to indicate different high-level capsules when there are multiple high-level capsules.
[0052] As an optional implementation, the second capsule neural network includes an encoder and a decoder, wherein the encoder includes a convolutional layer, a main capsule layer and a digital capsule layer connected in sequence, the main capsule layer includes a lower-level capsule layer and an upper-level capsule layer, wherein the convolutional layer is used to extract text features from the contract text, the main capsule layer is used to process the text features using a dynamic routing algorithm, and output digital capsule vectors corresponding to the text features through the digital capsule layer; the decoder is used to select a target digital capsule vector from multiple digital capsule vectors, and generate a reconstructed document based on the digital capsule vector, wherein the length of the reconstructed document is equal to the length of the contract text.
[0053] In some embodiments of the present application, extracting target text features from the contract text by using the second capsule neural network includes: after generating a reconstructed document, generating the target text features based on the reconstructed document.
[0054] Specifically, the structure of the encoder is as follows Figure 4 As shown, Figure 4The Conv Layer in is a convolutional layer used to extract text features. PrimaryCaps represents the lower-level capsule layer in the main capsule layer, and DigitCaps represents the upper-level capsule layer in the main capsule layer, which is also the main capsule layer using the dynamic routing algorithm. The upper-level capsule layer also contains the instantiation parameters required to reconstruct the object.
[0055] As an optional implementation, the digital capsule layer can generate multiple vectors based on the processing results of the main capsule layer, and the number of digital capsule layers can also be multiple. The decoder can select a 16-dimensional vector from the multiple vectors and learn to encode it into a contract document. When training the decoder, only the correct digital capsule vectors are used, and incorrect digital capsule vectors are ignored. The decoder will accept the output of the correct digital capsule as input and reconstruct a document with a length equal to the input length. The loss function is the Euclidean distance between the reconstructed document and the input document. In addition, when training the second neural network, a reconstructor can be used to force each capsule layer in the capsule network to learn features that are useful for reconstructing the original document, so that the reconstructed document is closer to the input document.
[0056] Step S206, fusing the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract;
[0057] In the solution provided in step S206, the target image features extracted by the first capsule neural network and the target text features extracted by the second capsule neural network can be fused to obtain a multimodal electronic contract feature vector. It should be noted that both the target image features and the target text features can be embodied in the form of feature vectors.
[0058] As an optional implementation, an aggregation-based fusion method can be used to perform feature fusion. The data of various modes are aggregated to obtain a common aggregated feature, which is then mapped to the output dimension through a global mapping to obtain the final output. The specific fusion formula is as follows:
[0059]
[0060] In the above formula, h represents the global mapping, and Agg is the aggregation function. There are many ways to implement the aggregation function, such as averaging multiple modal features, concatenating multiple modal features, and self-attention. (i) represents the extracted contract features, Represents contract features of different modalities, such as text features, image features, etc. Different values of M correspond to features of different modalities.
[0061] Step S208, detecting the multimodal vector through an anomaly detection model, and determining whether the electronic contract is abnormal based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects a preset detection item of the multimodal vector.
[0062] In the technical solution provided in step S208, the multimodal vector is detected by an anomaly detection model, and the step of determining whether the electronic contract is abnormal includes: obtaining the anomaly evaluation results output by each decision tree in the anomaly detection model; determining the anomaly evaluation score of the multimodal vector based on the anomaly evaluation results output by each decision tree; and determining that the electronic contract is abnormal when the anomaly evaluation score is within a preset value range.
[0063] Specifically, during the audit phase, rule engines or machine learning models can be used to perform anomaly detection and audit based on the extracted multimodal electronic contract features. For example, potential risks in contracts can be detected, including forged seals or clauses that do not meet specifications.
[0064] As an optional implementation, the anomaly detection model used can be RandomForest or SVM. Among them, the RandomForest model will build multiple decision trees and combine the results of each decision tree to perform anomaly detection. Each decision tree will detect a preset inspection item. The anomaly detection process of the RandomForest model can be expressed by the following formula:
[0065]
[0066] In the above formula, X is the fused multimodal feature vector of the electronic contract, T i (X) is the anomaly detection result of the i-th decision tree, N is the number of decision trees, and RF(X) represents the detection result of the RandomForest model. In addition, pruning control can be used to avoid overfitting when constructing a random forest model.
[0067] As an optional implementation, each decision tree can output 1 when the test result is passed, and can output 0 when the test result is abnormal. The weighted average of the output results of each decision tree can be used as the final test score. In addition, the weights of different decision trees are used to reflect the importance of the test items corresponding to the decision trees, which can be set by the staff.
[0068] In some embodiments of the present application, when training the anomaly detection model, a labeled contract dataset D may be used.train To train the RandomForest model. The training process involves the construction of decision trees and parameter tuning so that the model can effectively capture abnormal situations in the contract.
[0069] After the model training is completed, the anomaly detection model can be applied to the unknown contract dataset D test The model will perform anomaly detection on each contract and output an anomaly score. You can determine whether a contract is abnormal by setting a threshold and comparing the anomaly score with the threshold. For example, if the anomaly score is greater than the preset threshold, the contract is determined to be normal. It should be noted that when the decision tree is set to output 1 when the detection result is abnormal and 0 when it is normal, the contract is determined to be normal when the anomaly score is less than the preset threshold.
[0070] As an optional implementation, after detecting the multimodal vector through an anomaly detection model to determine whether the electronic contract is abnormal, the electronic contract audit method also includes: when the evaluation result is that the electronic contract is abnormal, determining the anomaly detection items of the electronic contract based on the detection results of each decision tree; based on the anomaly detection items, generating and displaying recommended modification plans corresponding to the electronic contract.
[0071] The invention adopts the method of extracting the contract text and contract image of the electronic contract to be audited, wherein the contract image includes the signature image and the seal image in the electronic contract; extracting the target image features from the contract image by a first capsule neural network, and extracting the target text features from the contract text by a second capsule neural network; fusing the target image features and the target text features to obtain the multimodal vector corresponding to the electronic contract; detecting the multimodal vector by an anomaly detection model, and determining whether the electronic contract is abnormal according to the detection result, wherein the anomaly detection model includes a plurality of decision trees, and each of the plurality of decision trees detects the preset detection items of the multimodal vector. By using a plurality of capsule networks to extract the target text features and the target image features of the electronic contract respectively, and fusing them to obtain the multimodal vector, and then detecting the multimodal vector to determine whether the electronic contract is abnormal, the purpose of auditing the electronic contract according to the data of various modes of the electronic contract is achieved, thereby realizing the technical effect of improving the accuracy of the audit result, and further solving the technical problem of low accuracy of the audit result of the electronic contract caused by not utilizing the multimodal data of the electronic contract when auditing the electronic contract in the related technology.
[0072] The present application embodiment provides an electronic contract auditing device, Figure 5 is a schematic diagram of the structure of the device. Figure 5It can be seen that the device includes: a first processing module 50, which is used to extract the contract text and contract image of the electronic contract to be audited, wherein the contract image includes the signature image and the seal image in the electronic contract; a second processing module 52, which is used to extract the target image features from the contract image through the first capsule neural network, and to extract the target text features from the contract text through the second capsule neural network; a third processing module 54, which is used to fuse the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract; a fourth processing module 56, which is used to detect the multimodal vector through an anomaly detection model, and determine whether the electronic contract is abnormal based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects a preset detection item of the multimodal vector.
[0073] In some embodiments of the present application, the first capsule network includes a first capsule network layer and a second capsule network layer; the step of the second processing module 52 extracting target image features from the contract image through the first capsule neural network includes: extracting initial image features from the contract image through the first capsule network layer, wherein the first capsule network layer includes a convolutional layer; iteratively processing the initial image features through the first capsule network layer and the second capsule network layer to obtain the target image features.
[0074] In some embodiments of the present application, the second processing module 52 iteratively processes the initial image features through the first capsule network layer and the second capsule network layer, and the step of obtaining the target image features includes: determining the weight vector of the first capsule network layer; extracting the initial target image features from the initial image features through the first capsule network layer; and iteratively processing the initial target image features through the first capsule network layer and the second capsule network layer according to the weight vector of the first capsule network layer, so as to obtain the target image features.
[0075] In some embodiments of the present application, the second capsule neural network includes an encoder and a decoder, wherein the encoder includes a convolutional layer, a main capsule layer and a digital capsule layer connected in sequence, the main capsule layer includes a lower-level capsule layer and an upper-level capsule layer, wherein the convolutional layer is used to extract text features from the contract text, the main capsule layer is used to process the text features using a dynamic routing algorithm, and output digital capsule vectors corresponding to the text features through the digital capsule layer; the decoder is used to select a target digital capsule vector from multiple digital capsule vectors, and generate a reconstructed document based on the digital capsule vector, wherein the length of the reconstructed document is equal to the length of the contract text.
[0076] In some embodiments of the present application, the step of extracting target text features from the contract text by the second processing module 52 through the second capsule neural network includes: after generating the reconstructed document, generating the target text features based on the reconstructed document.
[0077] In some embodiments of the present application, the fourth processing module 56 detects the multimodal vector through an anomaly detection model, and the step of determining whether the electronic contract is abnormal includes: obtaining the anomaly evaluation results output by each decision tree in the anomaly detection model; determining the anomaly evaluation score of the multimodal vector based on the anomaly evaluation results output by each decision tree; and determining that the electronic contract is abnormal when the anomaly evaluation score is within a preset value range.
[0078] In some embodiments of the present application, after detecting the multimodal vector through the anomaly detection model to determine whether the electronic contract is abnormal, the fourth processing module 56 is also used to: when the evaluation result is that the electronic contract is abnormal, determine the anomaly detection items of the electronic contract based on the detection results of each decision tree; based on the anomaly detection items, generate and display the recommended modification plan corresponding to the electronic contract.
[0079] It should be noted that the various modules in the above-mentioned electronic contract auditing device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.
[0080] According to an embodiment of the present application, a non-volatile storage medium is also provided, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the following electronic contract auditing method: extracting the contract text and contract image of the electronic contract to be audited, wherein the contract image includes the signature image and the seal image in the electronic contract; extracting target image features from the contract image through a first capsule neural network, and extracting target text features from the contract text through a second capsule neural network; fusing the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract; detecting the multimodal vector through an anomaly detection model, and determining whether the electronic contract is abnormal based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects a preset detection item of the multimodal vector.
[0081] According to an embodiment of the present application, an electronic device is also provided, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the following electronic contract auditing method is executed when the program is running: extracting the contract text and contract image of the electronic contract to be audited, wherein the contract image includes a signature image and a seal image in the electronic contract; extracting target image features from the contract image through a first capsule neural network, and extracting target text features from the contract text through a second capsule neural network; fusing the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract; detecting the multimodal vector through an anomaly detection model, and determining whether the electronic contract is abnormal based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects a preset detection item of the multimodal vector.
[0082] According to an embodiment of the present application, a computer program product is also provided, including a computer program, which implements the following electronic contract auditing method when executed by a processor: extracting the contract text and contract image of the electronic contract to be audited, wherein the contract image includes the signature image and the seal image in the electronic contract; extracting target image features from the contract image through a first capsule neural network, and extracting target text features from the contract text through a second capsule neural network; fusing the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract; detecting the multimodal vector through an anomaly detection model, and determining whether the electronic contract is abnormal based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects a preset detection item of the multimodal vector.
[0083] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0084] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0087] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.
[0088] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. An electronic contract auditing method, characterized in that: include: Extracting the contract text and contract image of the electronic contract to be audited, wherein the contract image includes the signature image and the seal image in the electronic contract; Extracting target image features from the contract image by a first capsule neural network, and extracting target text features from the contract text by a second capsule neural network; Fusion of the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract; The multimodal vector is detected by an anomaly detection model, and whether the electronic contract is abnormal is determined based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects preset detection items of the multimodal vector.
2. The electronic contract auditing method according to claim 1, characterized in that: Detecting the multimodal vector by using an anomaly detection model to determine whether the electronic contract is abnormal includes: Obtaining anomaly evaluation results output by each decision tree in the anomaly detection model; Determining an abnormality evaluation score of the multimodal vector according to the abnormality evaluation results output by each of the decision trees; When the abnormality evaluation score is within a preset value range, the electronic contract is determined to be abnormal.
3. The electronic contract auditing method according to claim 1, characterized in that: After detecting the multimodal vector by an anomaly detection model to determine whether the electronic contract is abnormal, the electronic contract audit method further includes: When the evaluation result is that the electronic contract is abnormal, determining the abnormal detection items of the electronic contract according to the detection results of each of the decision trees; Based on the abnormal detection items, a suggested modification plan corresponding to the electronic contract is generated and displayed.
4. The electronic contract auditing method according to claim 1, characterized in that: The first capsule network includes a first capsule network layer and a second capsule network layer; extracting target image features from the contract image by using the first capsule neural network includes: Extracting initial image features from the contract image through the first capsule network layer, wherein the first capsule network layer includes a convolutional layer; The initial image features are iteratively processed through the first capsule network layer and the second capsule network layer to obtain the target image features.
5. The electronic contract auditing method according to claim 4, characterized in that: The target image features are obtained by iteratively processing the initial image features through the first capsule network layer and the second capsule network layer, including: Determining a weight vector of the first capsule network layer; Extracting initial target image features from the initial image features through the first capsule network layer; The initial target image feature is iteratively processed by the first capsule network layer and the second capsule network layer according to the weight vector of the first capsule network layer, so as to obtain the target image feature.
6. The electronic contract auditing method according to claim 1, characterized in that: The second capsule neural network includes an encoder and a decoder, wherein: The encoder comprises a convolution layer, a main capsule layer and a digital capsule layer connected in sequence, wherein the main capsule layer comprises a lower capsule layer and an upper capsule layer, wherein the convolution layer is used to extract text features from the contract text, the main capsule layer is used to process the text features using a dynamic routing algorithm, and output a digital capsule vector corresponding to the text features through the digital capsule layer; The decoder is used to select a target digital capsule vector from the multiple digital capsule vectors, and generate a reconstructed document according to the digital capsule vector, wherein the length of the reconstructed document is equal to the length of the contract text.
7. The electronic contract auditing method according to claim 6, characterized in that: Extracting target text features from the contract text by the second capsule neural network includes: After the reconstructed document is generated, the target text features are generated according to the reconstructed document.
8. An electronic contract auditing device, characterized in that: include: A first processing module is used to extract the contract text and contract image of the electronic contract to be audited, wherein the contract image includes the signature image and the seal image in the electronic contract; A second processing module is used to extract target image features from the contract image through a first capsule neural network, and to extract target text features from the contract text through a second capsule neural network; A third processing module, configured to fuse the target image features and the target text features to obtain a multimodal vector corresponding to the electronic contract; The fourth processing module is used to detect the multimodal vector through an anomaly detection model, and determine whether the electronic contract is abnormal based on the detection result, wherein the anomaly detection model includes multiple decision trees, and each of the multiple decision trees detects preset detection items of the multimodal vector.
9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the electronic contract audit method described in any one of claims 1 to 7.
10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program, when running, executes the electronic contract audit method described in any one of claims 1 to 7.
11. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the electronic contract auditing method according to any one of claims 1 to 7.
Citation Information
Cited By
Electric power business data analysis processing method and system
CN120449066A
Intelligent contract management method and device for multi-modal large model, and medium
CN120523784A
Intelligent contract auditing method and device based on multi-modal large model and medium
CN120524239A
