Comparative detection method, device and electronic equipment for contract text

Abstract

The present application relates to a method, device and electronic device for comparing and detecting contract texts, wherein the comparison and detection method comprises: determining character pairs between two contract texts, respectively judging whether two characters in each character pair are the same, if not, determining the character pair as an abnormal character pair, and the character is represented by at least one byte of eight bits; calculating the information entropy value and similarity of any abnormal character pair, and adding a first mark to the abnormal character pair whose similarity is less than a threshold, and the first mark is used to mark the difference characters between the two contract texts. The present invention can correct the abnormal character pairs caused by OCR recognition errors to a certain extent, thereby improving the comparison and detection accuracy of the contract text, and solving the problem that the existing comparison and detection method of the contract text cannot correct the character recognition error.

Description

Technical Field

[0001] The present application relates to the field of text recognition, and in particular, to a method, apparatus, and electronic device for comparing and detecting contract texts. Background Art

[0002] The method for comparing and detecting contract texts is used to identify whether two contracts are exactly the same. In theory, the characters between two exactly the same contracts are in one-to-one correspondence and the character pairs are the same. The comparison and detection is to determine whether the character pairs between two contracts are the same after character recognition. If there are different character pairs, they will be marked. If not, it is determined that the two contracts are exactly the same. Due to the error in character recognition, a certain character may be recognized as a similar character. For example, the character "王" may be recognized as the character "玉". The existing method for comparing and detecting contract texts cannot correct the above character recognition error, so the originally same character pairs will be recognized as different character pairs, and thus the accuracy of comparing and detecting contract texts is relatively low. For such a method for comparing and detecting contract texts, it may be necessary for the back-end staff to conduct a review and correction. However, the review by the back-end staff takes time, resulting in a long time for the comparison and detection and making it difficult to achieve real-time comparison and detection.

[0003] Regarding the problem that the existing method for comparing and detecting contract texts cannot correct character recognition errors, no effective solution has been proposed yet. Summary of the Invention

[0004] In the present invention, a method, apparatus, and electronic device for comparing and detecting contract texts are provided to solve the problem that the existing method for comparing and detecting contract texts cannot correct character recognition errors.

[0005] In a first aspect, the present invention provides a method for comparing and detecting contract texts, including:

[0006] Determine the character pairs between two contract texts, and respectively determine whether the two characters in each character pair are the same. If not, determine the character pair as an abnormal character pair, where the character is represented by at least one byte of eight-bit bits;

[0007] Calculate the information entropy value H (X) and similarity sim :

[0008]

[0009] Wherein, m represents the total number of bytes in the character, b i represents the number of different bit positions between the i-th byte of the two characters in the character pair, sigmoid represents the activation function;

[0010] A first mark is added to abnormal character pairs whose similarity is less than a threshold, and the first mark is used to mark the different characters between the two contract texts; a second mark is added to abnormal character pairs whose similarity is greater than the threshold, and the second mark is used to mark the incorrectly recognized characters between the two contract texts.

[0011] In a second aspect, the present invention provides a contract text comparison detection device, comprising:

[0012] A character comparison module, used to determine the character pairs between the two contract texts, and to determine whether the two characters in each character pair are the same. If not, the character pair is determined to be an abnormal character pair, and the characters are represented by at least one byte of eight bits;

[0013] Data calculation module, used to calculate the information entropy value of any abnormal character pair H (X) and similarity sim :

[0014]

[0015] in, m Indicates the total number of bytes in a character, b i Represents the number of different bits between the i-th bytes of the two characters in the character pair, sigmoid represents the activation function;

[0016] The character marking module is used to add a first mark to abnormal character pairs whose similarity is less than a threshold, and the first mark is used to mark the different characters between the two contract texts; and to add a second mark to abnormal character pairs whose similarity is greater than a threshold, and the second mark is used to mark the incorrectly recognized characters between the two contract texts.

[0017] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the contract text comparison detection method described in the first aspect.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the contract text comparison and detection method described in the first aspect.

[0019] Compared with the related art, the present invention can correct abnormal character pairs caused by OCR recognition errors to a certain extent, thereby improving the accuracy of contract text comparison detection, eliminating the need for staff to perform secondary correction in the background, reducing the comparison detection cycle, and achieving real-time detection. The contract text comparison detection method of the present invention solves the problem that the existing contract text comparison detection method cannot correct character recognition errors.

[0020] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a comparison detection method of a contract text in some embodiments of the present invention;

[0022] Figure 2 It is a flow chart of a comparison and detection method of a contract text in other embodiments of the present invention. DETAILED DESCRIPTION

[0023] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0024] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0025] The comparison detection method of the contract text provided by the present invention is described below through an embodiment.

[0026] Reference Figure 1 and Figure 2 In this embodiment, the process includes at least step S110, step S120, step S130, and may also include step S140, step S150, step S160 and step S170.

[0027] Step S110, determine the character pairs between the two contract texts, and judge whether the two characters in each character pair are the same. If not, determine the character pair as an abnormal character pair, and the character is represented by at least one byte of eight bits.

[0028] The two contract texts are the source contract text and the contract text to be compared. In theory, the two contract texts should be exactly the same, that is, the characters between the two contract texts are one-to-one corresponding, and the corresponding characters are exactly the same. The corresponding characters constitute a character pair. In actual situations, some characters in the contract text that needs to be compared may be modified so that the corresponding characters are different. This embodiment needs to identify different character pairs, and any relevant technology can be used to determine the character pairs between the two contract texts. For example, it can be determined based on the relative position relationship of the characters. Exemplarily, the first character of the first line of the first page of the first contract text corresponds to the first character of the first line of the first page of the second contract text. At the computer data processing level, a character is represented by at least one byte, and a byte is composed of an eight-bit binary value. Usually, Chinese characters are represented by three bytes, and non-Chinese characters (symbols, letters, etc.) are represented by one byte.

[0029] In this embodiment, text difference detection is achieved through byte-level comparison. The bytes of characters are obtained by encoding the characters in UTF-8, and bitwise operations are used to improve the comparison efficiency. In order to distinguish Chinese characters from non-Chinese characters, for example, assume byte For a byte to be read, if , which represents the bytes of non-Chinese characters, if , which represents the first byte of a Chinese character.

[0030] The formula for determining whether the two characters in each character pair are the same is:

[0031]

[0032] in, represents the value of the jth bit of the i-th byte in the first character, represents the value of the jth bit of the i-th byte in the second character, represents the XOR operation,m Indicates the total number of bytes in a character. For Chinese characters m =3, for non-Chinese characters m =1, the first character and the second character form a character pair, Sum Indicates the comparison result. Sum When it is 0, the two characters are considered the same. Sum When non-zero, the two characters are considered different.

[0033] In this embodiment, if the result Sum If it is 0, it means that the two characters are exactly the same. Otherwise, it means that there is a difference between the compared character and the source character, that is, the following relationship exists:

[0034]

[0035] Assume that the character pair to be compared is ,but , Indicates the bit value of the two characters to be compared at the binary level. If the characters are Chinese, then , indicating that Chinese characters occupy three bytes; if the character is non-Chinese, there are , indicating that non-Chinese characters occupy one byte.

[0036] It should be noted that, in other embodiments, other methods may also be used to determine whether two characters in each character pair are identical.

[0037] Before determining the character pairs between the two contract texts, character information needs to be obtained. In this embodiment, since text difference detection is achieved through byte-level comparison, the OCR (optical character recognition) recognition method can be used to recognize the contract text image to obtain character information. The traditional contract text comparison detection method needs to be integrated with enterprise systems such as ERP, and this system integration method requires complex system adaptation and interface development. The contract text comparison detection method in this embodiment is integrated with a lightweight OCR text recognition model. Users only need to upload pictures of the relevant contract text to quickly and accurately identify and extract the content of the contract text without the need for complex system integration, which greatly reduces system deployment and maintenance costs.

[0038] Since there may be certain errors in the OCR recognition method or other character recognition methods. For Chinese characters, such errors are most likely to occur between some characters with similar shapes. For example, recognizing "王" as "玉" or "佐" as "佑". The same is true for non-Chinese characters, such as the number "1", the English letter "I", and the character "!". As a result, the originally identical character pairs form different character pairs, that is, normal character pairs are recognized as abnormal character pairs. It should be noted that the above errors are caused by character recognition, not by step S110 in this embodiment. To correct this error to a certain extent, this embodiment performs a re-recognition and judgment on the abnormal character pairs through step S120.

[0039] Step S120, calculate the information entropy value H (X) and similarity sim :

[0040]

[0041] Among them, m represents the total number of bytes of a single character in the character pair, b i represents the number of different bit positions between the i-th byte of the two characters in the character pair, sigmoid represents the activation function.

[0042] Specifically, the greater the uncertainty of a single system, the greater its information entropy value, and vice versa. This embodiment calculates the similarity of each abnormal character pair using the information entropy value of the abnormal character pair. Among them, represents the error probability of the i-th byte in the character pair, m is the number of bytes in the character pair to be calculated. Usually, for Chinese characters m is 3, and for non-Chinese characters m is 1. Through the above formula, the information entropy value and similarity of each abnormal character pair can be calculated. The similarity of the i-th abnormal character pair can be expressed as sim i or . t i and s i respectively represent the i-th character in the two contract texts.

[0043] Through step S120, the similarity between the two characters in each abnormal character pair can be calculated.

[0044] Step S130, attach a first mark to the abnormal character pairs with similarity less than the threshold. The first mark is used to mark the different characters between the two contract texts.

[0045] Considering that character recognition errors usually only identify a certain character as a relatively similar character, and character replacements or character errors in contract texts usually have large character differences (although there may also be small character differences, the overall probability is low). Therefore, for abnormal character pairs with a large similarity, it is more likely that they are caused by character recognition. For example, in the originally identical character pair, if one character is recognized as a similar character, then the character still has a large similarity with the other character. For abnormal character pairs with a small similarity, it is unlikely to be caused by character recognition, and usually there are large differences in the characters themselves. Therefore, in this embodiment, abnormal character pairs with a similarity less than a threshold are marked as difference characters, and the user is prompted through a first mark that the marked characters are difference characters.

[0046] Through the above similarity comparison, the abnormal character pairs caused by OCR recognition errors can be corrected to a certain extent, thereby improving the accuracy of the comparison detection of the contract text, eliminating the need for staff to perform secondary corrections in the background, reducing the comparison detection cycle, and achieving real-time detection. The comparison detection method of the contract text in this embodiment solves the problem that the existing comparison detection method of the contract text cannot correct character recognition errors.

[0047] Considering that step S130 cannot completely and accurately correct abnormal characters caused by OCR recognition errors, in order to improve the correction accuracy, this embodiment may also execute steps S140, S150, S160 and S170.

[0048] Step S140: Add a second mark to the abnormal character pairs whose similarity is greater than a threshold, where the second mark is used to mark the incorrectly recognized characters between the two contract texts.

[0049] Although the comparison detection method in this embodiment determines that the characters in the abnormal character pairs with similarity greater than the threshold are the same (different only due to OCR recognition errors), the second mark is still added to remind the user to watch. Therefore, the user can choose to fully believe the detection result given by the comparison detection method in this embodiment. For example, when the first mark is not given, the two contract documents are considered to be exactly the same. The user can also choose to confirm the characters given the first mark and the second mark, and provide feedback on the character pairs with incorrect marks. For example, for character pairs that are attached with the first mark but are the same, or for character pairs that are attached with the second mark but are different, feedback on marking errors can be provided; and for character pairs that are attached with the first mark and are different, or for character pairs that are attached with the second mark and are the same, feedback on marking correctness can be provided. Through user feedback, the comparison detection method in this embodiment can iteratively update the threshold and similarity of each abnormal character pair, thereby improving the correction accuracy of the abnormal character pair. Specifically, steps S150, S160 and S170 are executed.

[0050] Step S150: displaying the two contract texts with the first mark and / or the second mark attached via an interactive display device. The interactive display device may be a terminal device such as a mobile phone, a computer, etc. having a display or a screen.

[0051] Step S160, obtaining user feedback in response to the first mark and the second mark on the contract text, the user feedback being correct marking and incorrect marking. The contract text comparison detection method in this embodiment provides a user feedback interface. For example, after the user selects the corresponding mark, the user is provided with options of correct marking and incorrect marking. It should be noted that the user can selectively provide feedback, and the contract text comparison detection method in this embodiment does not force the user to provide feedback.

[0052] Step S170: updating the threshold and the similarity of each character pair according to user feedback.

[0053] Among them, the similarity update formula of character pairs is:

[0054]

[0055] in, and Respectively represent the similarity before and after the update of the i-th character pair, represents the learning rate, represents the derivative operation, L i represents the total loss of the token for the ith character pair, represents the total number of feedbacks for the i-th character pair, y kDenote the k-th user feedback, when the user feedback is marked as correct y k it is 1, when the user feedback is marked as incorrect y k it is 0, l which represents the update generation.

[0056] Through the above similarity update formula, for each character pair, the corresponding total marking loss can be calculated first, and then the similarity is updated. Among them, the total number of feedbacks is equal to the number of feedbacks marked correctly plus the number of feedbacks marked incorrectly. It should be noted that in this embodiment, the similarity obtained from the similarity calculation formula above is used as the initial similarity value.

[0057] The update formula for the threshold is:

[0058]

[0059] where, and respectively represent the thresholds before and after the update for the i-th character pair, represents the adjustment coefficient, represents the number of feedbacks marked correctly for the i-th character pair, represents the total number of feedbacks for the i-th character pair, l and represents the update generation. The purpose of setting to subtract 0.5 is to ensure that the threshold of the previous generation is maintained when the positive and negative feedbacks are balanced, so that the method can adaptively adjust the judgment criterion.

[0060] From the above two update formulas, it can be seen that for each character pair, there are its own independent similarity and threshold, and both the similarity and threshold updates are based on the user feedback of itself. For a certain character pair, by continuously updating the similarity and threshold, the marking accuracy of this character pair can be improved.

[0061] It should be noted that each character pair is distinguished based on the character content, and the serial number of the corresponding character pair is marked based on the character content, rather than based on its position in the contract text. For example, for the character pair "王" and "玉", its serial number can be defined as 109. Then, in any two contract texts to be compared, when the character pair "王" and "玉" appears, it is the 109th character pair. Then, the contract text comparison detection method in this embodiment will mark the 109th character pair ("王" and "玉") in the next contract text according to the similarity and threshold of the 109th character pair ("王" and "玉") updated according to the user feedback of the previous contract text.

[0062] As mentioned above, the comparison detection method of the contract text provided by the present invention has been specifically described through a specific embodiment. In summary, the comparison detection method of the contract text in the above embodiment adopts lightweight OCR text recognition technology. The user only needs to upload the picture of the relevant contract to quickly and accurately identify and extract the content of the contract text, and there is no need to perform complex system integration, which greatly reduces the deployment and maintenance costs of the method system. This embodiment is based on the text results obtained by the OCR recognition technology to perform contract comparison, but it is different from the existing text comparison method in terms of comparison. The prior art usually converts the text comparison problem into the shortest path problem, solves it through a dynamic programming algorithm, and combines manual review to ensure the comparison result. This embodiment realizes text difference detection through byte-level comparison. The text difference detection in this embodiment can achieve accurate one-to-one character comparison, which can improve the accuracy of the comparison detection method. According to the characteristics of the comparison detection method, this embodiment introduces a correction step based on reinforcement learning. It includes: similarity evaluation of information entropy calculation, adaptive threshold mechanism, and parameter optimization based on gradient descent. By using the correction steps, the recognition errors that may be caused by OCR can be effectively identified and corrected. The method is continuously optimized with use, and the accuracy continues to improve. Visual prompts are used to help users quickly locate problems. Finally, the interactive feedback mechanism of this embodiment includes: positive and negative feedback records, loss function optimization, and dynamic threshold adjustment. Using this mechanism, the comparison detection method can adaptively adjust the judgment criteria, continuously optimize the correction steps, and improve the long-term use value of the comparison detection method. In general, this embodiment realizes a smart contract comparison system that is easy to deploy and can be continuously optimized through a lightweight integration method, precise comparison detection steps, intelligent correction steps, and an interactive feedback system. While ensuring accuracy, the system greatly reduces deployment and maintenance costs, improves ease of use, and has significant practical value.

[0063] Based on the comparison detection method of the contract text provided in this embodiment, a comparison detection method device for the contract text is also provided in this embodiment, and the device is used to implement the comparison detection method of the contract text in the above embodiment, which has been described and will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0064] The comparison detection method device of the contract text comprises: a character comparison module, a data calculation module and a character marking module.

[0065] The character comparison module is used to determine the character pairs between the two contract texts, and to judge whether the two characters in each character pair are the same. If not, the character pair is determined to be an abnormal character pair, and the characters are represented by at least one byte of eight bits.

[0066] Data calculation module, used to calculate the information entropy value of any abnormal character pair H (X) and similarity sim :

[0067]

[0068] in, m Indicates the total number of bytes in a character, b i Represents the number of different bits between the i-th bytes of the two characters in the character pair, sigmoid represents the activation function;

[0069] The character marking module is used to add a first mark to abnormal character pairs whose similarity is less than a threshold, and the first mark is used to mark the different characters between the two contract texts; and to add a second mark to abnormal character pairs whose similarity is greater than a threshold, and the second mark is used to mark the incorrectly recognized characters between the two contract texts.

[0070] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0071] Based on the contract text comparison and detection method provided in this embodiment, an electronic device is also provided in this embodiment, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the contract text comparison and detection method described in this embodiment.

[0072] Based on the contract text comparison and detection method provided in this embodiment, a computer-readable storage medium is also provided in this embodiment, on which a computer program is stored. When the computer program is executed by a processor, the steps of the contract text comparison and detection method described in this embodiment are implemented.

[0073] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0074] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

Claims

1. A method for comparing and detecting contract texts, characterized in that: include: Determine the character pairs between the two contract texts, and determine whether two characters in each character pair are the same. If not, determine the character pair as an abnormal character pair, and the characters are represented by at least one byte of eight bits; Calculate the information entropy value of any abnormal character pair and similarity : in, m Indicates the total number of bytes in a character, b i Represents the number of different bits between the i-th bytes of the two characters in the character pair, sigmoid represents the activation function; A first mark is added to the abnormal character pairs whose similarity is less than a threshold value, and the first mark is used to mark the different characters between the two contract texts; a second mark is added to the abnormal character pairs whose similarity is greater than the threshold value, and the second mark is used to mark the characters between the two contract texts that are incorrectly recognized; Displaying two contract texts with the first mark and / or the second mark attached thereto through an interactive display device; Obtain user feedback in response to the first mark and the second mark on the contract text, where the user feedback includes correct marking and incorrect marking; Update the threshold and similarity of each character pair based on user feedback: in, and Respectively represent the similarity before and after the update of the i-th character pair, represents the learning rate and , represents the derivative operation, L i represents the total loss of the token for the ith character pair, represents the total number of feedbacks for the i-th character pair, y k Indicates the kth user feedback, when the user feedback is marked correct y k 1, when the user feedback is a marking error y k is 0.

2. The contract text comparison detection method according to claim 1 is characterized in that: The formula for determining whether two characters in a character pair are the same is: in, a i,j represents the value of the jth bit of the i-th byte in the first character, b i,j represents the value of the jth bit of the i-th byte in the second character, represents the XOR operation, m Indicates the total number of bytes in a character. The first and second characters form a character pair. Sum Indicates the comparison result. Sum When it is 0, the two characters are considered the same. Sum When non-zero, the two characters are considered different.

3. The contract text comparison detection method according to claim 1, characterized in that: The bytes of a character are obtained by encoding the character in UTF-8.

4. The method for comparing and detecting the contract text according to claim 1, characterized in that: The threshold update formula is: in, and Respectively represent the thresholds before and after the update for the i-th character pair, represents the adjustment coefficient and , represents the number of correct feedbacks for the mark of the i-th character pair, Represents the total number of feedbacks for the i-th character pair.

5. A contract text comparison detection device, characterized in that: include: A character comparison module, used to determine the character pairs between the two contract texts, and to determine whether the two characters in each character pair are the same. If not, the character pair is determined to be an abnormal character pair, and the characters are represented by at least one byte of eight bits; Data calculation module, used to calculate the information entropy value of any abnormal character pair and similarity : in, m Indicates the total number of bytes in a character, b i Represents the number of different bits between the i-th bytes of the two characters in the character pair, sigmoid represents the activation function; And for updating the threshold and similarity of each character pair based on user feedback: in, and Respectively represent the similarity before and after the update of the i-th character pair, represents the learning rate and , represents the derivative operation, L i represents the total loss of the token for the ith character pair, represents the total number of feedbacks for the i-th character pair, y k Indicates the kth user feedback, when the user feedback is marked correct y k 1, when the user feedback is a marking error y k is 0; A character marking module is used to attach a first mark to abnormal character pairs with a similarity less than a threshold, the first mark is used to mark the different characters between two contract texts, and to attach a second mark to abnormal character pairs with a similarity greater than a threshold, the second mark is used to mark the incorrectly recognized characters between the two contract texts; and is used to display the two contract texts with the first mark and / or the second mark attached through an interactive display device, and obtain user feedback in response to the first mark and the second mark on the contract text, the user feedback being correct marking and incorrect marking.

6. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the contract text comparison detection method described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the contract text comparison detection method described in any one of claims 1 to 4 are implemented.

Images