A text difference rate determination method, device and electronic equipment

Abstract

The application discloses a text difference rate determination method and device and electronic equipment. Two files to be compared are respectively parsed to obtain first text content and second text content; the first text content and the second text content are processed based on a file difference analysis algorithm to determine a text matching result; the text matching result is divided into a plurality of sub-text sequences based on text paragraph information in the first text content, and a to-be-verified paragraph group corresponding to the plurality of sub-text sequences is determined; a target similar paragraph group is determined based on the to-be-verified paragraph group and a similarity judgment model, and a total number of characters corresponding to the target similar paragraph group is determined; and a difference rate is determined based on a common character number, an exclusive character number, the total number of characters and a difference rate function, thereby solving the problems of large workload, low efficiency and high error rate in a file auditing process, improving the accuracy of determining the file difference rate, improving the efficiency of determining the file difference and reducing the error frequency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a text difference rate determination method and device and electronic equipment. BACKGROUND

[0002] Nowadays, many enterprises need to process a large number of files such as contracts, instructions, and tenders, and the similarity of such files is high during the revision of different versions, and only a small part of the differences exist. If the review version and the signed version are inconsistent, it may cause disputes in cooperation and even incalculable losses for the enterprise.

[0003] Currently, the review and verification of different versions of files rely on staff to flip through each page, find and compare, and manually identify differences. In addition, some file comparison tools have been developed to extract the text of standard format files, and then use a public sequence-based method or a semantic recognition model to compare and determine the difference content.

[0004] However, the manual review method has the problems of large workload, low efficiency, and easy errors, and the file comparison tool method usually only considers local differences in text fields or only determines global differences between files based on a semantic recognition model, and cannot accurately estimate the differences between different versions of files. SUMMARY

[0005] The present application provides a text difference rate determination method, device and electronic equipment, which improves the accuracy of determining the file difference rate, improves the efficiency of determining the file difference, and reduces the error frequency.

[0006] In a first aspect, the present application provides a text difference rate determination method, which comprises:

[0007] The two files to be compared are respectively parsed and processed to obtain first text content and second text content; wherein the two files to be compared have the same substantive content and different texts, and the text content includes text paragraph information;

[0008] The first text content and the second text content are processed based on a file difference analysis algorithm to determine a text matching result; wherein the text matching result is a long sequence including common characters, first text content unique characters, and second text content unique characters;

[0009] Based on the text paragraph information in the first text content, the text matching result is divided into at least one subtext sequence, and at least one to-be-verified paragraph group corresponding to the at least one subtext sequence is determined;

[0010] determine a target similar paragraph group based on the at least one to-be-verified paragraph group and a similarity judgment model, and determine a total number of characters corresponding to the target similar paragraph group;

[0011] determine a difference rate between the two to-be-compared files based on the number of common characters, the number of characters unique to the first text content, the number of characters unique to the second text content, the total number of characters corresponding to the target similar paragraph group, and a preset difference rate function.

[0012] In a second aspect, the present application provides a text difference rate determination device, which comprises:

[0013] a text content determination module configured to separately parse and process two to-be-compared files to obtain first text content and second text content, wherein the two to-be-compared files correspond to the same substantive content but have different texts, and the text content comprises text paragraph information;

[0014] a matching result determination module configured to process the first text content and the second text content based on a file difference analysis algorithm to determine a text matching result, wherein the text matching result is a long sequence comprising common characters, characters unique to the first text content, and characters unique to the second text content;

[0015] a paragraph group determination module configured to divide the text matching result into at least one subtext sequence based on the text paragraph information in the first text content, and determine at least one to-be-verified paragraph group corresponding to the at least one subtext sequence;

[0016] a similar paragraph determination module configured to determine a target similar paragraph group based on the at least one to-be-verified paragraph group and a similarity judgment model, and determine a total number of characters corresponding to the target similar paragraph group;

[0017] a difference rate determination module configured to determine a difference rate between the two to-be-compared files based on the number of common characters, the number of characters unique to the first text content, the number of characters unique to the second text content, the total number of characters corresponding to the target similar paragraph group, and a preset difference rate function.

[0018] In a third aspect, the present application provides a data processing electronic device, which comprises:

[0019] at least one processor; and

[0020] a memory in communication connection with the at least one processor; wherein

[0021] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the text difference rate determination method of any one of the embodiments of the present application.

[0022] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions for causing a processor to implement the text difference rate determination method of any of the embodiments of the present application when executed.

[0023] In a fifth aspect, the present application provides a computer program product, which comprises a computer program for implementing the text difference rate determination method of any of the embodiments of the present application when executed by a processor.

[0024] The technical solution provided by the embodiments of the present application comprises the following steps: two files to be compared are parsed respectively to obtain first text content and second text content, wherein the substantial content corresponding to the two files to be compared is the same, and the characters are different, and the text content comprises text paragraph information; then the first text content and the second text content are processed based on a file difference analysis algorithm to determine a text matching result, wherein the text matching result is a long sequence comprising common characters, characters unique to the first text content, and characters unique to the second text content; further, the text matching result is divided into at least one subtext sequence based on the text paragraph information in the first text content, and at least one to-be-verified paragraph group corresponding to the at least one subtext sequence is determined, then a target similar paragraph group is determined based on the at least one to-be-verified paragraph group and a similarity judgment model, and the total number of characters corresponding to the target similar paragraph group is determined, so that the difference rate between the two files to be compared is determined based on the number of common characters, the number of characters unique to the first text content, the number of characters unique to the second text content, the total number of characters corresponding to the target similar paragraph group, and a difference rate function set in advance. The technical solution provided by the present application solves the problems of large workload, low efficiency, and high error rate in the file review process, improves the accuracy of determining the file difference rate, improves the efficiency of determining the file difference, and reduces the error frequency.

[0025] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0027] Figure 1 A flowchart of a text difference rate determination method provided for the first embodiment of the present application;

[0028] Figure 2 A flow chart of a text difference rate determination method provided for Embodiment Two of the present application is shown in Figure 1.

[0029] Figure 3 A flow chart of a text difference rate determination method provided for Embodiment Three of the present application is shown in Figure 2.

[0030] Figure 4 A structural schematic diagram of a text difference rate determination apparatus provided for Embodiment Four of the present application is shown in Figure 3.

[0031] Figure 5 A structural schematic diagram of an electronic device provided for Embodiment Five of the present application is shown in Figure 4. DETAILED DESCRIPTION

[0032] In order to make the technical personnel in the art better understand the present application scheme, the following will be combined with the drawings in the embodiments of the present application, the technical solutions in the embodiments of the present application are described clearly and completely, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0033] It should be noted that the terms "first predetermined condition", "second predetermined condition" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] Before introducing the technical solution, the application scenario can be exemplarily described. Nowadays, many enterprises need to process a large number of files, such as contracts, instructions, and bid documents. The files have high similarity in different versions, and only a small part of the files are different. If the revised version and the signed version are inconsistent, it may cause disputes in cooperation and even incalculable losses for the enterprise. Therefore, it is necessary to determine the difference between different versions of files. At present, the audit and verification of different versions of files mainly rely on workers to flip through each page, find and compare, and manually mark the differences. The above-mentioned method has the problems of large workload, low efficiency, and easy to make mistakes. In addition, there are some file comparison tools that can extract the text of the standard format file, and then use the public sequence-based method or the semantic recognition model to compare and determine the difference between the files. The above-mentioned automatic determination of the difference between the files usually only considers the local difference of the text field or only determines the global difference between the files based on the semantic recognition model, and cannot accurately estimate the difference between different versions of files. Therefore, the technical solution provided by the embodiment of the present application can consider the local difference of the field and the global difference of the file, and effectively improve the accuracy of determining the text difference rate.

[0035] Embodiment one

[0036] Figure 1 A flowchart of a text difference rate determination method provided by the first embodiment of the present application. The present embodiment can be applied to the case of evaluating the difference between files. The method can be performed by a text difference rate determination device, which can be realized in the form of hardware and / or software. The device can be configured on a computer device, which can be a notebook, a desktop computer, and a smart tablet, etc. As shown in the figure, the method comprises the following steps. Figure 1

[0037] S110, respectively parse and process two files to be compared to obtain first text content and second text content.

[0038] The two files to be compared include a first file and a second file. The two files to be compared have the same substantial content, and the text is different. Exemplarily, for a contract, the initial draft of the contract is the first file, and then the content of the first file is changed, increased, or reduced according to the actual situation. At this time, the determined file is the second file.

[0039] ​The first text content is the literal content, paragraph information, sequence information, and version revision information corresponding to the first file. The second text content is the literal content, paragraph information, sequence information, and version revision information corresponding to the second file. For example, the first file has a total of 3 natural paragraphs, and each paragraph has 25 characters. The first text content includes a character sequence composed of 75 characters, 3 paragraph marks, the sequence information before and after each character, and the annotated character.

[0040] On the basis of the above embodiment, the first text content and the second text content are determined, specifically including: determining the file type to which the two files to be compared belong; and performing parsing processing on each file to be compared and the corresponding file type to determine the text content corresponding to each file to be compared.

[0041] In this embodiment, the type of the file to be compared can include multiple types, for example, the file to be compared includes but is not limited to a Word document, a PDF document, a scanned document picture, and the like. Based on this, the file types of the two files to be compared are first determined, and then a parser corresponding to the file type is used to parse the file, thereby obtaining the first text content and the second text content.

[0042] For example, the two files to be compared include a first file and a second file, the first file is a Word document, a first parser corresponding to the Word document can be used to parse the first file to obtain the first text content, and the second file is a PDF document, a second parser corresponding to the PDF document can be used to parse the second file to obtain the second text content.

[0043] In S120, the first text content and the second text content are processed based on a file difference analysis algorithm to determine a text matching result.

[0044] The file difference analysis algorithm is a pre-defined operation method. The file difference analysis algorithm can obtain the text matching result by performing operation processing on the first text content and the second text content. The text matching result is a long sequence including common characters, first text content unique characters, and second text content unique characters.

[0045] Specifically, on the basis of determining the first text content and the second text content, the literal content in the first text content is taken as a first sequence, and the literal content in the second text content is taken as a second sequence. The first sequence and the second sequence are input into the file difference analysis algorithm, and the text matching result can be determined through operation.

[0046] For example, if the text content in the first text content is dabbcd, the first sequence is S1=(d, a, b, b, c, d); if the text content in the first text content is ddbbca, the second sequence is S2=(d, d, b, b, c, a). The text matching result obtained by operating S1=(d, a, b, b, c, d) and S2=(d, d, b, b, c, a) through the file difference analysis algorithm is:

[0047] S=(‘=, d’, ‘-, a’, ‘+, d’, ‘=, b’, ‘=, b’, ‘=, c’, ‘-, d’, ‘+, a’)

[0048] In the formula, ‘=’ represents the common character of the two text contents, ‘-’ represents the character unique to the first text content, and ‘+’ represents the character unique to the second text content.

[0049] S130, based on the text paragraph information in the first text content, the text matching result is divided into at least one subtext sequence, and at least one to-be-verified paragraph group corresponding to the at least one subtext sequence is determined.

[0050] In the formula, the long sequence corresponding to the text matching result is divided into multiple sequences, and each small sequence obtained is a subtext sequence. The to-be-verified paragraph group is two paragraphs reconstructed according to the characters in the subtext sequence.

[0051] Specifically, the first text content includes a paragraph mark corresponding to the text content. The characters corresponding to the paragraph marks can be used as boundaries to divide the text matching result into multiple subtext sequences. For example, the first text content includes three paragraphs and two paragraph marks. The first paragraph mark corresponds to the last character of the first paragraph, and the second paragraph mark corresponds to the last character of the second paragraph. Based on this, as long as the characters corresponding to the paragraph marks are found in the long sequence corresponding to the text matching result, a long sequence can be divided into three sub-sequences.

[0052] On the basis of the above embodiment, determining at least one to-be-verified paragraph group corresponding to the at least one subtext sequence includes: determining a first text paragraph based on the common characters in the subtext sequence and the characters unique to the first text content in the subtext sequence; determining a second text paragraph based on the common characters in the subtext sequence and the characters unique to the second text content in the subtext sequence; and taking the first text paragraph and the second text paragraph as the to-be-verified paragraph group corresponding to the at least one subtext sequence.

[0053] In this embodiment, a subsequence is used as an example. The subsequence contains common characters, characters unique to the first text content, and characters unique to the second text content. By reconstructing the common characters and the characters unique to the first text content in the subsequence in sequence, the first text segment can be determined. By reconstructing the common characters and the characters unique to the second text content in the subsequence in sequence, the second text segment can be determined. The first and second text segments are the segments to be verified corresponding to the subsequence. In practical applications, the first text content includes several segments, and thus the number of segments to be verified is determined.

[0054] S140. Based on at least one group of paragraphs to be verified and a similarity judgment model, determine the target similar paragraph group and determine the total number of characters corresponding to the target similar paragraph group.

[0055] The similarity judgment model is a pre-trained similarity value determination model. For example, the similarity judgment model can be a semantic model or a large-scale prediction model. The target similar paragraph group is the group of paragraphs to be verified whose similarity value is greater than a preset threshold.

[0056] Specifically, the first and second text paragraphs corresponding to the paragraph group to be verified are taken as input and fed into the similarity judgment model. First, the first and second text paragraphs are converted into a numerical vector of a specific length x = (x1, x2, ..., x...). k ) and y = (y1, y2, ..., y k Then, the similarity between the two numerical vectors is calculated using the cosine distance function. Thus, the similarity judgment model can output the similarity values ​​of the first text paragraph and the second text paragraph. In practical applications, the similarity value corresponding to each group of paragraphs to be verified can be calculated separately. The group of paragraphs to be verified with a similarity value greater than a preset threshold is taken as the target similar paragraph group. Then, the sub-text sequence corresponding to the target similar paragraph group is determined, and the total number of characters contained in the sub-text sequence is determined.

[0057] S150. Based on the number of common characters, the number of unique characters in the first text content, the number of unique characters in the second text content, the total number of characters corresponding to the target similar paragraph group, and the pre-set difference rate function, determine the difference rate between the two files to be compared.

[0058] In this embodiment, the pre-set difference rate function is:

[0059]

[0060] Where α is an optional parameter, C L C is the number of unique characters in the first text content. RC is the number of characters unique to the second text content A C is the number of common characters S C is the total number of characters corresponding to the target similar paragraph group.

[0061] In the embodiment, the first term of the difference rate function can represent the local difference between the two files to be compared. The second term of the difference rate function indicates that if the similarity value of the target similar paragraph group is greater than the threshold value, it can be indicated that the content of the entire paragraph is not different, and the individual text difference is not considered, which can represent the global difference of the file.

[0062] In the embodiment, the text matching result is a long sequence including common characters, first text content unique characters and second text content unique characters. Based on this, the number of common characters, the number of first text content unique characters and the number of second text content unique characters can be determined based on the text matching result. The number of common characters, the number of first text content unique characters, the number of second text content unique characters and the total number of characters corresponding to the target similar paragraph group are respectively brought into the difference rate function, so as to determine the difference rate between the two files to be compared.

[0063] The technical scheme provided by the embodiment of the application is that two files to be compared are respectively analyzed and processed to obtain first text content and second text content, wherein the two files to be compared correspond to the same substantial content, and the characters are different. The text content includes text paragraph information. Then, the first text content and the second text content are processed based on a file difference analysis algorithm to determine a text matching result, wherein the text matching result is a long sequence including common characters, first text content unique characters and second text content unique characters. Further, based on the text paragraph information in the first text content, the text matching result is divided into at least one subtext sequence, and at least one paragraph group to be verified corresponding to the at least one subtext sequence is determined. Then, based on the at least one paragraph group to be verified and a similarity judgment model, a target similar paragraph group is determined, and a total number of characters corresponding to the target similar paragraph group is determined. Thus, based on the number of common characters, the number of first text content unique characters, the number of second text content unique characters, the total number of characters corresponding to the target similar paragraph group and a pre-set difference rate function, the difference rate between the two files to be compared is determined. The technical scheme provided by the application solves the problems of large workload, low efficiency and easy error in the file review process, improves the accuracy of determining the file difference rate, improves the efficiency of determining the file difference, and reduces the error frequency.

[0064] On the basis of the above embodiment, further comprising: determining the first key field and the second key field corresponding to the preset key word based on at least one preset key word, the first text content and the second text content; verifying the first key field and the second key field, determining the key difference field, and feeding back.

[0065] In the embodiment, the key words can be defined in advance, and the adjacent positions of the key words are often some important information of the file. For example, the preset key words can be: amount, money, person in charge, legal person, etc. Based on the first text content and the second text content, the first key field and the second key field corresponding to the preset key words are searched by searching, and then the first key field and the second key field are respectively verified to determine the key difference field. If the key difference field exists, it can be fed back to the terminal device corresponding to the staff in time. In this way, the verification of the key information can ensure the accuracy of the key information.

[0066] For example, the preset key word is the person in charge, and then the "person in charge" is taken as the search word to search in the first text content and the second text content respectively, and the text content associated with the preset key word is determined. The result obtained by searching the first text content is: "person in charge Wang Gong", and the result obtained by searching the second text content is: "person in charge Li Gong". The first key field is "person in charge Wang Gong", the second key field is "person in charge Li Gong", and the key difference field is "Wang" and "Li" which can be determined by further verification. The difference result is fed back.

[0067] Embodiment two

[0068] Figure 2 A flowchart of a text difference rate determination method provided for the second embodiment of the application is shown in the figure. The second embodiment of the application is based on the above-mentioned embodiment, and the step S130 of the second embodiment of the application is further refined. The second embodiment of the application can be combined with one or more optional schemes in the above-mentioned embodiments. As shown in the figure, the method comprises: Figure 2

[0069] S210, respectively parsing and processing two files to be compared to obtain first text content and second text content.

[0070] S220, dividing the first text content and the second text content into at least two sub-text contents based on the longest common character.

[0071] The longest common character is a character segment composed of the character with the largest number of common characters corresponding to the first text content and the second text content.

[0072] ​For example, the first text content is adabbcd and the second text content is cdbbca, the longest common character is bbc, which divides the first text content adabbcd into ada and d, and divides the second text content cdbbca into cd and a, and the ada and d are the subtext content corresponding to the first text content, and the cd and a are the subtext content corresponding to the second text content.

[0073] S230, dividing the subtext content into at least two subtext contents again based on the sub-longest common character corresponding to the two subtext contents.

[0074] For example, the sub-longest common character of the subtext content ada and the subtext content cd is d, which divides the subtext content ada into a and a, and divides the subtext content cd into c and a space, wherein the space represents the null character.

[0075] S240, repeating the step of dividing the first text content and the second text content based on the longest common character in the subtext content until there is no common character in the first text content and the second text content, and obtaining all common characters.

[0076] For example, the finally determined common characters are d, b, b, and c.

[0077] S250, determining the first text content unique character based on the first text content and the common character, and determining the second text content unique character based on the second text content and the common character.

[0078] In this embodiment, the common characters d, b, b, and c are removed from the text content in the first text content, and the finally obtained characters are the first text content unique characters a, a, and d; the common characters d, b, b, and c are removed from the text content in the second text content, and the finally obtained characters are the second text content unique characters c and a. It should be particularly noted that the first text content and the second text content obtained in the process of separately analyzing and processing the two files to be compared include the order information before and after each character, so although the first text content unique character and the second text content unique character both have a, the position information corresponding to the a is very different.

[0079] S260, sequentially splicing the common character, the first text content unique character, and the second text content unique character to determine the text matching result.

[0080] In the embodiment, based on the determination of the common character, the first text content unique character and the second text content unique character, the common character, the first text content unique character and the second text content unique character are sequentially spliced according to the sequence information corresponding to each character in the original file, and the common character, the first text content unique character and the second text content unique character are respectively defined with different marks for distinguishing, so as to obtain the text matching result.

[0081] Based on the above example, the text matching result can be expressed as:

[0082] S = ('-,a', '+,c" =,d', '-', 'a', '=', 'b', '=', 'b', '=', 'c', '-', 'd', '+,a')

[0083] Wherein,'='represents the common character of two text contents,'-'represents the first text content unique character,'+'represents the second text content unique character.'= ','-'and'+'are different marks for distinguishing the common character, the first text content unique character and the second text content unique character.

[0084] S270, based on the text paragraph information in the first text content, the text matching result is divided into at least one subtext sequence, and at least one to-be-verified paragraph group corresponding to the at least one subtext sequence is determined.

[0085] S280, based on the at least one to-be-verified paragraph group and the similarity judgment model, a target similar paragraph group is determined, and a total number of characters corresponding to the target similar paragraph group is determined.

[0086] S290, based on the number of common characters, the number of first text content unique characters, the number of second text content unique characters, the total number of characters corresponding to the target similar paragraph group and the pre-set difference rate function, the difference rate between the two files to be compared is determined.

[0087] The technical scheme provided by the embodiment of the present application, when determining the text matching result, divides the first text content and the second text content into at least two subtext contents based on the longest common character, then divides the subtext contents into at least two subtext contents again based on the sublongest common character corresponding to the two subtext contents, repeatedly performs the step of dividing the first text content and the second text content based on the longest common character in the subtext content until there is no common character between the first text content and the second text content, obtains all common characters, further, determines the character unique to the first text content based on the first text content and the common character, and determines the character unique to the second text content based on the second text content and the common character, thereby sequentially splicing the common character, the character unique to the first text content and the character unique to the second text content to determine the text matching result. The common character of the first text content and the second text content is determined according to the longest common character in turn, and based on the determined common character, the character unique to the first text content and the character unique to the second text content can be quickly and conveniently determined. This way can quickly and efficiently determine the text matching result, thereby providing support for improving the efficiency of determining file differences.

[0088] Embodiment three

[0089] Figure 3 The flowchart of the text difference rate determination method provided by the third embodiment of the present application, the third embodiment of the present application is further refined on the basis of the above-mentioned embodiments, and can be combined with each optional scheme in one or more of the above-mentioned embodiments. As shown in the figure, the method comprises: Figure 3

[0090] S310, respectively parsing and processing two files to be compared to obtain first text content and second text content.

[0091] S320, processing the first text content and the second text content based on a file difference analysis algorithm to determine a text matching result.

[0092] S330, dividing the text matching result into at least one subtext sequence based on text paragraph information in the first text content, and determining at least one to-be-verified paragraph group corresponding to the at least one subtext sequence.

[0093] S340, inputting the at least one to-be-verified paragraph group into a pre-determined similarity judgment model to determine a similarity value corresponding to the at least one to-be-verified paragraph group.

[0094] In this embodiment, the determined to-be-verified paragraph group is input into the similarity judgment model, and the similarity judgment model outputs a similarity value corresponding to the to-be-verified paragraph group.

[0095] ​For example, the to-be-verified paragraph groups include a to-be-verified paragraph group 1, a to-be-verified paragraph group 2, and a to-be-verified paragraph group 3. The text content corresponding to the to-be-verified paragraph group 1, the to-be-verified paragraph group 2, and the to-be-verified paragraph group 3 is input into the similarity judgment model respectively. The similarity judgment model outputs a similarity value of the to-be-verified paragraph group 1 as 70%, a similarity value of the to-be-verified paragraph group 2 as 96%, and a similarity value of the to-be-verified paragraph group 3 as 95%.

[0096] In S350, if the similarity value is greater than the preset threshold value, at least one to-be-verified paragraph group is taken as a target similar paragraph group.

[0097] In the above example, the preset threshold value is a fixed value set in advance, and the preset threshold value can be adjusted adaptively in the application process. If the preset threshold value is 96%, the to-be-verified paragraph group 2 and the to-be-verified paragraph group 3 are target similar paragraph groups.

[0098] In S360, a subtext sequence corresponding to the target similar paragraph group is determined.

[0099] In the embodiment, on the basis of determining the target similar paragraph group, a subtext sequence corresponding to the target similar paragraph group is further determined.

[0100] In S370, the total number of characters contained in the subtext sequence is taken as the total number of characters corresponding to the target similar paragraph group.

[0101] In the above example, the to-be-verified paragraph group 2 and the to-be-verified paragraph group 3 are target similar paragraph groups. The total number of characters contained in the subtext sequence corresponding to the to-be-verified paragraph group 2 and the total number of characters contained in the subtext sequence corresponding to the to-be-verified paragraph group 3 are summed up, and the sum is the total number of characters corresponding to the target similar paragraph group.

[0102] In S380, based on the public character number, the first text content unique character number, the second text content unique character number, the total number of characters corresponding to the target similar paragraph group, and a preset difference rate function, a difference rate between the two to-be-compared files is determined.

[0103] The technical scheme provided by the embodiment of the application, when determining the total number of characters corresponding to the target similar paragraph group, at least one to-be-verified paragraph group is input into a pre-determined similarity judgment model, a similarity value corresponding to the at least one to-be-verified paragraph group is determined, if the similarity value is greater than a preset threshold value, at least one to-be-verified paragraph group is taken as a target similar paragraph group, a subtext sequence corresponding to the target similar paragraph group is further determined, and the total number of characters contained in the subtext sequence is taken as the total number of characters corresponding to the target similar paragraph group. The method for efficiently and accurately determining the total number of characters corresponding to the target similar paragraph group is provided, and the accuracy of determining the file difference rate is further improved, and the efficiency of determining the file difference is improved.

[0104] Embodiment Four

[0105] Figure 4 A structural schematic diagram of a text difference rate determination apparatus provided for Embodiment Four of the present application is provided, which can execute the text difference rate determination method provided by the embodiments of the present application. The apparatus comprises a text content determination module 410, a matching result determination module 420, a paragraph group determination module 430, a similar paragraph determination module 440, and a difference rate determination module 450.

[0106] The text content determination module 410 is configured to separately parse and process two files to be compared to obtain first text content and second text content; wherein the two files to be compared have the same substantial content but different characters, and the text content comprises text paragraph information.

[0107] The matching result determination module 420 is configured to process the first text content and the second text content based on a file difference analysis algorithm to determine a text matching result; wherein the text matching result is a long sequence comprising common characters, first text content unique characters, and second text content unique characters.

[0108] The paragraph group determination module 430 is configured to divide the text matching result into at least one subtext sequence based on the text paragraph information in the first text content, and determine at least one to-be-verified paragraph group corresponding to the at least one subtext sequence.

[0109] The similar paragraph determination module 440 is configured to determine a target similar paragraph group based on the at least one to-be-verified paragraph group and a similarity judgment model, and determine a total number of characters corresponding to the target similar paragraph group.

[0110] The difference rate determination module 450 is configured to determine a difference rate between the two files to be compared based on the number of common characters, the number of first text content unique characters, the number of second text content unique characters, the total number of characters corresponding to the target similar paragraph group, and a pre-set difference rate function.

[0111] On the basis of the above technical solutions, the text content determination module 410 comprises:

[0112] A file type determination unit is configured to determine the file types to which the two files to be compared belong.

[0113] A text content determination unit is configured to parse and process each of the files to be compared and the corresponding file types to determine the text content corresponding to each of the files to be compared.

[0114] On the basis of the above technical solutions, the matching result determination module 420 comprises:

[0115] The subtext determining unit is configured to divide the first text content and the second text content into at least two subtext contents based on the longest common character.

[0116] The subtext dividing unit is configured to divide the subtext contents into at least two subtext contents again based on the sub-longest common character corresponding to the two subtext contents.

[0117] The common character determining unit is configured to repeatedly perform the step of dividing the first text content and the second text content based on the longest common character in the subtext content until there is no common character in the first text content and the second text content, so as to obtain all common characters.

[0118] The unique content determining unit is configured to determine the first text content unique character based on the first text content and the common character, and determine the second text content unique character based on the second text content and the common character.

[0119] The matching result determining unit is configured to sequentially splice the common character, the first text content unique character and the second text content unique character to determine the text matching result.

[0120] On the basis of the above technical solutions, the paragraph group determining module 430 comprises:

[0121] The scene distribution diagram obtaining unit is configured to determine a first text paragraph based on the common character in the subtext sequence and the first text content unique character in the subtext sequence.

[0122] Determine a second text paragraph based on the common character in the subtext sequence and the second text content unique character in the subtext sequence.

[0123] The first text paragraph and the second text paragraph are taken as the to-be-verified paragraph group corresponding to the at least one subtext sequence.

[0124] On the basis of the above technical solutions, the similar paragraph determining module 440 comprises:

[0125] The similarity value determining unit is configured to input the at least one to-be-verified paragraph group into a pre-determined similarity judgment model to determine the similarity value corresponding to the at least one to-be-verified paragraph group.

[0126] The target paragraph group determining unit is configured to take the at least one to-be-verified paragraph group as the target similar paragraph group if the similarity value is greater than a pre-set threshold value.

[0127] On the basis of the above technical solutions, the similar paragraph determining module 440 further comprises:

[0128] a subsequence determining unit configured to determine a subtext sequence corresponding to the target similar paragraph group;

[0129] a similar character determining unit configured to take the total number of characters contained in the subtext sequence as the total number of characters corresponding to the target similar paragraph group.

[0130] On the basis of the above technical solutions, the text difference rate determining device further comprises:

[0131] a key field determining module configured to determine a first key field and a second key field corresponding to the preset key word based on the at least one preset key word, the first text content and the second text content;

[0132] a difference field feedback module configured to examine the first key field and the second key field, determine a key difference field, and feed back.

[0133] The technical solution provided by the embodiment of the present application comprises the following steps: two files to be compared are respectively parsed to obtain a first text content and a second text content, wherein the two files to be compared have the same substantial content but different texts, and the text content comprises text paragraph information; then the first text content and the second text content are processed based on a file difference analysis algorithm to determine a text matching result, wherein the text matching result is a long sequence comprising common characters, characters unique to the first text content, and characters unique to the second text content; further, the text matching result is divided into at least one subtext sequence based on the text paragraph information in the first text content, and at least one paragraph group to be verified corresponding to the at least one subtext sequence is determined, then a target similar paragraph group is determined based on the at least one paragraph group to be verified and a similarity judgment model, and the total number of characters corresponding to the target similar paragraph group is determined, so that a difference rate between the two files to be compared is determined based on the number of common characters, the number of characters unique to the first text content, the number of characters unique to the second text content, the total number of characters corresponding to the target similar paragraph group, and a difference rate function set in advance. The technical solution provided by the present application solves the problems of large workload, low efficiency and high error rate in the file review process, improves the accuracy of determining the file difference rate, enhances the efficiency of determining the file difference, and reduces the error frequency.

[0134] The text difference rate determining device provided by the embodiments of the present disclosure can perform the text difference rate determining method provided by any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of performing the method.

[0135] It is to be noted that the units and modules included in the above apparatus are only divided according to the function logic, and are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for the convenience of mutual differentiation, and do not serve to limit the protection scope of the embodiments of the present disclosure.

[0136] Embodiment Five

[0137] Figure 5 A structural schematic diagram of an electronic device is provided for Embodiment Five of the present disclosure. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0138] As shown in Figure 5 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0139] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0140] The processor 11 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the text difference rate determination method.

[0141] In some embodiments, the text difference rate determination method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the text difference rate determination method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the text difference rate determination method by any other suitable means, such as by means of firmware.

[0142] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0143] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable text difference rate determination apparatuses to produce a machine, such that the computer program, when executed, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0144] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0146] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0147] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service. It should be understood that the various forms of flow shown above can be reordered, added, or deleted steps. For example, each step described in the present application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein. The above specific embodiments do not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method of determining a text difference rate, characterized by, The method comprises the following steps: Respectively parsing two files to be compared to obtain first text content and second text content; wherein the two files to be compared have the same substantial content but different characters, and the text content comprises text paragraph information; Processing the first text content and the second text content based on a file difference analysis algorithm to determine a text matching result; wherein the text matching result is a long sequence comprising common characters, first text content unique characters and second text content unique characters; Dividing the text matching result into at least one subtext sequence based on the text paragraph information in the first text content; Determining a first text paragraph based on the common characters in the subtext sequence and the first text content unique characters in the subtext sequence; Determining a second text paragraph based on the common characters in the subtext sequence and the second text content unique characters in the subtext sequence; Taking the first text paragraph and the second text paragraph as a to-be-verified paragraph group corresponding to the at least one subtext sequence; Determining a target similar paragraph group based on the at least one to-be-verified paragraph group and a similarity judgment model, and determining a total number of characters corresponding to the target similar paragraph group; wherein the target similar paragraph group is a to-be-verified paragraph group with a similarity value greater than a preset threshold value; Determining a difference rate between the two files to be compared based on the number of common characters, the number of first text content unique characters, the number of second text content unique characters, the total number of characters corresponding to the target similar paragraph group and a preset difference rate function; wherein the preset difference rate function is: ； wherein, is an optional parameter, is the number of unique characters of the first text content, is the number of unique characters of the second text content, is the number of common characters, is the total number of characters corresponding to the target similar passage group.

2. The method of claim 1, wherein, The method comprises the following steps: Determining the file types to which the two files to be compared belong; Analyzing each file to be compared and the corresponding file type to determine the text content corresponding to each file to be compared.

3. The method of claim 1, wherein, The method comprises the following steps: Dividing the first text content and the second text content into at least two subtext contents based on the longest common characters; Dividing the subtext contents into at least two subtext contents again based on the sub-longest common characters corresponding to the two subtext contents; Repeating the step of dividing the first text content and the second text content based on the longest common characters in the subtext contents until there are no common characters in the first text content and the second text content, thereby obtaining all common characters; Determining the first text content unique characters based on the first text content and the common characters; Determining the second text content unique characters based on the second text content and the common characters; Sequentially splicing the common characters, the first text content unique characters and the second text content unique characters to determine the text matching result. The method comprises the following steps:

4. The method of claim 1, wherein, ​ input the at least one to-be-verified paragraph group into a predetermined similarity judgment model to determine a similarity value corresponding to the at least one to-be-verified paragraph group; if the similarity value is greater than a preset threshold, the at least one to-be-verified paragraph group is taken as a target similar paragraph group.

5. The method of claim 1, wherein, The determination of the total number of characters corresponding to the target similar paragraph group comprises: determining a subtext sequence corresponding to the target similar paragraph group; taking the total number of characters contained in the subtext sequence as the total number of characters corresponding to the target similar paragraph group.

6. The method of claim 1, wherein, Further comprising: determining a first key field and a second key field corresponding to the preset key vocabulary based on at least one preset key vocabulary, the first text content and the second text content; verifying the first key field and the second key field to determine a key difference field and feedback.

7. A text difference rate determination apparatus characterized by comprising: Comprise: a text content determination module for separately parsing and processing two to-be-compared files to obtain first text content and second text content; wherein the two to-be-compared files correspond to the same substantive content and have some differences in the text; the text content includes text paragraph information; a matching result determination module for processing the first text content and the second text content based on a file difference analysis algorithm to determine a text matching result; wherein the text matching result is a long sequence including common characters, first text content unique characters and second text content unique characters; a paragraph group determination module for dividing the text matching result into at least one subtext sequence based on the text paragraph information in the first text content, and determining a first text paragraph based on the common characters in the subtext sequence and the first text content unique characters in the subtext sequence; determining a second text paragraph based on the common characters in the subtext sequence and the second text content unique characters in the subtext sequence; taking the first text paragraph and the second text paragraph as a to-be-verified paragraph group corresponding to the at least one subtext sequence; a similar paragraph determination module for determining a target similar paragraph group based on the at least one to-be-verified paragraph group and a similarity judgment model, and determining a total number of characters corresponding to the target similar paragraph group; wherein the target similar paragraph group is a to-be-verified paragraph group with a similarity value greater than a preset threshold; a difference rate determination module for determining a difference rate between the two to-be-compared files based on the number of common characters, the number of first text content unique characters, the number of second text content unique characters, the total number of characters corresponding to the target similar paragraph group and a preset difference rate function; wherein the preset difference rate function is: ； wherein, is an optional parameter, is the number of unique characters of the first text content, is the number of unique characters of the second text content, is the number of common characters, is the total number of characters corresponding to the target similar passage group.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the text difference rate determination method of any one of claims 1-6.

Images