Document comparison method, device, equipment and medium

Abstract

The present disclosure provides a document comparison method, apparatus, device and medium, which relate to the field of artificial intelligence, specifically to the field of natural language processing and document conversion, and can be applied to scenarios such as OCR recognition and document comparison. The specific implementation scheme of the document comparison method is: segment two documents to be compared into paragraphs respectively to obtain a paragraph set of each document to be compared; determine the aligned paragraphs of the two documents to be compared based on the two paragraph sets of the two documents to be compared; segment the two documents to be compared based on the aligned paragraphs to obtain multiple paragraph segment pairs; each paragraph segment pair includes two paragraph segments belonging to the two documents to be compared respectively; the two paragraph segments are obtained based on the same aligned paragraph segmentation; and determine the difference paragraphs of the two documents to be compared based on the edit distance of the two paragraph segments in each paragraph segment pair in units of paragraphs.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to the field of natural language processing and document conversion, and can be applied to scenarios such as OCR recognition and document comparison. Background Art

[0002] With the advancement of computer and electronic technologies, electronic devices can now use document comparison technology to parse document text and perform differential analysis. This differential analysis typically reflects the differences between two documents in character units. Consequently, when reviewing the comparison results, reviewers must perform character-by-character verification, which can be labor-intensive. Summary of the Invention

[0003] The present disclosure aims to provide a document comparison method, apparatus, device and medium that are conducive to improving comparison results and reducing review workload.

[0004] According to one aspect of the present disclosure, a document comparison method is provided, comprising: segmenting two documents to be compared into paragraphs to obtain a paragraph set of each document to be compared; determining aligned paragraphs of the two documents to be compared based on the two paragraph sets of the two documents to be compared; segmenting the two documents to be compared based on the aligned paragraphs to obtain a plurality of paragraph segment pairs; each paragraph segment pair includes two paragraph segments belonging to the two documents to be compared, respectively; the two paragraph segments are obtained based on the same aligned paragraph segmentation; and determining the difference paragraphs of the two documents to be compared based on the edit distance in paragraph units between the two paragraph segments in each paragraph segment pair.

[0005] According to another aspect of the present disclosure, a document comparison device is provided, comprising: a paragraph segmentation module for performing paragraph segmentation on two documents to be compared, respectively, to obtain a paragraph set of each document to be compared; an aligned paragraph determination module for determining the aligned paragraphs of the two documents to be compared based on the two paragraph sets of the two documents to be compared; the aligned segmentation module for segmenting the two documents to be compared based on the aligned paragraphs to obtain a plurality of paragraph segment pairs; each paragraph segment pair includes two paragraph segments belonging to the two documents to be compared, respectively; the two paragraph segments are obtained based on the same aligned paragraph segmentation; and a different paragraph determination module for determining the different paragraphs of the two documents to be compared based on the edit distance in paragraph units between the two paragraph segments in each paragraph segment pair.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so as to enable the at least one processor to execute the document comparison method provided by the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the document comparison method provided by the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, wherein the computer program / instruction is stored on at least one of a readable storage medium and an electronic device, and when the computer program / instruction is executed by a processor, the document comparison method provided by the present disclosure is implemented.

[0009] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of an application scenario of the document comparison method and apparatus according to an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of a document comparison method according to an embodiment of the present disclosure;

[0013] Figure 3 2 is a schematic diagram of the principle of segmenting the document to be compared according to an embodiment of the present disclosure;

[0014] Figure 4 2 is a schematic diagram of the principle of determining the aligned paragraphs of two documents to be compared according to an embodiment of the present disclosure;

[0015] Figure 5 is a structural block diagram of a document comparison device according to an embodiment of the present disclosure; and

[0016] Figure 6 It is a schematic block diagram of an example electronic device for implementing the document comparison method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0018] Intelligent document comparison technology can solve the problems of manual document comparison, which are labor-intensive, time-consuming, and prone to errors. Intelligent document comparison technology can quickly compare differences between different versions of documents, allowing reviewers to quickly review the differences, improving document comparison efficiency.

[0019] Document comparison technology typically compares differences at a character-level. For example, if characters at corresponding locations in two documents are inconsistent, the two documents are considered to differ at that location. Accordingly, when reviewers review the content of the differences, they also need to review them at a character-level. However, when comparing contract terms for consistency, the differences of interest are typically at a paragraph-level. Comparing and reviewing at this character-level not only incurs excessive review costs, but also fails to meet actual needs.

[0020] In order to solve this problem, the present disclosure provides a document comparison method, apparatus, device and medium for comparing documents at paragraph granularity. Figure 1 The application scenarios of the method and device provided by the present disclosure are described.

[0021] Figure 1 Schematic diagram of an application scenario of the document comparison method and device according to an embodiment of the present disclosure.

[0022] like Figure 1 As shown, the application scenario 100 of this embodiment may include a terminal device 110, which may be various electronic devices with processing functions, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc.

[0023] The terminal device 110 can be used to segment the two documents 120 that need to be compared into paragraphs, and then perform a difference comparison on the two documents 120 at the paragraph level to obtain the difference content 130 of the two documents 120 .

[0024] Among them, when performing difference comparison at the paragraph granularity, the two documents 120 can be aligned first, and then the two documents 120 can be divided according to the alignment point. The paragraphs with differences can be determined based on the editing distance between the divided paragraph segments in units of paragraphs, and the paragraphs with differences can be used as the difference content 130.

[0025] In one embodiment, after determining the difference content 130 between the two documents 120 , the terminal device may further display the two documents 120 and highlight the difference content 130 to achieve the effect of showing the comparison result to the user.

[0026] In one embodiment, the application scenario 100 may further include a server 140, and the terminal device 110 may be communicatively connected to the server 140 via a network. The network may include wireless or wired communication links. For example, the terminal device 110 may send two documents 120 to be compared to the server 140 via the network. The server 140 then compares the two documents 120 on a paragraph-by-paragraph basis to obtain difference content 130. The server 140 may also send the determined difference content 130 to the terminal device 110, so that the terminal device can prominently display the difference content 130 while displaying the two documents 120.

[0027] In one embodiment, the timing of comparing the two documents 120 may be, for example, the timing of the user operating a comparison button displayed on the terminal device 110 . The two documents 120 may be documents determined in response to the user operation, which is not limited in this disclosure.

[0028] It is understandable that the server 140 can be, for example, a background management server that provides support for the operation of an application with a document comparison function installed in a terminal device, or it can be a cloud server or a blockchain server, etc., and this disclosure does not limit this.

[0029] It should be noted that the document comparison method provided by the present disclosure can be executed by the terminal device 110 or by the server 140. Accordingly, the document comparison apparatus provided by the present disclosure can be provided in the terminal device 110 or in the server 140.

[0030] It should be understood that Figure 1 The number and type of terminal devices 110 and servers 140 in the embodiment are merely illustrative. Any number and type of terminal devices 110 and servers 140 may be provided as required.

[0031] The following will be combined Figures 2 to 4 The document comparison method provided in the present disclosure is described in detail.

[0032] Figure 2 4 is a flowchart of a document comparison method according to an embodiment of the present disclosure.

[0033] like Figure 2 As shown, the document comparison method 200 of this embodiment may include operations S210 to S240.

[0034] In operation S210 , paragraph segmentation is performed on each of the two documents to be compared to obtain a paragraph set of each document to be compared.

[0035] According to an embodiment of the present disclosure, a text segmentation algorithm such as the TextTiling algorithm can be used to segment each document to be compared into paragraphs. The TextTiling algorithm can calculate the similarity between two adjacent sentence groups, and then determine the sub-topic segmentation points based on the calculated similarity, so as to segment the document to be compared into paragraphs based on the sub-topics, and divide the document to be compared into several paragraphs based on different sub-topics. It is understandable that the above-mentioned TextTiling algorithm is only used as an example to facilitate understanding of the present disclosure, and the present disclosure does not limit this.

[0036] According to an embodiment of the present disclosure, based on the experience that the beginning of a paragraph generally contains white space information and the end of a paragraph generally contains white space information, the document to be compared can be detected for white space information and white space areas. Subsequently, the document to be compared can be segmented into paragraphs based on the positions of the detected white space information and white space areas in the document to be compared.

[0037] By segmenting the document to be compared into paragraphs, a plurality of paragraphs included in the document to be compared can be obtained. In this embodiment, the plurality of paragraphs can be sorted from front to back according to their positions in the document to be compared to obtain a paragraph set consisting of the plurality of paragraphs. The paragraph set is the paragraph set of the document to be compared.

[0038] In operation S220 , aligned paragraphs of the two documents to be compared are determined according to the two paragraph sets of the two documents to be compared.

[0039] According to an embodiment of the present disclosure, the similarity between each paragraph in a paragraph set and each paragraph in another paragraph set can be calculated. Then, the paragraph in the other paragraph set that has the greatest similarity to each paragraph is determined. If the greatest similarity is greater than a predetermined similarity threshold, the paragraph with the greatest similarity is used as the paragraph that matches each paragraph, and is used as an aligned paragraph representing the same paragraph. In this way, aligned paragraphs of two documents to be compared can be obtained. The predetermined similarity threshold can be set according to actual needs. For example, the predetermined similarity threshold can be 0.8, etc., which is not limited by the present disclosure.

[0040] According to embodiments of the present disclosure, for example, a semantic feature extraction model can be used to extract semantic features of two paragraphs whose similarity needs to be calculated. Subsequently, the cosine similarity between the semantic features of the two paragraphs can be used as the similarity between the two paragraphs. It should be understood that the above principle of calculating the similarity between two paragraphs is merely an example to facilitate understanding of the present disclosure and is not limited thereto.

[0041] In operation S230 , the two documents to be compared are segmented according to the aligned paragraphs to obtain a plurality of paragraph segment pairs; each paragraph segment pair includes two paragraph segments belonging to the two documents to be compared respectively; and the two paragraph segments are obtained according to the same aligned paragraph segmentation.

[0042] According to an embodiment of the present disclosure, each document to be compared is segmented according to the aligned paragraphs, and for example, a paragraph fragment set can be obtained. This embodiment can align the paragraph fragments in the two paragraph fragment sets obtained by segmenting the two documents to be compared according to the position of each aligned paragraph in the two documents to be compared, and obtain a plurality of paragraph fragment pairs. Each paragraph fragment pair includes two paragraph fragments belonging to the two documents to be compared, that is, each paragraph fragment pair includes a paragraph fragment belonging to the paragraph fragment set obtained by segmenting one document to be compared, and a paragraph fragment belonging to the paragraph fragment set obtained by segmenting another document to be compared, and the two paragraph fragments in each paragraph fragment pair are obtained based on the same aligned paragraph segmentation.

[0043] In operation S240 , different paragraphs of the two documents to be compared are determined based on the edit distance in paragraph units between the two paragraph segments in each paragraph segment pair.

[0044] According to an embodiment of the present disclosure, after obtaining multiple paragraph segment pairs, the edit distance between the two paragraph segments in each paragraph segment pair can be calculated in parallel. Subsequently, the difference paragraphs between the two paragraph segments in each paragraph segment pair can be obtained based on the edit distance.

[0045] When calculating the edit distance between two paragraph fragments, for example, each paragraph fragment can be first mapped with a paragraph as the granularity. For example, each paragraph can be used as a token, and each paragraph fragment can be mapped to a token sequence. It is understandable that the same paragraph will be mapped to the same token. Subsequently, this embodiment can calculate the edit distance between the two token sequences obtained by mapping the two paragraph fragments in each paragraph fragment pair, thereby serving as the edit distance in paragraph units.

[0046] Among them, the edit distance refers to the minimum number of editing operations required to convert a string (in this embodiment, a token sequence obtained by mapping one of the two paragraph fragments) into another string (in this embodiment, a token sequence obtained by mapping the other of the two paragraph fragments). For example, an algorithm for calculating the Levenshtein distance can be used to calculate the edit distance between the two paragraph fragments, or an arbitrary edit distance calculation algorithm can be used to determine the edit distance between the two paragraph fragments. It is understandable that each difference paragraph can correspond to an editing operation representing the edit distance. Editing operations can include, for example, adding a token, deleting a token, and replacing a token.

[0047] For example, the Levenshtein.opcodes tool can be called to calculate the edit distance, which is used to find a series of editing operations to convert a string into another string. In this embodiment, the two paragraph fragments included in each paragraph fragment pair can be used as the input of the Levenshtein.opcodes tool, and the Levenshtein.opcodes tool outputs a series of five-tuple sequences. Each five-tuple in the five-tuple sequence can be expressed as [operator, s1, e1, s2, e2]. Among them, operator represents an operator, which can be one of the insertion operator insert, the deletion operator delete and the replacement operator replace. s1 represents converting a token sequence (for example, a first sequence) into another token sequence (for example, a second sequence), and the first sequence needs to change the starting position of the token under the operation represented by the operator operator, and the position of the changed token includes the starting position. e1 represents converting the first sequence into the second sequence, and the first sequence needs to change the ending position of the token under the operation represented by the operator operator, and the position of the changed token does not include the ending position. s2 indicates that the first sequence is converted into the second sequence, and the starting position of the token in the second sequence needs to be changed under the operation represented by the operator operator, and the position of the changed token includes the starting position. e2 indicates that the first sequence is converted into the second sequence, and the ending position of the token in the second sequence needs to be changed under the operation represented by the operator operator, and the position of the changed token does not include the ending position. It can be understood that the starting position and ending position mentioned in this embodiment can be represented by the position sorting of the token in the token sequence. For example, if the starting position is 2, it means that the starting position is the position of the token ranked 2 in the token sequence.

[0048] It is understood that in the field of natural language processing, a token represents a mapping representation of a single character or word obtained by segmenting a text at a character granularity, a word granularity, etc. In the embodiment of the present disclosure, a token represents a mapping representation of a single paragraph.

[0049] The disclosed embodiments enable paragraph-based document comparison by determining the differences between two documents. This eliminates the need to determine differences at the character level, especially in scenarios where the documents being compared are contract clauses and require attention to differences at the paragraph level. This improves both document comparison efficiency and effectiveness, reducing the workload for reviewers.

[0050] According to an embodiment of the present disclosure, the two documents to be compared may be, for example, documents with suffixes such as .doc, .wps, or .pdf. The present disclosure does not limit the formats of the two documents to be compared.

[0051] In one embodiment, the two documents to be compared can be converted into documents in a portable file format first, and the two documents in the portable file format obtained are used as two documents to be compared, and the paragraph segmentation and alignment of the documents are performed. Among them, the portable file format can be a file format with the suffix .pdf. In this way, it is conducive to the visual display of the determined difference paragraphs. This is because the document reader of the portable file format usually has multiple types of annotation tools and rendering tools, with the help of these annotation tools and rendering tools, different types of difference paragraphs can be displayed differently. Among them, the type of difference paragraph can include, for example, the insertion type, deletion type and replacement type described above, etc., which is not limited in this disclosure.

[0052] Figure 3 It is a schematic diagram of the principle of segmenting the document to be compared according to an embodiment of the present disclosure.

[0053] like Figure 3 As shown, in embodiment 300, when segmenting documents 311 and 312 to be compared into paragraphs, for example, the two documents to be compared can be first parsed to obtain text elements and position information of the text elements included in each document to be compared, and the text elements and position information of the text elements are used as the parsing results of each document to be compared. Subsequently, based on the parsing results of each document to be compared, each document to be compared is segmented into paragraphs to obtain a set of paragraphs included in each document to be compared.

[0054] For example, a document parsing tool may be used to parse the document to be compared 311 and the document to be compared 312 separately. The document parsing tool may include, for example, a tool built based on OCR technology, or a parsing operator. For example, tools built based on OCR technology may include PaddleOCR, TrOCR, etc. The parsing operator may be, for example, a Parser parser, or a PyPDF operator provided by the Python language, etc., which is not limited in this disclosure.

[0055] According to an embodiment of the present disclosure, by parsing the document to be compared 311, the text elements included in the document to be compared 311 and the position information of the text elements in the document to be compared 311 can be obtained. For example, the text elements may include characters, pictures, table lines, etc. The position information of the text elements in the document to be compared 311 can be represented by the position of the minimum rectangular box surrounding the text elements in the coordinate system constructed based on the document to be compared 311, for example, it can include the vertex coordinates of the upper left corner of the minimum rectangular box and the vertex coordinates of the lower right corner of the minimum rectangular box, or it can include the vertex coordinates of the upper left corner of the minimum rectangular box and the width and height of the minimum rectangular box.

[0056] For example, the position information of a text element can be expressed as [x0, y0, x1-x0, y1-y0]. Here, x0 is the starting value of the text element's coordinates in the width direction, which can be understood as the horizontal coordinate of the upper left corner of the minimum rectangular box. y0 is the starting value of the text element's coordinates in the height direction, which can be understood as the vertical coordinate of the upper left corner of the minimum rectangular box. x1 is the ending value of the text element's coordinates in the width direction, which can be understood as the horizontal coordinate of the lower right corner of the minimum rectangular box. y1 is the ending value of the text element's coordinates in the height direction, which can be understood as the vertical coordinate of the lower right corner of the minimum rectangular box.

[0057] In one embodiment, for example, format information of text elements included in the document to be compared 311 may also be parsed. When the text elements are characters, the format information may include, for example, the font type, font size, etc. When the text elements are images, the format information may include, for example, the image embedding type, etc. When the text elements are table lines, the format information may include, for example, the width of the table lines, etc.

[0058] This embodiment can use all information obtained by parsing the document to be compared 311 as the parsing result of the document to be compared 311. For example, this embodiment can arrange the information of the parsed text elements line by line according to the positions of the text elements in the document to be compared 311 to obtain a set of text element information as the parsing result 321.

[0059] Based on a principle similar to that of obtaining the parsing result 321 , by parsing the document to be compared 312 , a parsing result 322 included in the document to be compared 312 can be obtained.

[0060] After obtaining parsing result 321, can determine the text element that is positioned at paragraph starting position and the text element that is positioned at paragraph termination position according to the positional information of text element in parsing result 321.For example, if the horizontal coordinate of the text element that is positioned at the leftmost in a certain row is larger than the horizontal coordinate of the text element that is positioned at the leftmost in the previous row, then can determine that this certain row has indentation, and the text element that is positioned at the leftmost in this certain row is the starting element of a paragraph.Similarly, can determine the text element that is positioned at paragraph termination position.This embodiment can divide the text element between the text element at the paragraph termination position whose longitudinal coordinate and the longitudinal coordinate of this paragraph starting element are minimum into the text element of a paragraph, thereby realizes the division of paragraph.Like this, according to parsing result 321, treat and compare document 311 to be segmented, can obtain the paragraph set 331 that treat and compare document 311 comprises.Similarly, according to parsing result 322, treat and compare document 312 to be segmented, can obtain the paragraph set 332 that treat and compare document 312 comprises.

[0061] The disclosed embodiment parses the documents to be compared to obtain parsing results, which can provide sufficient information for paragraph segmentation of the documents to be compared. Combined with prior knowledge, the document can be segmented into paragraphs without calling a paragraph segmentation tool. This can improve paragraph segmentation accuracy and paragraph segmentation costs.

[0062] In one embodiment, the row where the title in the document to be compared is located can be determined based on the text element representing the serial number in the parsing result. Alternatively, the parsing result can also include, for example, format information of the text element. Based on this format information, the title in the document can be distinguished from the text paragraph. For example, the characters in the title are usually in bold format, or the font size of the characters in the title is usually larger than the font size of the characters in the text paragraph. In this way, the row where the characters in bold format or the larger font size are located can be used as the row where the title in the document to be compared is located. In this way, the title included in the document to be compared can be located. For example, the titles in the document can also be classified into multiple different levels of titles based on the text element representing the serial number or the font size of the characters in the title. For example, the higher the level of the title, the larger the font size of the characters. For example, the multiple different levels can include chapter level, section level, item level, etc., which are successively reduced in level, and this disclosure does not limit this.

[0063] In this embodiment, when it is determined that the first title included in the document to be compared includes a text paragraph in the adjacent position after the first title, the first title and the text paragraph in the adjacent position after the first title can be regarded as one paragraph. When it is determined that the second title included in the document to be compared does not include a text paragraph in the adjacent position after the second title, the second title and a third title of the same level as the second title included in the document to be compared can be regarded as one paragraph. In this way, this embodiment can group the detailed explanation paragraph of the title with the title into one paragraph, and group the paragraph with less content with other content into one paragraph, which is beneficial to improving the richness of the information expressed in each paragraph obtained by division and improving the efficiency of determining the difference paragraphs.

[0064] The subsequent adjacent position may have a larger ordinate than the ordinate of the row where the title is located and is adjacent to the row where the title is located. It is understood that the above-mentioned technical solution of treating a title and a text paragraph, or treating a title and a title of the same level as a single paragraph is merely an example to facilitate understanding of the present disclosure. The present disclosure may also redefine paragraphs in any manner according to actual needs.

[0065] After obtaining the paragraph set 331 and the paragraph set 332, this embodiment can determine the aligned paragraphs 340 of the document to be compared 311 and the document to be compared 312 according to the two paragraph sets, and segment the document to be compared 311 and the document to be compared 312 according to the aligned paragraphs 340.

[0066] For example, Figure 3 As shown, after obtaining the aligned paragraphs 340 of the two documents to be compared, this embodiment can segment the document to be compared 311 and the document to be compared 312 according to the aligned paragraphs 340. For example, the portion before the aligned paragraph 340 in the document to be compared 311 can be regarded as a paragraph segment, the portion after the aligned paragraph 340 can be regarded as a paragraph segment, and the aligned paragraphs can be regarded as a paragraph segment. In this way, segmenting the document to be compared 311 can obtain multiple paragraph segments, and these multiple paragraph segments can serve as the initial paragraph segment group 351 included in the document to be compared 311. Similarly, the document to be compared 312 can be segmented to obtain the initial paragraph segment group 352.

[0067] After obtaining initial paragraph fragment group 351 and initial paragraph fragment group 352, this embodiment may further, for example, delete the aligned paragraphs from the two initial paragraph fragment groups and use the paragraph fragment groups from which the aligned paragraphs are deleted as the paragraph fragment groups of the two documents to be compared, thereby obtaining paragraph fragment group 361 and paragraph fragment group 362. Subsequently, the two paragraph fragments located at the same position in paragraph fragment group 361 and paragraph fragment group 362 may be formed into a paragraph fragment pair, thereby obtaining multiple paragraph fragment pairs 370.

[0068] By removing the aligned paragraphs from the initial paragraph segment group formed from the segmented paragraph segments, the disclosed embodiment can reduce the number of resulting paragraph segment pairs and thus improve the efficiency of determining different paragraphs while ensuring accuracy. This is because aligned paragraphs are identical paragraphs in the two documents being compared and are not necessarily different paragraphs. Therefore, there is no need to determine whether there are different paragraphs in the aligned paragraphs based on the edit distance.

[0069] Figure 4 It is a schematic diagram of the principle of determining the aligned paragraphs of two documents to be compared according to an embodiment of the present disclosure.

[0070] In one embodiment, when determining aligned paragraphs, for example, the paragraphs in the paragraph set can be divided according to sliding windows of a predetermined size, and the aligned paragraphs can be determined based on the paragraphs included in each sliding window. This reduces the number of paragraph segment pairs determined, thereby further improving the efficiency of determining different paragraphs. The predetermined sliding window size can, for example, consist of n paragraphs, where n is a natural number greater than or equal to 2.

[0071] For example, Figure 4 As shown, in embodiment 400, it is possible to set two documents to be compared: after mapping, the paragraphs included in document 411 to be compared can obtain a token sequence [a, c, d, b, f, i, e, g, k]; after mapping, the paragraphs included in document 412 to be compared can obtain a token sequence [a, d, c, b, f, e, g, m]. A sliding window of a predetermined size is set with two paragraphs as one window. Then, the paragraphs in the paragraph set of document 411 to be compared are divided, and the resulting paragraph group set 421 consisting of multiple paragraph groups can be expressed as [ac, cd, db, bf, fi, ie, eg, gk]. The paragraphs in the paragraph set of document 412 to be compared are divided, and the resulting paragraph group set 422 consisting of multiple paragraph groups can be expressed as [ad, dc, cb, bf, fe, eg, gm].

[0072] Subsequently, this embodiment can determine the aligned paragraphs in units of paragraph groups based on the paragraph groups included in both the paragraph group set 421 and the paragraph group set 422. Specifically, for example, the paragraph groups included in both the paragraph group set 421 and the paragraph group set 422 can be represented as bf and eg, as aligned paragraphs 430. In this way, the document to be compared is segmented according to the aligned paragraphs 430, and the resulting paragraph segment groups can be represented as (acd, bf, i, eg, k), for example. The document to be compared is segmented according to the aligned paragraphs 430, and the resulting paragraph segment groups can be represented as (adc, bf, eg, m), for example.

[0073] It is understood that the principle of dividing paragraphs in the paragraph set described above can be similar to the principle of forming byte fragment sequences in the n-gram model in natural language processing. The principle of dividing paragraphs in the paragraph set described above is merely an example to facilitate understanding of the present disclosure and is not limited to this principle in the present disclosure.

[0074] In one embodiment, paragraph groups included in both paragraph group sets can be used as candidate paragraph groups. Subsequently, paragraph groups within the candidate paragraph groups that have unique positions in both paragraph group sets are determined as target paragraph groups, i.e., as aligned paragraphs based on paragraph groups. By limiting aligned paragraphs to target paragraph groups that have unique positions in both paragraph group sets, it is possible to avoid the situation where the two documents to be compared cannot be accurately aligned due to the inclusion of the same paragraph group at multiple positions in a single document. This can thereby improve the accuracy of the determined aligned paragraphs and the precision of the determined difference paragraphs.

[0075] Based on the document comparison method provided by the present disclosure, the present disclosure also provides a document comparison device. Figure 5 The device is described in detail.

[0076] Figure 5 It is a structural block diagram of a document comparison device according to an embodiment of the present disclosure.

[0077] like Figure 5 As shown, the document comparison device 500 of this embodiment may include a paragraph segmentation module 510 , an aligned paragraph determination module 520 , an aligned segmentation module 530 and a difference paragraph determination module 540 .

[0078] The paragraph segmentation module 510 is used to segment the two documents to be compared into paragraphs, respectively, to obtain a paragraph set of each document to be compared. In one embodiment, the paragraph segmentation module 510 can be used to perform the operation S210 described above, which will not be described in detail here.

[0079] The aligned paragraph determination module 520 is used to determine the aligned paragraphs of the two documents to be compared based on the two paragraph sets of the two documents to be compared. In one embodiment, the aligned paragraph determination module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0080] The alignment and segmentation module 530 is configured to segment the two documents to be compared based on the aligned paragraphs, thereby obtaining a plurality of paragraph segment pairs. Each paragraph segment pair includes two paragraph segments belonging to the two documents to be compared, respectively, and the two paragraph segments are obtained based on the same aligned paragraph segmentation. In one embodiment, the alignment and segmentation module 530 can be configured to perform operation S230 described above, and will not be further described herein.

[0081] The difference paragraph determination module 540 is used to determine the difference paragraphs of the two documents to be compared based on the edit distance in paragraph units between the two paragraph segments in each paragraph segment pair. In one embodiment, the difference paragraph determination module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0082] According to an embodiment of the present disclosure, the paragraph segmentation module 510 may include, for example, a document parsing submodule and a paragraph segmentation submodule. The document parsing submodule is configured to parse the two documents to be compared, respectively, and obtain the text elements and position information of each document to be compared as the parsing result of each document to be compared. The paragraph segmentation submodule is configured to segment each document to be compared into paragraphs based on the parsing result of each document to be compared, and obtain a set of paragraphs included in each document to be compared.

[0083] According to an embodiment of the present disclosure, the aligned paragraph determination module 520 may include, for example, a paragraph division submodule and an aligned paragraph determination submodule. The paragraph division submodule is configured to divide the paragraphs in the paragraph set of each document to be compared according to a sliding window of a predetermined size, thereby obtaining a paragraph group set consisting of a plurality of paragraph groups for each document to be compared. The aligned paragraph determination submodule is configured to determine aligned paragraphs based on the paragraph groups included in both paragraph group sets of the two documents to be compared, in units of paragraph groups.

[0084] According to an embodiment of the present disclosure, the above-mentioned aligned paragraph determination submodule may, for example, include a candidate paragraph determination unit and an aligned paragraph determination unit. The candidate paragraph determination unit is used to determine a paragraph group included in both paragraph group sets of the two documents to be compared as a candidate paragraph group. The aligned paragraph determination unit is used to determine a target paragraph group in the candidate paragraph group as an aligned paragraph based on the paragraph group. The target paragraph group is a paragraph group having a unique position in the two paragraph group sets.

[0085] According to an embodiment of the present disclosure, the alignment and segmentation module 530 may include, for example, an alignment segmentation submodule, a paragraph deletion submodule, and a paragraph pair acquisition submodule. The alignment segmentation submodule is used to segment each document to be compared according to the aligned paragraphs to obtain an initial paragraph segment group. The paragraph deletion submodule is used to delete the aligned paragraphs included in the initial paragraph segment group to obtain a paragraph segment group for each document to be compared. The paragraph pair acquisition submodule is used to combine two paragraph segments at the same position in two paragraph segment groups of two documents to be compared into a paragraph segment pair to obtain a plurality of paragraph segment pairs.

[0086] According to an embodiment of the present disclosure, the above-mentioned paragraph segmentation submodule may include, for example, a first paragraph determination unit and a second paragraph determination unit. The first paragraph determination unit is used to determine, in response to the parsing result of each document to be compared, that the subsequent adjacent position of the first title included in each document to be compared includes a text paragraph, and to treat the first title and the text paragraph at the subsequent adjacent position of the first title as one paragraph. The second paragraph determination unit is used to determine, in response to the parsing result of each document to be compared, that the subsequent adjacent position of the second title included in each document to be compared does not include a text paragraph, and to treat the second title and a third title at the same level as the second title included in the document to be compared as one paragraph.

[0087] According to an embodiment of the present disclosure, the document comparison device 500 may further include a document conversion module for converting two documents to be compared into documents in a portable file format to obtain two documents to be compared.

[0088] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information in the technical solutions disclosed herein comply with relevant laws and regulations, employ necessary confidentiality measures, and do not violate public order and good morals. In the technical solutions disclosed herein, user authorization or consent is obtained before obtaining or collecting user personal information.

[0089] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0090] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement the document comparison method of an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0091] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0092] Various components in device 600 are connected to I / O interface 605, including an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0093] The computing unit 601 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the document comparison method. For example, in some embodiments, the document comparison method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the document comparison method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the document comparison method by any other appropriate means (e.g., by means of firmware).

[0094] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0095] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0096] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0098] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0099] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system. It solves the problems of difficult management and poor business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server in a distributed system, or a server integrated with blockchain.

[0100] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0101] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A document comparison method, comprising: Parsing the two documents to be compared respectively to obtain text elements and position information of the text elements included in each document to be compared as the parsing result of each document to be compared; According to the parsing result of each document to be compared, each document to be compared is segmented into paragraphs to obtain a paragraph set included in each document to be compared, including: in response to determining, according to the parsing result of each document to be compared, that a text paragraph is included in a position adjacent to a first title included in each document to be compared, the first title and the text paragraph adjacent to the position adjacent to the first title are combined as one paragraph; in response to determining, according to the parsing result of each document to be compared, that a text paragraph is not included in a position adjacent to a second title included in each document to be compared, the second title and a third title of the same level as the second title included in the document to be compared are combined as one paragraph; Determining aligned paragraphs of the two documents to be compared based on the two paragraph sets of the two documents to be compared; Segmenting the two documents to be compared according to the aligned paragraphs to obtain a plurality of paragraph segment pairs; each paragraph segment pair includes two paragraph segments belonging to the two documents to be compared respectively; the two paragraph segments are obtained according to the same aligned paragraph segmentation; and The different paragraphs of the two documents to be compared are determined according to the edit distance in paragraph units between the two paragraph segments in each paragraph segment pair.

2. The method according to claim 1, wherein Determining the aligned paragraphs of the two documents to be compared according to the two paragraph sets of the two documents to be compared includes: Dividing the paragraphs in the paragraph set of each document to be compared according to a sliding window of a predetermined size to obtain a paragraph group set consisting of a plurality of paragraph groups for each document to be compared; and Aligned paragraphs in units of paragraph groups are determined according to the paragraph groups included in both paragraph group sets of the two documents to be compared.

3. The method according to claim 2, wherein: The step of determining aligned paragraphs based on the paragraph groups included in both paragraph group sets of the two documents to be compared comprises: determining, as candidate paragraph groups, paragraph groups included in both paragraph group sets of the two documents to be compared; and Determine a target paragraph group in the candidate paragraph group as an aligned paragraph based on the paragraph group, The target paragraph group is a paragraph group having a unique position in the two paragraph group sets.

4. The method according to claim 1, wherein Segmenting the two documents to be compared according to the aligned paragraphs to obtain a plurality of paragraph segment pairs includes: Segment each of the documents to be compared according to the aligned paragraphs to obtain an initial paragraph segment group; Deleting the aligned paragraphs included in the initial paragraph segment group to obtain a paragraph segment group for each document to be compared; and Two paragraph fragments at the same position in the two paragraph fragment groups of the two documents to be compared are combined into a paragraph fragment pair to obtain the multiple paragraph fragment pairs.

5. The method according to any one of claims 1 to 4, further comprising: The two documents to be compared are converted into documents in a portable document format to obtain the two documents to be compared.

6. A document comparison device comprising: The document parsing submodule is used to parse the two documents to be compared respectively, and obtain the text elements and position information of each document to be compared as the parsing result of each document to be compared; a paragraph segmentation submodule for segmenting each document to be compared into paragraphs based on the parsing results of each document to be compared, thereby obtaining a set of paragraphs included in each document to be compared, the paragraph segmentation submodule comprising: a first paragraph determination unit for determining, in response to the parsing results of each document to be compared, that a text paragraph is included in the subsequent adjacent position of a first title included in each document to be compared, and treating the first title and the text paragraph in the subsequent adjacent position of the first title as one paragraph; a second paragraph determination unit for determining, in response to the parsing results of each document to be compared, that a text paragraph is not included in the subsequent adjacent position of a second title included in each document to be compared, and treating the second title and a third title of the same level as the second title included in the document to be compared as one paragraph; An aligned paragraph determining module, configured to determine aligned paragraphs of the two documents to be compared based on the two paragraph sets of the two documents to be compared; an alignment and segmentation module, configured to segment the two documents to be compared according to the aligned paragraphs to obtain a plurality of paragraph segment pairs; each paragraph segment pair includes two paragraph segments belonging to the two documents to be compared respectively; the two paragraph segments are obtained according to the same aligned paragraph segmentation; and The difference paragraph determination module is used to determine the difference paragraphs of the two documents to be compared based on the edit distance in paragraph units between the two paragraph segments in each paragraph segment pair.

7. The device according to claim 6, wherein The aligned paragraph determination module includes: a paragraph division submodule, configured to divide the paragraphs in the paragraph set of each document to be compared according to a sliding window of a predetermined size, to obtain a paragraph group set consisting of a plurality of paragraph groups for each document to be compared; and The aligned paragraph determining submodule is configured to determine aligned paragraphs based on paragraph groups included in both paragraph group sets of the two documents to be compared.

8. The device according to claim 7, wherein The alignment paragraph determination submodule includes: a candidate paragraph determining unit, configured to determine, as a candidate paragraph group, a paragraph group included in both paragraph group sets of the two documents to be compared; and An aligned paragraph determining unit is configured to determine a target paragraph group in the candidate paragraph group as an aligned paragraph based on the paragraph group. The target paragraph group is a paragraph group having a unique position in the two paragraph group sets.

9. The device according to claim 6, wherein The alignment and segmentation module includes: An alignment segmentation submodule, configured to segment each document to be compared according to the aligned paragraphs to obtain an initial paragraph segment group; a paragraph deletion submodule, configured to delete the aligned paragraphs included in the initial paragraph segment group to obtain a paragraph segment group for each document to be compared; and The paragraph pair obtaining submodule is configured to form a paragraph segment pair from two paragraph segments at the same position in the two paragraph segment groups of the two documents to be compared, thereby obtaining the plurality of paragraph segment pairs.

10. The device according to any one of claims 6 to 9, further comprising: The document conversion module is used to convert the two documents to be compared into documents in a portable document format to obtain the two documents to be compared.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product, comprising a computer program / instructions, wherein the computer program / instructions are stored on at least one of a readable storage medium and an electronic device, and when the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images