Text comparison method and device, computer device and readable storage medium

Abstract

The present disclosure provides a text comparison method, device, computer equipment and readable storage medium. A comparison system calculates paragraph distances between each first paragraph of a first text to be compared and each second paragraph of a second text to be compared, and determines a pairing relationship between each first paragraph and each second paragraph based on a maximum flow minimum cost algorithm and the paragraph distances, to obtain a plurality of paired paragraph sets. The comparison system calculates short sentence distances between each first short sentence of a first paired paragraph in each paired paragraph set and each second short sentence of a second paired paragraph. Finally, based on the maximum flow minimum cost algorithm and the short sentence distances, a pairing relationship between each first short sentence and each second short sentence is determined, and a plurality of first short sentences and second short sentences without a pairing relationship are filtered from each paired paragraph set as distinguished texts. The overall processing logic of the present disclosure has a low complexity, a fast processing speed, and a low requirement for hardware resources.

Description

Technical Field

[0001] This disclosure relates to the field of text processing technology, and in particular to a text comparison method, apparatus, computer device, and readable storage medium. Background Technology

[0002] Whether in finance or office work, the comparison of business and office documents (including docx, pdf, and scanned documents) is essential to identify and detect differences between two documents. Existing text comparison methods typically require pre-training a neural network model for comparison. This approach is not only costly but also resource-intensive, hindering large-scale application. Summary of the Invention

[0003] The purpose of this disclosure is to provide a text comparison method, apparatus, computer device, and readable storage medium that does not require a neural network model when performing text comparison, has low resource requirements for application hardware, and has a low overall application cost.

[0004] To achieve the above objectives, this disclosure adopts the following technical solution: a text comparison method, comprising:

[0005] Calculate the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared, wherein each paragraph distance corresponds to a single first paragraph and a single second paragraph;

[0006] Based on the maximum flow minimum cost algorithm and the distance between each segment, the pairing relationship between each first segment and each second segment is determined, resulting in several pairs of segments. Each pair of segments contains a first paired segment and a second paired segment that have a pairing relationship.

[0007] Calculate the sentence distance between each first sentence of the first paired paragraph and each second sentence of the second paired paragraph in each of the paired paragraph sets, wherein each sentence distance corresponds to a single first sentence and a single second sentence;

[0008] Based on the maximum flow minimum cost algorithm and the distance between each phrase, the pairing relationship between each first phrase and each second phrase is determined;

[0009] From each of the paired paragraph sets, select several first and second short sentences that do not have a pairing relationship as distinguishing texts.

[0010] This disclosure also provides a text comparison apparatus, comprising:

[0011] The first calculation module is used to calculate the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared, wherein each paragraph distance corresponds to a single first paragraph and a single second paragraph;

[0012] The first pairing module is used to determine the pairing relationship between each first paragraph and each second paragraph based on the maximum flow minimum cost algorithm and the distance between each paragraph, so as to obtain a number of paired paragraph sets, wherein a single paired paragraph set contains a first paired paragraph and a second paired paragraph that have a pairing relationship;

[0013] The second calculation module is used to calculate the sentence distance between each first sentence of the first paired paragraph and each second sentence of the second paired paragraph in each of the paired paragraph sets, wherein each sentence distance corresponds to a single first sentence and a single second sentence;

[0014] The second pairing module is used to determine the pairing relationship between each first short phrase and each second short phrase based on the maximum flow minimum cost algorithm and the distance between each phrase;

[0015] The filtering module is used to filter out several first and second short sentences that do not have a pairing relationship from each of the paired paragraph sets as distinguishing text.

[0016] This disclosure also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0017] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0018] This disclosure provides a text comparison method, apparatus, computer device, and readable storage medium. The comparison system calculates the paragraph distances between each first paragraph of a first text to be compared and each second paragraph of a second text to be compared, where a single paragraph distance corresponds to a single first paragraph and a single second paragraph. Then, based on the maximum flow minimum cost algorithm and the paragraph distances, the pairing relationships between each first paragraph and each second paragraph are determined, resulting in several sets of paired paragraphs. Each set of paired paragraphs contains one first paired paragraph and one second paired paragraph with a pairing relationship. The comparison system calculates the sentence distances between each first clause of the first paired paragraph and each second clause of the second paired paragraph in each set of paired paragraphs, where a single sentence distance corresponds to a single first clause and a single second clause. Finally, based on the maximum flow minimum cost algorithm and the sentence distances, the pairing relationships between each first clause and each second clause are determined, and several first and second clauses without pairing relationships are selected from each set of paired paragraphs as distinguishing text. This disclosure first identifies paragraphs with similar content in the first and second texts to be compared (i.e., the first and second paragraphs in the paired paragraph set have similar content) through paragraph pairing. Next, short sentence pairing is performed based on the paired paragraph set to filter out distinguishing texts that do not have a pairing relationship. The overall processing logic has low complexity, thus enabling fast processing speed, while also requiring less hardware resources, which is conducive to large-scale application. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the steps of a text comparison method in one embodiment of this disclosure;

[0020] Figure 2 This is a schematic diagram of the structure of a cost-capacity network in one embodiment of this disclosure;

[0021] Figure 3 This is an overall structural block diagram of a text comparison device in one embodiment of the present disclosure;

[0022] Figure 4 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present disclosure.

[0023] The realization of the purpose, functional features and advantages of this disclosure will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0025] Reference Figure 1One embodiment of this disclosure provides a text comparison method, which includes:

[0026] S1: Calculate the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared, wherein each paragraph distance corresponds to a single first paragraph and a single second paragraph;

[0027] S2: Based on the maximum flow minimum cost algorithm and the distance between each paragraph, determine the pairing relationship between each first paragraph and each second paragraph to obtain several pairs of paragraph sets, wherein a single pair of paragraph sets contains a first paired paragraph and a second paired paragraph that have a pairing relationship;

[0028] S3: Calculate the sentence distance between each first sentence of the first paired paragraph and each second sentence of the second paired paragraph in each of the paired paragraph sets, wherein each sentence distance corresponds to a single first sentence and a single second sentence;

[0029] S4: Based on the maximum flow minimum cost algorithm and the distance between each of the short sentences, determine the pairing relationship between each of the first short sentences and each of the second short sentences;

[0030] S5: Select several first and second short sentences that do not have a pairing relationship from the sets of paired paragraphs as distinguishing texts.

[0031] In this embodiment, the text comparison system acquires a first text to be compared and a second text to be compared, and then calculates the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared; wherein, a single first paragraph and a single second paragraph correspond to a single paragraph distance. Specifically, the text comparison system groups a single first paragraph and a single second paragraph together, and uses the string edit distance of the first sentence of each paragraph in the same group as the paragraph distance between the first paragraph and the second paragraph of that group. After obtaining the paragraph distance between each first paragraph and its corresponding second paragraph, the text comparison system constructs a paragraph matrix based on the number of first paragraphs in the first text to be compared (the number of first paragraphs is the total number of first paragraphs contained in the first text to be compared), the number of second paragraphs in the second text to be compared, and the paragraph distances; wherein, the size of the paragraph matrix is ​​N×M, where N represents the number of first paragraphs, M represents the number of second paragraphs, and the elements of the paragraph matrix represent the paragraph distance between the corresponding paragraphs of the first and second texts to be compared. The text comparison system, based on the maximum flow minimum cost algorithm and the distance between paragraphs, determines the pairwise pairing relationships between each first paragraph in the first text to be compared and each second paragraph in the second text to be compared, thus obtaining several sets of paired paragraphs. Each set of paired paragraphs contains one first paired paragraph and one second paired paragraph with a pairing relationship. The first and second paired paragraphs with a pairing relationship have similar or related semantics, both describing the same topic, such as tourist attractions or policy interpretations. Specifically, the text comparison system constructs virtual source and virtual sink nodes as the left and right endpoints of the cost-capacity network, and constructs intermediate nodes of the cost-capacity network based on the number of first and second paragraphs. The first intermediate node corresponding to the number of first paragraphs is closer to the virtual source node, and the second intermediate node corresponding to the number of second paragraphs is closer to the virtual sink node. Each first intermediate node and each second intermediate node belongs to different columns. The virtual source node establishes connection edges with each first intermediate node, and the virtual sink node establishes connection edges with each second intermediate node. Then, the text comparison system retrieves the distance threshold and establishes connection edges between each first intermediate node and each second intermediate node based on the elements contained in the paragraph matrix (i.e., the paragraph distance between the corresponding paragraphs of the first and second texts to be compared) and the distance threshold. In the cost-capacity network, the edge capacity of each connection edge is 1, and the transportation cost of the connection edge is the paragraph distance between the first and second intermediate nodes at both ends of the connection edge. The text comparison system solves the cost-capacity network using the maximum flow minimum cost algorithm, selecting a first paragraph and a second paragraph with similar semantics to form a paired paragraph set, thus obtaining several paired paragraph sets.Based on each paired paragraph set, the sentence distances between each first clause of the first paired paragraph and each second clause of the second paired paragraph are calculated for each paired paragraph set. The single sentence distance corresponds to a single first clause and a single second clause. For example, if the first paired paragraph contains first clause a, first clause b, and first clause c; and the second paired paragraph contains second clause A and second clause B, then the text comparison system needs to calculate the sentence distances between first clause a and second clause A, between first clause a and second clause B, between first clause b and second clause A, between first clause b and second clause B, between first clause c and second clause A, and between first clause c and second clause B (i.e., calculating sentence distances in pairs). Specifically, the short sentences referred to in this embodiment are text segments using symbols such as commas, colons, exclamation marks, periods, and question marks. The short sentence distance is the string editing distance between the first short sentence and the second short sentence (the calculation logic is similar to that of the paragraph distance mentioned above). After obtaining the short sentence distances between the first short sentences of the first paired paragraph and the second short sentences of the second paired paragraph in each paired paragraph set, a short sentence matrix is ​​constructed for each paired paragraph set based on the number of first short sentences in the first paired paragraph and the number of second short sentences in the second paired paragraph. On this basis, a cost-capacity network and each node and connecting edge in the cost-capacity network are constructed (the construction logic of the short sentence matrix, cost-capacity network, nodes, and connecting edges is the same as that used when pairing paragraphs above, and will not be repeated here; it should be noted that the transportation cost of the connecting edge of the cost-capacity network corresponding to the short sentence is the short sentence distance between the first intermediate node and the second intermediate node at both ends of the connecting edge). After constructing a cost-capacity network corresponding to the short sentences, the text comparison system, based on the maximum flow minimum cost algorithm and the distance between each short sentence, determines the pairing relationship between each first short sentence in the first paired paragraph and each second short sentence in the second paired paragraph within each paired paragraph set. A first short sentence and a second short sentence forming a pairing relationship indicate that the two short sentences have the same or similar semantics. The purpose of this disclosure is to filter out the hidden differences in paragraphs with similar semantics in two texts (i.e., the first text to be compared and the second text to be compared). Therefore, the text comparison system filters out several first short sentences and second short sentences that do not have a pairing relationship from each paired paragraph set (i.e., short sentences in each paired paragraph set that cannot form a pairing relationship between the first and second paired paragraphs) as distinguishing texts, enabling readers to quickly focus on different content from two similar texts, facilitating rapid retrieval and filtering by readers.

[0032] In this embodiment, the text comparison system first identifies paragraphs with similar content in the first and second texts to be compared (i.e., the first and second paragraphs in the paired paragraph set have similar content). Next, based on the paired paragraph set, short sentence pairing is performed to filter out texts that do not have a pairing relationship. The overall processing logic has low complexity, thus enabling fast processing speed, and also requires less hardware resources, which is beneficial for large-scale application.

[0033] Furthermore, the step of calculating the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared includes:

[0034] S101: Grouping a single first paragraph and a single second paragraph together, the string edit distance of the first sentence text of each of the two paragraphs in the same group is used as the paragraph distance;

[0035] S102: Based on the number of first paragraphs in the first text to be compared, the number of second paragraphs in the second text to be compared, and the distance between each paragraph, a paragraph matrix is ​​constructed, wherein the size of the paragraph matrix is ​​NxM, N represents the number of first paragraphs, M represents the number of second paragraphs, and the elements of the paragraph matrix represent the paragraph distance between the corresponding paragraphs of the first text to be compared and the second text to be compared.

[0036] In this embodiment, the text comparison system groups a single first paragraph and a single second paragraph together (i.e., each paragraph of the first and second texts to be compared forms a pair; for example, the first text to be compared contains first paragraphs a, b, and c, and the second text to be compared contains second paragraphs A and B; then first paragraphs a and A form one group, first paragraphs a and B form another group, first paragraphs b and A form yet another group, first paragraphs a and B form yet another group, first paragraphs c and A form yet another group, and first paragraphs c and B form yet another group). The character creation distance between the first sentence of each of the two paragraphs in the same group (i.e., the first paragraph and the second paragraph) is used as the paragraph distance between the first and second paragraphs in that group. Following this logic, the paragraph distances of each group (i.e., the paragraph distances between each pair of first paragraphs and each pair of second paragraphs) are obtained. The text comparison system constructs a paragraph matrix based on the number of first paragraphs in the first text to be compared, the number of second paragraphs in the second text to be compared, and the number of paragraphs in each group. The paragraph matrix has a size of N x M, where N represents the number of first paragraphs, M represents the number of second paragraphs, and the elements of the paragraph matrix represent the paragraph distance between corresponding paragraphs in the first and second comparison texts. For example, if the first comparison text has 4 first paragraphs and the second comparison text has 3 second paragraphs, the constructed paragraph matrix D would be: The value of element D(1,1) is 7, which means that the paragraph distance between the first paragraph of the first text to be compared and the first paragraph of the second text to be compared is 7.

[0037] Furthermore, the step of determining the pairing relationship between each first segment and each second segment based on the maximum flow minimum cost algorithm and the distance between each segment, to obtain several sets of paired segments, includes:

[0038] S201: Construct virtual source and virtual sink as the left and right endpoints of the cost capacity network, and construct intermediate nodes of the cost capacity network according to the first number of segments and the second number of segments. The first intermediate node corresponding to the first number of segments is close to the virtual source, and the second intermediate node corresponding to the second number of segments is close to the virtual sink. The virtual source establishes connection edges with each of the first intermediate nodes, and the virtual sink establishes connection edges with each of the second intermediate nodes.

[0039] S202: Retrieve the distance threshold and establish the connection edge between each first intermediate node and each second intermediate node according to the paragraph matrix and the distance threshold, wherein the edge capacity of the connection edge in the cost-capacity network is 1, and the transportation cost of the connection edge is the paragraph distance between the first intermediate node and the second intermediate node at both ends of the connection edge.

[0040] S203: Solve the cost-capacity network according to the maximum flow minimum cost algorithm to obtain several sets of paired segments.

[0041] like Figure 2 As shown, in this embodiment, the text comparison system first constructs virtual source nodes and virtual sink nodes as the left and right endpoints of the cost-capacity network, and then constructs intermediate nodes of the cost-capacity network based on the number of first paragraphs (i.e., the total number of first paragraphs contained in the first text to be compared) and the number of second paragraphs. Taking matrix D in the embodiment corresponding to steps S101 and S102 as an example, the first intermediate nodes corresponding to the first text to be compared are 4, close to the virtual source nodes; the second intermediate nodes corresponding to the second text to be compared are 3, close to the virtual sink nodes; each first intermediate node and each second intermediate node belongs to a different column. The virtual source node establishes connection edges with each of the first intermediate nodes in the closest column, and the virtual sink node establishes connection edges with each of the second intermediate nodes in the closest column. The text comparison system retrieves a preset distance threshold, and based on the first paragraph corresponding to each first intermediate node and the second paragraph corresponding to each second intermediate node, it finds the corresponding first paragraph distance (the first paragraph distance is the paragraph distance between the first paragraph and the second paragraph) in the constructed paragraph matrix. Then, it is determined whether the distance of the first paragraph is less than a distance threshold. If the distance of the first paragraph is less than the distance threshold, a connecting edge is constructed between the first intermediate node and the second intermediate node corresponding to the distance of the first paragraph. For example... Figure 2 As shown, the edge capacity of each connection edge in the cost-capacity network is 1, and the transportation cost of each connection edge is the segment distance between the first and second intermediate nodes at both ends of the connection edge (the transportation cost of the connection edge between the virtual source, virtual sink, and intermediate nodes is 0 by default). The text comparison system solves the cost-capacity network using the maximum flow minimum cost algorithm to obtain several paired segment sets. Similarly, using... Figure 2 Taking an example, the first paragraph is connected to the second and third paragraphs respectively. Since the transportation cost between the first paragraph and the second paragraph is 2, and the transportation cost between the first paragraph and the second paragraph is 3, it can be seen that the total transportation cost through the intermediate nodes corresponding to the first paragraph and the second paragraph is lower in the cost-capacity network. Therefore, the first paragraph and the second paragraph form a pair, constituting a paired paragraph set. According to this logic, Figure 2 The solution for the cost-capacity network is: the first paragraph is paired with the second paragraph, the third paragraph is paired with the first paragraph, and the fourth paragraph is paired with the third paragraph.

[0042] Furthermore, in the step of establishing the connection edges between each first intermediate node and each second intermediate node based on the paragraph matrix and the distance threshold, the step of establishing the connection edge corresponding to a single first intermediate node and a single second intermediate node includes:

[0043] S20201: Based on the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the second intermediate node, find the corresponding first paragraph distance in the paragraph matrix;

[0044] S20202: Determine whether the distance of the first paragraph is less than the distance threshold;

[0045] S20203: If the distance of the first paragraph is less than the distance threshold, then construct a connection edge between the first intermediate node and the second intermediate node.

[0046] In this embodiment, the construction logic of the connection edges between each first intermediate node and each second intermediate node is the same. The steps for establishing the connection edge between a single first intermediate node and a single second intermediate node are used as an example. The text comparison system, based on the order of the first paragraph corresponding to the first intermediate node in the first text to be compared, and the order of the second paragraph corresponding to the second intermediate node in the second text to be compared, finds the distance between the first paragraph and the second paragraph in the paragraph matrix. Then, it determines whether the distance between the first paragraphs is less than a preset distance threshold. If the distance is less than the threshold, a connection edge is constructed between the first intermediate node and the second intermediate node. If the distance is greater than the preset threshold, no connection edge is constructed between the first intermediate node and the second intermediate node. For example, the distance threshold is set to 6, and the matrix D corresponding to the first and second texts to be compared is used as an example. Figure 2 As shown, the paragraph distance between the first paragraph of the first text to be compared and the first paragraph of the second text to be compared is 7, which is greater than 6. Therefore, no connecting edge will be built between the first intermediate node and the second intermediate node corresponding to the first paragraph and the first paragraph respectively. However, the paragraph distances between the first paragraph of the first text to be compared and the second and third paragraphs of the first text to be compared are 2 and 3, respectively, which are both less than 6. Therefore, connecting edges will be built between the first intermediate node and the second intermediate node corresponding to the first paragraph and the second paragraph, and between the first paragraph and the third paragraph respectively.

[0047] Furthermore, in the step of solving the cost-capacity network according to the maximum flow minimum cost algorithm to obtain several sets of paired segments, the step of solving a single set of paired segments includes:

[0048] S20301: Determine whether the several second intermediate nodes connected to the first intermediate node have been paired;

[0049] S20302: If none of the several second intermediate nodes connected to the first intermediate node are paired, calculate the total cost of the feasible flow corresponding to the first intermediate node, and select the second intermediate node corresponding to the feasible flow with the smallest total cost as the pairing node of the first intermediate node.

[0050] S20303: Construct the paired paragraph set using the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the paired node.

[0051] In this embodiment, the solution logic for each paired paragraph set is the same. The solution process for a single paired paragraph set is used as an example for explanation. The text comparison system first determines whether the several second intermediate nodes connected to the current first intermediate node have already been paired (i.e., whether the second intermediate node forms a pairing relationship with other first intermediate nodes). If the first intermediate node is already paired with one of the second intermediate nodes, to ensure maximum flow (i.e., the cost-capacity network has the most feasible flows), this second intermediate node is not considered for further pairing (i.e., the current first intermediate node does not perform pairing solutions with already paired second intermediate nodes). If none of the several second intermediate nodes connected to the first intermediate node have been paired, the total cost of the feasible flows corresponding to the first intermediate node is calculated, and the second intermediate node corresponding to the feasible flow with the smallest total cost is selected as the paired node of the first intermediate node. Finally, the text comparison system uses the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the paired node to construct a paired paragraph set.

[0052] Furthermore, after the step of selecting several first and second short sentences that do not have a pairing relationship from each of the paired paragraph sets as distinguishing texts, the process includes:

[0053] S6: Highlight the distinguishing text in both the first text to be compared and the second text to be compared.

[0054] In this embodiment, after filtering out the distinguishing texts, the text comparison system highlights the distinguishing texts in both the first and / or second texts to be compared. For example, the distinguishing texts can be bolded, underlined, or displayed in a prominent color (such as red font) to allow users to quickly find the distinguishing content (i.e., the distinguishing texts) between the two texts, thus improving the user experience.

[0055] Reference Figure 3 This disclosure provides a text comparison apparatus, comprising:

[0056] The first calculation module 1 is used to calculate the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared, wherein each paragraph distance corresponds to a single first paragraph and a single second paragraph;

[0057] The first pairing module 2 is used to determine the pairing relationship between each first paragraph and each second paragraph based on the maximum flow minimum cost algorithm and the distance between each paragraph, so as to obtain a number of paired paragraph sets, wherein a single paired paragraph set contains a first paired paragraph and a second paired paragraph that have a pairing relationship;

[0058] The second calculation module 3 is used to calculate the sentence distance between each first sentence of the first paired paragraph and each second sentence of the second paired paragraph in each of the paired paragraph sets, wherein each sentence distance corresponds to a single first sentence and a single second sentence;

[0059] The second pairing module 4 is used to determine the pairing relationship between each first short phrase and each second short phrase based on the maximum flow minimum cost algorithm and the distance between each short phrase;

[0060] The filtering module 5 is used to filter out several first and second short sentences that do not have a pairing relationship from each of the paired paragraph sets as distinguishing texts.

[0061] Furthermore, the first computing module 1 includes:

[0062] A calculation unit is configured to group a single first paragraph and a single second paragraph together, and use the string edit distance of the first sentence text of each of the two paragraphs in the same group as the paragraph distance;

[0063] The first construction unit is configured to construct a paragraph matrix based on the first number of paragraphs in the first text to be compared, the second number of paragraphs in the second text to be compared, and the distance between each paragraph. The paragraph matrix has a size of N×M, where N represents the first number of paragraphs, M represents the second number of paragraphs, and the elements of the paragraph matrix represent the paragraph distance between the corresponding paragraphs of the first text to be compared and the second text to be compared.

[0064] Furthermore, the first pairing module 2 includes:

[0065] The second construction unit is used to construct virtual source points and virtual sink points as the left and right endpoints of the cost capacity network, and to construct intermediate nodes of the cost capacity network according to the first number of segments and the second number of segments. The first intermediate node corresponding to the first number of segments is close to the virtual source point, and the second intermediate node corresponding to the second number of segments is close to the virtual sink point. The virtual source point establishes connection edges with each of the first intermediate nodes, and the virtual sink point establishes connection edges with each of the second intermediate nodes.

[0066] A unit is established to retrieve a distance threshold and establish a connection edge between each first intermediate node and each second intermediate node according to the paragraph matrix and the distance threshold. The edge capacity of the connection edge in the cost-capacity network is 1, and the transportation cost of the connection edge is the paragraph distance between the first intermediate node and the second intermediate node at both ends of the connection edge.

[0067] The solution unit is used to solve the cost-capacity network according to the maximum flow minimum cost algorithm to obtain a set of paired segments.

[0068] Furthermore, the establishment unit includes:

[0069] The search sub-unit is used to find the corresponding first paragraph distance in the paragraph matrix based on the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the second intermediate node;

[0070] The first judgment subunit is used to determine whether the distance of the first paragraph is less than the distance threshold.

[0071] The first construction subunit is used to construct a connection edge between the first intermediate node and the second intermediate node if the distance between the first paragraphs is less than the distance threshold.

[0072] Furthermore, the solution unit includes:

[0073] The second judgment subunit is used to determine whether the several second intermediate nodes connected to the first intermediate node have been paired.

[0074] The calculation subunit is used to calculate the total cost of the feasible flow corresponding to the first intermediate node if none of the several second intermediate nodes connected to the first intermediate node are paired, and select the second intermediate node corresponding to the feasible flow with the smallest total cost as the pairing node of the first intermediate node.

[0075] The second construction subunit is used to construct the paired paragraph set using the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the paired node.

[0076] Furthermore, the text comparison device also includes:

[0077] Display module 6 is used to highlight the distinguishing text in the first text to be compared and the second text to be compared, respectively.

[0078] In this embodiment, each module, unit, and subunit in the text comparison device is used to perform each step in the text comparison method described above, and the specific implementation process is not described in detail here.

[0079] This disclosure provides a text comparison device. The comparison system calculates the paragraph distance between each first paragraph of a first text to be compared and each second paragraph of a second text to be compared, where a single paragraph distance corresponds to a single first paragraph and a single second paragraph. Then, based on the maximum flow minimum cost algorithm and the paragraph distances, the pairing relationship between each first paragraph and each second paragraph is determined, resulting in several sets of paired paragraphs. Each set of paired paragraphs contains one first paired paragraph and one second paired paragraph with a pairing relationship. The comparison system calculates the sentence distance between each first clause of the first paired paragraph and each second clause of the second paired paragraph in each set of paired paragraphs, where a single sentence distance corresponds to a single first clause and a single second clause. Finally, based on the maximum flow minimum cost algorithm and the sentence distances, the pairing relationship between each first clause and each second clause is determined, and several first clauses and second clauses without pairing relationships are selected from each set of paired paragraphs as distinguishing text. This disclosure first determines paragraphs with similar content in the first and second texts to be compared (i.e., the first paragraphs and second paragraphs in the paired paragraph sets have similar content) through paragraph pairing. Next, short sentence pairing is performed based on the paired paragraph set to filter out distinguishing texts that do not have pairing relationships. The overall processing logic has low complexity, thus enabling fast processing speed, and it also has low requirements for application hardware resources, which is conducive to large-scale application.

[0080] Reference Figure 4 This disclosure also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data such as the first text to be compared. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a text comparison method.

[0081] The processor described above performs the following steps in the text comparison method described above:

[0082] S1: Calculate the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared, wherein each paragraph distance corresponds to a single first paragraph and a single second paragraph;

[0083] S2: Based on the maximum flow minimum cost algorithm and the distance between each paragraph, determine the pairing relationship between each first paragraph and each second paragraph to obtain several pairs of paragraph sets, wherein a single pair of paragraph sets contains a first paired paragraph and a second paired paragraph that have a pairing relationship;

[0084] S3: Calculate the sentence distance between each first sentence of the first paired paragraph and each second sentence of the second paired paragraph in each of the paired paragraph sets, wherein each sentence distance corresponds to a single first sentence and a single second sentence;

[0085] S4: Based on the maximum flow minimum cost algorithm and the distance between each of the short sentences, determine the pairing relationship between each of the first short sentences and each of the second short sentences;

[0086] S5: Select several first and second short sentences that do not have a pairing relationship from the sets of paired paragraphs as distinguishing texts.

[0087] Furthermore, the step of calculating the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared includes:

[0088] S101: Grouping a single first paragraph and a single second paragraph together, the string edit distance of the first sentence text of each of the two paragraphs in the same group is used as the paragraph distance;

[0089] S102: Based on the number of first paragraphs in the first text to be compared, the number of second paragraphs in the second text to be compared, and the distance between each paragraph, a paragraph matrix is ​​constructed, wherein the size of the paragraph matrix is ​​NxM, N represents the number of first paragraphs, M represents the number of second paragraphs, and the elements of the paragraph matrix represent the paragraph distance between the corresponding paragraphs of the first text to be compared and the second text to be compared.

[0090] Furthermore, the step of determining the pairing relationship between each first segment and each second segment based on the maximum flow minimum cost algorithm and the distance between each segment, to obtain several sets of paired segments, includes:

[0091] S201: Construct virtual source and virtual sink as the left and right endpoints of the cost capacity network, and construct intermediate nodes of the cost capacity network according to the first number of segments and the second number of segments. The first intermediate node corresponding to the first number of segments is close to the virtual source, and the second intermediate node corresponding to the second number of segments is close to the virtual sink. The virtual source establishes connection edges with each of the first intermediate nodes, and the virtual sink establishes connection edges with each of the second intermediate nodes.

[0092] S202: Retrieve the distance threshold and establish the connection edge between each first intermediate node and each second intermediate node according to the paragraph matrix and the distance threshold, wherein the edge capacity of the connection edge in the cost-capacity network is 1, and the transportation cost of the connection edge is the paragraph distance between the first intermediate node and the second intermediate node at both ends of the connection edge.

[0093] S203: Solve the cost-capacity network according to the maximum flow minimum cost algorithm to obtain several sets of paired segments.

[0094] Furthermore, in the step of establishing the connection edges between each first intermediate node and each second intermediate node based on the paragraph matrix and the distance threshold, the step of establishing the connection edge corresponding to a single first intermediate node and a single second intermediate node includes:

[0095] S20201: Based on the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the second intermediate node, find the corresponding first paragraph distance in the paragraph matrix;

[0096] S20202: Determine whether the distance of the first paragraph is less than the distance threshold;

[0097] S20203: If the distance of the first paragraph is less than the distance threshold, then construct a connection edge between the first intermediate node and the second intermediate node.

[0098] Furthermore, in the step of solving the cost-capacity network according to the maximum flow minimum cost algorithm to obtain several sets of paired segments, the step of solving a single set of paired segments includes:

[0099] S20301: Determine whether the several second intermediate nodes connected to the first intermediate node have been paired;

[0100] S20302: If none of the several second intermediate nodes connected to the first intermediate node are paired, calculate the total cost of the feasible flow corresponding to the first intermediate node, and select the second intermediate node corresponding to the feasible flow with the smallest total cost as the pairing node of the first intermediate node.

[0101] S20303: Construct the paired paragraph set using the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the paired node.

[0102] Furthermore, after the step of selecting several first and second short sentences that do not have a pairing relationship from each of the paired paragraph sets as distinguishing texts, the process includes:

[0103] S6: Highlight the distinguishing text in both the first text to be compared and the second text to be compared.

[0104] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements a text comparison method, wherein the text comparison method specifically includes:

[0105] S1: Calculate the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared, wherein each paragraph distance corresponds to a single first paragraph and a single second paragraph;

[0106] S2: Based on the maximum flow minimum cost algorithm and the distance between each paragraph, determine the pairing relationship between each first paragraph and each second paragraph to obtain several pairs of paragraph sets, wherein a single pair of paragraph sets contains a first paired paragraph and a second paired paragraph that have a pairing relationship;

[0107] S3: Calculate the sentence distance between each first sentence of the first paired paragraph and each second sentence of the second paired paragraph in each of the paired paragraph sets, wherein each sentence distance corresponds to a single first sentence and a single second sentence;

[0108] S4: Based on the maximum flow minimum cost algorithm and the distance between each of the short sentences, determine the pairing relationship between each of the first short sentences and each of the second short sentences;

[0109] S5: Select several first and second short sentences that do not have a pairing relationship from the sets of paired paragraphs as distinguishing texts.

[0110] Furthermore, the step of calculating the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared includes:

[0111] S101: Grouping a single first paragraph and a single second paragraph together, the string edit distance of the first sentence text of each of the two paragraphs in the same group is used as the paragraph distance;

[0112] S102: Based on the number of first paragraphs in the first text to be compared, the number of second paragraphs in the second text to be compared, and the distance between each paragraph, a paragraph matrix is ​​constructed, wherein the size of the paragraph matrix is ​​NxM, N represents the number of first paragraphs, M represents the number of second paragraphs, and the elements of the paragraph matrix represent the paragraph distance between the corresponding paragraphs of the first text to be compared and the second text to be compared.

[0113] Furthermore, the step of determining the pairing relationship between each first segment and each second segment based on the maximum flow minimum cost algorithm and the distance between each segment, to obtain several sets of paired segments, includes:

[0114] S201: Construct virtual source and virtual sink as the left and right endpoints of the cost capacity network, and construct intermediate nodes of the cost capacity network according to the first number of segments and the second number of segments. The first intermediate node corresponding to the first number of segments is close to the virtual source, and the second intermediate node corresponding to the second number of segments is close to the virtual sink. The virtual source establishes connection edges with each of the first intermediate nodes, and the virtual sink establishes connection edges with each of the second intermediate nodes.

[0115] S202: Retrieve the distance threshold and establish the connection edge between each first intermediate node and each second intermediate node according to the paragraph matrix and the distance threshold, wherein the edge capacity of the connection edge in the cost-capacity network is 1, and the transportation cost of the connection edge is the paragraph distance between the first intermediate node and the second intermediate node at both ends of the connection edge.

[0116] S203: Solve the cost-capacity network according to the maximum flow minimum cost algorithm to obtain several sets of paired segments.

[0117] Furthermore, in the step of establishing the connection edges between each first intermediate node and each second intermediate node based on the paragraph matrix and the distance threshold, the step of establishing the connection edge corresponding to a single first intermediate node and a single second intermediate node includes:

[0118] S20201: Based on the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the second intermediate node, find the corresponding first paragraph distance in the paragraph matrix;

[0119] S20202: Determine whether the distance of the first paragraph is less than the distance threshold;

[0120] S20203: If the distance of the first paragraph is less than the distance threshold, then construct a connection edge between the first intermediate node and the second intermediate node.

[0121] Furthermore, in the step of solving the cost-capacity network according to the maximum flow minimum cost algorithm to obtain several sets of paired segments, the step of solving a single set of paired segments includes:

[0122] S20301: Determine whether the several second intermediate nodes connected to the first intermediate node have been paired;

[0123] S20302: If none of the several second intermediate nodes connected to the first intermediate node are paired, calculate the total cost of the feasible flow corresponding to the first intermediate node, and select the second intermediate node corresponding to the feasible flow with the smallest total cost as the pairing node of the first intermediate node.

[0124] S20303: Construct the paired paragraph set using the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the paired node.

[0125] Furthermore, after the step of selecting several first and second short sentences that do not have a pairing relationship from each of the paired paragraph sets as distinguishing texts, the process includes:

[0126] S6: Highlight the distinguishing text in both the first text to be compared and the second text to be compared.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this disclosure and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0128] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, first object, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, first object, or method. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, first object, or method that includes that element.

[0129] The above description is only a preferred embodiment of this disclosure and does not limit the patent scope of this disclosure. Any equivalent structural or procedural changes made based on the content of this disclosure and its drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this disclosure.

Claims

1. A text comparison method, wherein, include: Calculate the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared, wherein each paragraph distance corresponds to a single first paragraph and a single second paragraph; Based on the maximum flow minimum cost algorithm and the distance between each segment, the pairing relationship between each first segment and each second segment is determined, resulting in several pairs of segments. Each pair of segments contains a first paired segment and a second paired segment that have a pairing relationship. Calculate the sentence distance between each first sentence of the first paired paragraph and each second sentence of the second paired paragraph in each of the paired paragraph sets, wherein each sentence distance corresponds to a single first sentence and a single second sentence; Based on the maximum flow minimum cost algorithm and the distance between each phrase, the pairing relationship between each first phrase and each second phrase is determined; Select several first and second sentences that do not have a pairing relationship from each of the aforementioned paired paragraph sets as distinguishing texts; The step of calculating the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared includes: The paragraph distance is defined as the string edit distance of the first sentence text of each of the two paragraphs in the same group, with each first paragraph and each second paragraph as a group. A paragraph matrix is ​​constructed based on the number of the first paragraph of the first text to be compared, the number of the second paragraph of the second text to be compared, and the distance between each paragraph. The size of the paragraph matrix is ​​N×M, where N represents the number of the first paragraph, M represents the number of the second paragraph, and the elements of the paragraph matrix represent the paragraph distance between the corresponding paragraphs of the first text to be compared and the second text to be compared. The step of determining the pairing relationship between each first segment and each second segment based on the maximum flow minimum cost algorithm and the distance between each segment, to obtain several sets of paired segments, includes: Virtual source and virtual sink are constructed as the left and right endpoints of the cost capacity network, respectively. The intermediate nodes of the cost capacity network are constructed according to the first number of segments and the second number of segments. The first intermediate node corresponding to the first number of segments is close to the virtual source, and the second intermediate node corresponding to the second number of segments is close to the virtual sink. The virtual source establishes a connection edge with each of the first intermediate nodes, and the virtual sink establishes a connection edge with each of the second intermediate nodes. Retrieve the distance threshold and establish the connection edges between each first intermediate node and each second intermediate node according to the paragraph matrix and the distance threshold, wherein the edge capacity of the connection edge in the cost-capacity network is 1, and the transportation cost of the connection edge is the paragraph distance between the first intermediate node and the second intermediate node at both ends of the connection edge. The cost-capacity network is solved using the maximum flow minimum cost algorithm to obtain several sets of paired segments.

2. The text comparison method according to claim 1, wherein, In the step of establishing the connection edges between each first intermediate node and each second intermediate node based on the paragraph matrix and the distance threshold, the step of establishing the connection edge corresponding to a single first intermediate node and a single second intermediate node includes: Based on the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the second intermediate node, find the corresponding first paragraph distance in the paragraph matrix; Determine whether the distance of the first paragraph is less than the distance threshold; If the distance between the first paragraphs is less than the distance threshold, then a connection edge is constructed between the first intermediate node and the second intermediate node.

3. The text comparison method according to claim 2, wherein, In the step of solving the cost-capacity network according to the maximum flow minimum cost algorithm to obtain several sets of paired segments, the step of solving a single set of paired segments includes: Determine whether the several second intermediate nodes connected to the first intermediate node have been paired; If none of the several second intermediate nodes connected to the first intermediate node are paired, the total cost of the feasible flow corresponding to the first intermediate node is calculated, and the second intermediate node corresponding to the feasible flow with the smallest total cost is selected as the pairing node of the first intermediate node. The paired paragraph set is constructed using the first paragraph corresponding to the first intermediate node and the second paragraph corresponding to the paired node.

4. The text comparison method according to claim 1, wherein, After the step of selecting several first and second short sentences that do not have a pairing relationship from each of the paired paragraph sets as distinguishing texts, the following steps are included: The distinguishing text is highlighted in both the first and second texts to be compared.

5. A text comparison device, wherein, include: The first calculation module is used to calculate the paragraph distance between each first paragraph of the first text to be compared and each second paragraph of the second text to be compared, wherein each paragraph distance corresponds to a single first paragraph and a single second paragraph; The first pairing module is used to determine the pairing relationship between each first paragraph and each second paragraph based on the maximum flow minimum cost algorithm and the distance between each paragraph, so as to obtain a number of paired paragraph sets, wherein a single paired paragraph set contains a first paired paragraph and a second paired paragraph that have a pairing relationship; The second calculation module is used to calculate the sentence distance between each first sentence of the first paired paragraph and each second sentence of the second paired paragraph in each of the paired paragraph sets, wherein each sentence distance corresponds to a single first sentence and a single second sentence; The second pairing module is used to determine the pairing relationship between each first short phrase and each second short phrase based on the maximum flow minimum cost algorithm and the distance between each phrase; The filtering module is used to filter out several first and second short sentences that do not have a pairing relationship from each of the paired paragraph sets as distinguishing text; The first computing module includes: A calculation unit is configured to group a single first paragraph and a single second paragraph together, and use the string edit distance of the first sentence text of each of the two paragraphs in the same group as the paragraph distance; The first construction unit is configured to construct a paragraph matrix based on the number of first paragraphs of the first text to be compared, the number of second paragraphs of the second text to be compared, and the distance between each paragraph. The size of the paragraph matrix is ​​N×M, where N represents the number of first paragraphs, M represents the number of second paragraphs, and the elements of the paragraph matrix represent the paragraph distance between the corresponding paragraphs of the first text to be compared and the second text to be compared. The first pairing module includes: The second construction unit is used to construct virtual source points and virtual sink points as the left and right endpoints of the cost capacity network, and to construct intermediate nodes of the cost capacity network according to the first number of segments and the second number of segments. The first intermediate node corresponding to the first number of segments is close to the virtual source point, and the second intermediate node corresponding to the second number of segments is close to the virtual sink point. The virtual source point establishes connection edges with each of the first intermediate nodes, and the virtual sink point establishes connection edges with each of the second intermediate nodes. A unit is established to retrieve a distance threshold and establish a connection edge between each first intermediate node and each second intermediate node according to the paragraph matrix and the distance threshold. The edge capacity of the connection edge in the cost-capacity network is 1, and the transportation cost of the connection edge is the paragraph distance between the first intermediate node and the second intermediate node at both ends of the connection edge. The solution unit is used to solve the cost-capacity network according to the maximum flow minimum cost algorithm to obtain a set of paired segments.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein... When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1-4.

Images