Methods, systems, media, and computer program products for text alignment

Abstract

The present application relates to the field of data processing, and more particularly to a method, system, computer storage medium and computer program product for text comparison. One aspect of the present application provides a method for text comparison, comprising: taking a reference text as a reference, determining an edit distance similarity between each to-be-compared text and the reference text based on at least an edit distance of each to-be-compared text relative to the reference text; determining a target matching text in the one or more to-be-compared texts matching the reference text based on the edit distance similarity between each to-be-compared text and the reference text using a double-layer screening condition associated with the edit distance similarity; generating a semantic judgment result based on a text similarity between the reference text and the target matching text; and outputting the semantic judgment result and an edit operation sequence of the target matching text relative to the reference text.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically to methods, systems, computer storage media, and computer program products for text comparison. Background Technology

[0002] With the rapid development of information technology, the generation, flow, and iteration of text data are becoming increasingly frequent. Text comparison has become an important tool in many fields such as information system compliance, risk management, and document management. For example, in information system compliance and risk management scenarios, textual normative documents such as laws and regulations, industry standards, and internal rules are constantly being updated, revised, or repealed. Accurately matching the new version associated with the old version and identifying the differences between the old and new versions is of great significance for ensuring the continuous compliance of information systems and reducing legal risks.

[0003] Currently, text comparison typically utilizes the longest common subsequence (LCS) algorithm and hash-based comparison algorithms for matching and comparing text. However, the LCS algorithm can only identify common subsequences within text, failing to recognize the specific differences between these subsequences. Hash-based comparison algorithms generate hash values ​​by hashing the text and compare these hash values ​​to determine text similarity, but they cannot identify differences at the text-level. Furthermore, current comparison algorithms lack effective filtering mechanisms, often relying solely on simple sorting based on similarity, leading to matching errors and omissions. Summary of the Invention

[0004] To address or at least alleviate one or more of the above problems, the following technical solutions are provided.

[0005] According to a first aspect of this application, a method for text comparison is provided, the method comprising the following steps: using a reference text as a reference, determining an edit distance similarity between each text to be compared and the reference text based at least on the edit distance of each text to be compared relative to the reference text; using a two-layer filtering condition associated with the edit distance similarity, determining a target matching text among the one or more texts to be compared that matches the reference text based on the edit distance similarity between each text to be compared and the reference text; generating a semantic judgment result based on the text similarity between the reference text and the target matching text; and outputting the semantic judgment result and a sequence of editing operations of the target matching text relative to the reference text, wherein the sequence of editing operations includes one or more of a replacement operation, a deletion operation, and an insertion operation of the target matching text relative to the reference text.

[0006] According to an embodiment of the text comparison method of this application, determining the edit distance similarity between each text to be compared and the reference text based at least on the edit distance of each text to be compared relative to the reference text includes: determining the total number of characters in the reference text; determining the ratio of the edit distance to the total number of characters in the reference text; and determining the edit distance similarity between each text to be compared and the reference text based at least on the ratio of the edit distance to the total number of characters in the reference text.

[0007] According to the text comparison method of one embodiment or any of the above embodiments of this application, the edit distance of each text to be compared relative to the reference text is determined by the following method: constructing an edit distance matrix between the reference text and each text to be compared; assigning values ​​to each element in the edit distance matrix according to a preset assignment rule; and determining the value of the lower right corner element in the edit distance matrix after assignment as the edit distance.

[0008] According to the text comparison method of one embodiment or any of the above embodiments of this application, the preset assignment rule includes: assigning each element of the first row of the edit distance matrix to the corresponding column number and assigning each element of the first column of the edit distance matrix to the corresponding row number to initialize the boundary of the edit distance matrix; traversing the internal elements of the edit distance matrix that have not been initialized by boundary in the order from top to bottom and from left to right, judging one by one whether the reference text content corresponding to the current position is consistent with the text content to be compared, and assigning values ​​to the internal elements that have not been initialized by boundary according to the judgment result.

[0009] According to one embodiment or any of the above embodiments of the text comparison method described in this application, the two-layer filtering conditions associated with the edit distance similarity include: a first-layer filtering condition where the edit distance similarity is greater than a similarity threshold; and a second-layer filtering condition where the word error rate is minimized.

[0010] According to one embodiment or any of the above embodiments of the present application, the method for text comparison, wherein determining the target matching text among the one or more texts to be compared based on the edit distance similarity between each text to be compared and the reference text, using a two-layer filtering condition associated with the edit distance similarity, comprises: determining a set of candidate matching texts from the one or more texts to be compared based on the edit distance similarity between each text to be compared and the reference text using the first layer filtering condition; and determining the candidate matching text with the smallest word error rate in the set of candidate matching texts as the target matching text using the second layer filtering condition.

[0011] According to one embodiment or any of the above embodiments of the text comparison method of this application, generating a semantic judgment result based on the text similarity between the reference text and the target matching text includes: comparing the text similarity with a similarity threshold; generating a semantic judgment result in which the reference text and the target matching text have the same semantics in response to the text similarity being greater than or equal to the similarity threshold; and generating a semantic judgment result in which the reference text and the target matching text have different semantics in response to the text similarity being less than the similarity threshold.

[0012] According to the text comparison method of one embodiment or any of the above embodiments of this application, the text similarity between the reference text and the target matching text is determined by: determining the feature vector of the reference text and the feature vector of the target matching text; and determining the text similarity based on the feature vector of the reference text and the feature vector of the target matching text.

[0013] According to the text comparison method of one embodiment or any of the above embodiments of this application, the editing operation sequence of the target matching text relative to the reference text is determined by the following method: performing a reverse backtracking on the edit distance matrix after the assignment between the reference text and the target matching text is completed, backtracking from the lower right element to the upper left element; during the reverse backtracking process, comparing the current element with adjacent elements numerically, determining the operation type based on whether the source direction of the current element relative to the adjacent elements and the corresponding text content are consistent, the operation type including replacement, deletion and insertion; and determining the editing operation sequence of the target matching text relative to the reference text based on the operation type.

[0014] According to the text comparison method of one embodiment or any of the above embodiments of this application, in the reverse backtracking process, the current element is compared with the adjacent elements in terms of value. The operation type is determined based on whether the source direction of the current element relative to the adjacent elements and the corresponding text content are consistent. The operation type includes: if the source direction is the top-left diagonal adjacent element, the corresponding text content is inconsistent, and the value of the current element is 1 greater than the value of the top-left diagonal adjacent element, then it is a replacement operation; if the source direction is the top-up adjacent element, and the value of the current element is 1 greater than the value of the top-up adjacent element, then it is a deletion operation; if the source direction is the left-right adjacent element, and the value of the current element is 1 greater than the value of the left-right adjacent element, then it is an insertion operation.

[0015] According to a second aspect of this application, a text comparison system is provided, the system comprising: a memory; a processor coupled to the memory; and a computer program stored on the memory and running on the processor, the execution of the computer program causing the execution of the steps of the text comparison method according to a first aspect of this application.

[0016] According to a third aspect of this application, a computer storage medium is provided, comprising instructions that, when executed, perform the steps of the text comparison method according to a first aspect of this application.

[0017] According to a fourth aspect of this application, a computer program product is provided, the computer program product including instructions that, when executed by a processor, implement the steps of the text comparison method according to a first aspect of this application.

[0018] The text comparison scheme according to one or more embodiments of this application, by utilizing a two-layer filtering condition associated with edit distance similarity, can accurately identify target matching text that matches the baseline text, effectively reducing matching errors and omissions, and improving text matching accuracy. Simultaneously, the text comparison scheme according to one or more embodiments of this application, by outputting the semantic judgment results between the baseline text and the target matching text, can effectively distinguish between literal wording adjustments and substantive content changes. By outputting the editing operation sequence of the target matching text relative to the baseline text, it can precisely locate the specific location and content of text differences, accurately reflecting the details of text differences and assisting users in efficiently locating text differences. Attached Figure Description

[0019] The above and / or other aspects and advantages of this application will become clearer and more readily understood from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are denoted by the same reference numerals. The drawings include: Figure 1 A flowchart of a text comparison method according to one or more embodiments of this application is shown.

[0020] Figure 2 A flowchart illustrating a method for comparing old and new laws and regulations according to an embodiment of this application is shown.

[0021] Figure 3 A block diagram of a text comparison system according to one or more embodiments of this application is shown. Detailed Implementation

[0022] The present application will now be described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments thereof. However, the present application may be implemented in various forms and should not be construed as being limited to the embodiments given herein. The foregoing embodiments are intended to make the disclosure herein complete and thorough, so as to more fully convey the scope of protection of the present application to those skilled in the art.

[0023] In this specification, terms such as “comprising” and “including” indicate that, in addition to having the units and steps that are directly and explicitly stated in the specification and claims, the technical solution of this application does not exclude the presence of other units and steps that are not directly or explicitly stated.

[0024] Unless otherwise specified, terms such as “first” and “second” do not indicate the order of units in terms of time, space, size, etc., but are merely used to distinguish between units.

[0025] In the following, various exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings.

[0026] Figure 1 A flowchart of a text comparison method according to one or more embodiments of this application is shown.

[0027] like Figure 1 As shown, in step S101, with reference to the reference text, the edit distance similarity between each text to be compared and the reference text is determined based on at least the edit distance of each text to be compared relative to the reference text in one or more texts to be compared.

[0028] Optionally, in step S101, the total number of characters in the reference text can be determined, the ratio of the edit distance of each text to be compared relative to the reference text to the total number of characters in the reference text can be determined, and the edit distance similarity between each text to be compared and the reference text can be determined at least based on the ratio of the edit distance to the total number of characters in the reference text. It should be noted that the edit distance refers to the minimum number of editing operations required to convert one string into another, such as inserting characters, deleting characters, and replacing characters. The edit distance of the text to be compared relative to the reference text refers to the minimum total number of inserting, deleting, and replacing character operations required to convert the reference text into the text to be compared.

[0029] In one embodiment, the character error rate (CER) between the text to be compared and the reference text can be determined by the following formula (1): CER=(S+D+I) / N formula (1) Where CER represents the word error rate between the text to be compared and the reference text, N represents the total number of characters in the reference text, S represents the number of character substitution operations required to convert the reference text to the text to be compared, D represents the number of character deletion operations required to convert the reference text to the text to be compared, and I represents the number of character insertion operations required to convert the reference text to the text to be compared. For example, N can be the total number of characters in the reference text after removing redundant spaces, and punctuation marks can be selected as needed.

[0030] In one embodiment, the edit distance similarity between the text to be compared and the reference text can be determined by the following formula (2): Similarity = 1 - (S + D + I) / N (Formula 2) Where N represents the total number of characters in the reference text, S represents the number of character substitution operations required to convert the reference text into the comparison text, D represents the number of character deletion operations required to convert the reference text into the comparison text, and I represents the number of character insertion operations required to convert the reference text into the comparison text.

[0031] In one embodiment, an edit distance matrix can be constructed between the reference text and each text to be compared. Values ​​can be assigned to each element in the edit distance matrix according to a preset assignment rule. The value of the lower right element in the edit distance matrix after assignment can be determined as the edit distance of each text to be compared relative to the reference text.

[0032] In one embodiment, the preset assignment rule may include: assigning each element of the first row of the edit distance matrix to the corresponding column index and assigning each element of the first column of the edit distance matrix to the corresponding row index to initialize the boundary of the edit distance matrix; traversing the internal elements of the edit distance matrix that have not been initialized by boundary in a top-to-bottom and left-to-right order, determining one by one whether the reference text content corresponding to the current position is consistent with the text content to be compared, and assigning values ​​to the internal elements that have not been initialized by boundary according to the determination result. In one embodiment, if the reference text content corresponding to the current position is consistent with the text content to be compared, the value of the current element is equal to the value of its upper-left diagonal neighbor; if the reference text content corresponding to the current position is inconsistent with the text content to be compared, the minimum value among the upper-left diagonal neighbor, the upper neighbor, and the left neighbor of the current element is taken, and the minimum value is incremented by one as the value of the current element.

[0033] For example, assuming the base text is XYZ and the text to be compared is XMZW, the following edit distance matrix can be obtained after assignment using the above preset assignment rules: Among them, the value of the lower right element in the edit distance matrix after assignment can be determined as the edit distance of the text to be compared relative to the reference text, which is 2.

[0034] In step S103, using a two-layer filtering condition associated with edit distance similarity, the target matching text among one or more comparison texts that match the reference text is determined based on the edit distance similarity between each comparison text and the reference text.

[0035] In one embodiment, the two-layer filtering criteria associated with edit distance similarity may include: a first-layer filtering criterion where the edit distance similarity is greater than a similarity threshold, and a second-layer filtering criterion where the character error rate (CER) is minimized. For example, the similarity threshold may be set to 80% or a value around 80%. In one embodiment, the character error rate (CER) is the ratio of the total number of character-level edit operations on the text to be compared to the reference text to the total number of characters in the reference text (CER = (S + D + I) / N), which is also equal to 1 minus the edit distance similarity. In other words, the mathematical expression of the above two-layer filtering criteria is: the edit distance similarity (1 - CER) is greater than the similarity threshold (e.g., 80%), and the character error rate (CER) is minimized, i.e., (1 - CER) > similarity threshold and min(CER). Specifically, the first-layer filtering criterion is used to filter out candidate matching texts with edit distance similarity higher than the similarity threshold, and then the second-layer filtering criterion is used to select the candidate matching text with the minimum character error rate (CER) from the candidate matching text set as the target matching text.

[0036] Optionally, in step S103, a first-level filtering condition can be used to determine a set of candidate matching texts from one or more texts to be compared based on the edit distance similarity between each text to be compared and the reference text; and a second-level filtering condition can be used to determine the candidate matching text with the smallest word error rate (CER) in the candidate matching text set as the target matching text. For example, assuming A is the baseline text, and B1, B2, and B3 are the texts to be compared, assuming the edit distance similarity between B1 and A is 94%, the edit distance similarity between B2 and A is 83%, and the edit distance similarity between B3 and A is 20%, and the similarity threshold is set to 80%, the first layer of filtering conditions can be used to determine the candidate matching text set {B1, B2} from the texts to be compared B1, B2, and B3 with an edit distance similarity greater than the similarity threshold of 80%, where B1 has a word error rate (CER) of 6% and B2 has a word error rate (CER) of 17%. The second layer of filtering conditions can be used to determine the candidate matching text B1 with the smallest word error rate (CER) in the candidate matching text set {B1, B2} as the target matching text.

[0037] By setting a first-level filtering condition where the edit distance similarity is greater than the similarity threshold, irrelevant texts that do not have a structural inheritance relationship with the benchmark text can be quickly filtered out from a large number of texts to be compared. This effectively avoids the problem of incorrect pairing caused by ranking solely based on similarity and ensures that the candidate matching text set entering the subsequent comparison stage is relevant to the benchmark text. By setting a second-level filtering condition where the word error rate (CER) is minimized, the matching text with the smallest revision can be further filtered out after confirming the relevance, ensuring that the target matching text is the closest matching text to the benchmark text.

[0038] In step S105, a semantic judgment result is generated based on the text similarity between the baseline text and the target matching text.

[0039] Optionally, in step S105, the feature vectors of the reference text and the target matching text can be determined, and the text similarity can be determined based on the feature vectors of the reference text and the target matching text. In one embodiment, the reference text and the target matching text can be respectively input into a pre-trained semantic vectorization model (e.g., BERT, RoBERTa, etc.) for feature extraction to obtain their respective feature vectors. In another embodiment, cosine similarity or Euclidean distance can be determined based on the feature vectors of the reference text and the target matching text to characterize the text similarity, wherein a larger cosine similarity value and a smaller Euclidean distance value represent a higher similarity between the two texts.

[0040] Optionally, in step S105, the text similarity between the reference text and the target matching text can be compared with a similarity threshold. When the text similarity is greater than or equal to the similarity threshold, a semantic judgment result can be generated that the reference text and the target matching text are semantically identical; when the text similarity is less than the similarity threshold, a semantic judgment result can be generated that the reference text and the target matching text are semantically different. For example, the similarity threshold can be set to 0.8 or a value around 0.8.

[0041] In one embodiment, the semantic judgment result can include semantically identical, semantically similar, and semantically different. The similarity threshold can be set to multiple values ​​accordingly. For example, when the text similarity is greater than or equal to 0.85, a semantic judgment result can be generated that the base text and the target matching text are semantically identical; when the text similarity is greater than or equal to 0.7 and less than 0.85, a semantic judgment result can be generated that the base text and the target matching text are semantically similar; when the text similarity is less than 0.7, a semantic judgment result can be generated that the base text and the target matching text are semantically different.

[0042] In step S107, the semantic judgment result and the editing operation sequence of the target matching text relative to the reference text are output, wherein the editing operation sequence includes one or more of the replacement operation, deletion operation and insertion operation of the target matching text relative to the reference text.

[0043] In one embodiment, the edit distance matrix between the base text and the target matching text after assignment can be backtracked, from the bottom right element to the top left element. During the backtracking process, the current element is compared with its neighboring elements. Based on the source direction of the current element relative to its neighboring elements and whether the corresponding text content is consistent, the operation type is determined. This operation type includes replacement, deletion, and insertion. Based on the determined operation type, the editing operation sequence of the target matching text relative to the base text is determined. It should be noted that the editing operation sequence of the target matching text relative to the base text refers to the ordered set of replacement, deletion, and insertion operations performed sequentially when converting the base text character by character to the target matching text. This sequence is obtained by backtracking the already assigned edit distance matrix, completely recording the form and corresponding position of every character-level change between the two text segments.

[0044] In one embodiment, during the reverse backtracking process, the current element is compared numerically with its adjacent elements. If the source direction is the top-left diagonal adjacent element, the corresponding text content is inconsistent, and the current element's value is 1 greater than the top-left diagonal adjacent element's value, then it is a replacement operation. If the source direction is the directly above adjacent element, and the current element's value is 1 greater than the directly above adjacent element's value, then it is a deletion operation. If the source direction is the directly left adjacent element, and the current element's value is 1 greater than the directly left adjacent element's value, then it is an insertion operation. For example, taking the edit distance matrix after the above assignment as an example, through reverse backtracking, it can be determined that the editing operation sequence of text XMZW relative to text XYZ is to replace character Y with character M and insert W at the end.

[0045] It should be noted that the text comparison method according to one or more embodiments of this application can be applied not only to scenarios involving version updates, revisions, and repeals of laws, regulations, industry standards, and internal rules for matching and analyzing differences between old and new texts, but also to scenarios involving matching and analyzing differences between old and new texts, such as software engineering and code version management, financial and auditing document management, intellectual property and patent management, and educational publishing and academic literature. The following will combine... Figure 2 This application describes a process for comparing new and old laws and regulations according to one embodiment of the present application.

[0046] The text comparison method according to one or more embodiments of this application, by utilizing a two-layer filtering condition associated with edit distance similarity, can accurately identify target matching text that matches the baseline text, effectively reducing matching errors and omissions, and improving text matching accuracy. Simultaneously, the text comparison method according to one or more embodiments of this application, by outputting the semantic judgment result between the baseline text and the target matching text, can effectively distinguish between literal wording adjustments and substantive content changes. By outputting the editing operation sequence of the target matching text relative to the baseline text, it can precisely locate the specific location and content of text differences, accurately reflecting the details of text differences and assisting users in efficiently locating text differences.

[0047] Figure 2 A flowchart illustrating a method for comparing old and new laws and regulations according to an embodiment of this application is shown.

[0048] It should be noted that, in this embodiment, the old legal provision corresponds to the aforementioned benchmark text, and the new legal provision corresponds to the aforementioned text to be compared. For example... Figure 2 As shown, in step S201, with reference to the old legal provisions, the edit distance similarity between each new legal provision and the old legal provisions is determined based at least on the edit distance of each new legal provision in one or more new legal provisions relative to the old legal provisions.

[0049] Optionally, in step S201, the total number of characters in the old legal provisions can be determined, the ratio of the edit distance of each new legal provision relative to the old legal provisions to the total number of characters in the old legal provisions can be determined, and the word error rate (CER) between each new legal provision and the old legal provisions can be determined at least based on the ratio of the edit distance to the total number of characters in the old legal provisions, and the edit distance similarity can be determined based on the word error rate (CER). The edit distance of the new legal provisions relative to the old legal provisions refers to the minimum total number of insertion, deletion, and replacement character operations required to convert the old legal provisions into new legal provisions. In one embodiment, the word error rate (CER) between the new legal provisions and the old legal provisions can be determined using the above formula (1), and the edit distance similarity between the new legal provisions and the old legal provisions can be determined using the above formula (2).

[0050] In step S203, using a two-layer filtering condition associated with edit distance similarity, the target matching legal provision among the new legal provisions that match the old legal provisions is determined based on the edit distance similarity between the new legal provisions and the old legal provisions.

[0051] In one embodiment, the two-layer filtering condition associated with edit distance similarity may include: a first-layer filtering condition where the edit distance similarity is greater than a similarity threshold, and a second-layer filtering condition where the character error rate (CER) is minimized. For example, the similarity threshold may be set to 80% or a value around 80%. In one embodiment, the character error rate (CER) is the ratio of the total number of character-level editing operations of the new rule relative to the old rule to the total number of characters in the old rule (CER = (S + D + I) / N), which is also equal to 1 minus the edit distance similarity. In other words, the mathematical expression of the two-layer filtering condition is: the edit distance similarity (1 - CER) is greater than the similarity threshold (e.g., 80%), and the character error rate (CER) is minimized, i.e., (1 - CER) > similarity threshold and min(CER). Specifically, the first-layer filtering condition is used to filter out a set of candidate matching rules with an edit distance similarity higher than the similarity threshold, and then the second-layer filtering condition is used to select the candidate matching rule with the minimum character error rate (CER) from the set of candidate matching rules as the target matching rule.

[0052] Optionally, in step S203, the first layer of filtering conditions can be used to determine a set of candidate matching legal provisions from one or more new legal provisions based on the edit distance similarity between each new legal provision and the old legal provisions; and the second layer of filtering conditions can be used to determine the candidate matching legal provision with the smallest word error rate (CER) in the set of candidate matching legal provisions as the target matching legal provision.

[0053] In step S205, a semantic judgment result is generated based on the textual similarity between the old legal provision and the target matching legal provision.

[0054] Optionally, in step S205, the text similarity between the old legal provision and the target matching legal provision can be compared with a similarity threshold. When the text similarity is greater than or equal to the similarity threshold, a semantic judgment result can be generated that the old legal provision and the target matching legal provision are semantically the same; when the text similarity is less than the similarity threshold, a semantic judgment result can be generated that the old legal provision and the target matching legal provision are semantically different. For example, the similarity threshold can be set to 0.8 or a value around 0.8.

[0055] In one embodiment, the semantic judgment result can include semantically identical, semantically similar, and semantically different. The similarity threshold can be set to multiple values ​​accordingly. For example, when the text similarity is greater than or equal to 0.85, a semantic judgment result can be generated that the old legal provision and the target matching legal provision are semantically identical; when the text similarity is greater than or equal to 0.7 and less than 0.85, a semantic judgment result can be generated that the old legal provision and the target matching legal provision are semantically similar; when the text similarity is less than 0.7, a semantic judgment result can be generated that the old legal provision and the target matching legal provision are semantically different.

[0056] In step S207, the semantic judgment result and the editing operation sequence of the target matching legal provision relative to the old legal provision are output.

[0057] For example, the old legal provision could be: "Article 1. This method is formulated in accordance with the 'Emergency Response Law of the People's Republic of China' and other laws and administrative regulations to regulate the management of emergency response plans (hereinafter referred to as emergency plans) and enhance their pertinence, practicality, and operability." The new legal provision matching the old provision (i.e., the target-matching legal provision determined in step S203) could be: "Article 1. This method is formulated in accordance with the 'Emergency Response Law of the People's Republic of China' and other laws and administrative regulations to strengthen the construction of the emergency response plan system (hereinafter referred to as emergency plans), regulate emergency plan management, and enhance the pertinence, practicality, and operability of emergency plans." The output semantic judgment result... If the result can be "semantic approximation", the editing operation sequence of the output target matching legal provision relative to the old legal provision can be "Article 1 is (Standardize->Strengthen) Emergency Response Plan for Sudden Events (hereinafter referred to as Emergency Response Plan) (*->System Construction, Standardize Emergency Response Plan) Management, to enhance the pertinence, practicality and operability of the Emergency Response Plan, in accordance with the "Emergency Response Law of the People's Republic of China" and other laws and administrative regulations, this method is (Formulate->Determine)", where "Standardize->Strengthen" means replacing "Standardize" with "Strengthen", "Formulate->Determine" means replacing "Formulate" with "Determine", and "*->System Construction, Standardize Emergency Response Plan)" means inserting "System Construction, Standardize Emergency Response Plan".

[0058] The method for comparing old and new laws and regulations according to one or more embodiments of this application, by utilizing a two-layer filtering condition associated with edit distance similarity, can accurately identify new legal provisions that match the old ones, effectively reducing matching errors and omissions, and improving matching accuracy. Simultaneously, by outputting the semantic judgment results between the old and matched new legal provisions, the method can effectively distinguish between literal wording adjustments and substantive content changes. By outputting the editing operation sequence of the matched new legal provisions relative to the old ones, it can precisely locate the specific location and content of textual differences, accurately reflecting the details of textual differences and assisting users in efficiently locating the differences between old and new laws and regulations.

[0059] Figure 3 A block diagram of a text comparison system according to one or more embodiments of this application is shown.

[0060] like Figure 3 As shown, the text comparison system 300 includes a memory 310, a processor 320, and a computer program 330 stored in the memory 310 and executable on the processor 320. The processor 320 executes the computer program 330 to implement a text comparison method according to one aspect of this application.

[0061] This application can also be implemented as a computer-readable storage medium including instructions that, when executed, perform a text comparison method according to one aspect of this application.

[0062] Where applicable, the various embodiments provided in this application may be implemented using hardware, software, or a combination of hardware and software. Furthermore, where applicable, without departing from the scope of this application, the various hardware and / or software components described herein may be combined into composite components comprising software, hardware, and / or both. Where applicable, without departing from the scope of this application, the various hardware and / or software components described herein may be divided into sub-components comprising software, hardware, or both. Additionally, where applicable, it is contemplated that software components may be implemented as hardware components, and vice versa.

[0063] The software (such as program code and / or data) according to this application can be stored on one or more computer storage media. It is also contemplated that the software identified herein can be implemented using one or more networked and / or otherwise general-purpose or special-purpose computers and / or computer systems. Where applicable, the order of the various steps described herein can be changed, combined into compound steps, and / or divided into sub-steps to provide the features described herein.

[0064] The embodiments and examples presented herein are provided to best illustrate embodiments of this application and its particular applications, thereby enabling those skilled in the art to implement and use this application. However, those skilled in the art will understand that the above description and examples are provided for ease of illustration and example only. The descriptions presented are not intended to cover all aspects of this application or to limit this application to the precise forms disclosed.

Claims

1. A method for text comparison, characterized in that, The method includes the following steps: Using a reference text, at least based on the edit distance of each text to be compared relative to the reference text, the edit distance similarity between each text to be compared and the reference text is determined; Using a two-layer filtering condition associated with the edit distance similarity, the target matching text among the one or more matching texts that match the benchmark text is determined based on the edit distance similarity between each text to be compared and the benchmark text; Generate a semantic judgment result based on the text similarity between the baseline text and the target matching text; and Output the semantic judgment result and the editing operation sequence of the target matching text relative to the reference text, wherein the editing operation sequence includes one or more of the following: replacement operation, deletion operation, and insertion operation of the target matching text relative to the reference text. The two-layer filtering conditions associated with the edit distance similarity include: The first-level filtering condition where the edit distance similarity is greater than the similarity threshold; and The second layer of filtering criteria with the lowest character error rate. The method of determining the target matching text among the one or more comparison texts that match the benchmark text based on the edit distance similarity between each comparison text and the benchmark text, using a two-layer filtering condition associated with the edit distance similarity, includes: Using the first layer of filtering conditions, a set of candidate matching texts is determined from the one or more texts to be compared based on the edit distance similarity between each text to be compared and the reference text; and Using the second layer of filtering conditions, the candidate matching text with the lowest word error rate in the candidate matching text set is determined as the target matching text.

2. The method of claim 1, wherein determining the edit distance similarity between each text to be compared and the reference text, based at least on the edit distance of each text to be compared relative to the reference text, comprises: Determine the total number of characters in the baseline text; Determine the ratio of the edit distance to the total number of characters in the reference text; as well as The edit distance similarity between each text to be compared and the reference text is determined based at least on the ratio of the edit distance to the total number of characters in the reference text.

3. The method of claim 1, wherein the edit distance of each text to be compared relative to the reference text is determined by: Construct the edit distance matrix between the baseline text and each text to be compared; The elements in the edit distance matrix are assigned values ​​according to a preset assignment rule; and The value of the bottom right element in the edit distance matrix after assignment is determined as the edit distance.

4. The method according to claim 3, wherein the preset assignment rule includes: The elements of the first row of the edit distance matrix are assigned the corresponding column index, and the elements of the first column of the edit distance matrix are assigned the corresponding row index to initialize the boundary of the edit distance matrix. Traverse the internal elements of the edit distance matrix that have not been initialized by boundary in the order from top to bottom and from left to right, determine one by one whether the reference text content corresponding to the current position is consistent with the text content to be compared, and assign values ​​to the internal elements that have not been initialized by boundary according to the determination results.

5. The method according to claim 1, wherein generating a semantic judgment result based on the text similarity between the reference text and the target matching text includes: Compare the text similarity with the similarity threshold; In response to the text similarity being greater than or equal to the similarity threshold, a semantic judgment result is generated indicating that the base text and the target matching text are semantically identical. In response to the text similarity being less than the similarity threshold, a semantic judgment result is generated indicating that the base text and the target matching text are semantically different.

6. The method of claim 1, wherein the text similarity between the reference text and the target matching text is determined by the following method: Determine the feature vector of the reference text and the feature vector of the target matching text; and The text similarity is determined based on the feature vectors of the baseline text and the feature vectors of the target matching text.

7. The method of claim 3, wherein the sequence of editing operations on the target matching text relative to the reference text is determined by: After the assignment between the baseline text and the target matching text is completed, the edit distance matrix is ​​backtracked in reverse, from the bottom right element to the top left element; During the reverse backtracking process, the current element is numerically compared with its adjacent elements. Based on the source direction of the current element relative to its adjacent elements and whether the corresponding text content is consistent, the operation type is determined. The operation types include replacement, deletion, and insertion. Based on the operation type, determine the sequence of editing operations for the target matching text relative to the reference text.

8. The method according to claim 7, wherein during the reverse backtracking process, the current element is numerically compared with adjacent elements, and the operation type is determined based on whether the current element's source direction relative to adjacent elements and the corresponding text content are consistent, including: If the source direction is the top-left diagonal adjacent element, the corresponding text content is inconsistent, and the value of the current element is 1 greater than the value of the top-left diagonal adjacent element, then it is a replacement operation; If the source direction is the adjacent element directly above, and the value of the current element is 1 greater than the value of the adjacent element directly above, then it is a deletion operation; If the source direction is the adjacent element to the left, and the value of the current element is 1 greater than the value of the adjacent element to the left, then it is an insertion operation.

9. A text comparison system, characterized in that, The system includes: Memory; A processor coupled to the memory; and A computer program stored on the memory and running on the processor, the execution of which causes the text comparison method according to any one of claims 1-8 to be performed.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed, perform the text comparison method according to any one of claims 1-8.

11. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a processor, implement the text comparison method according to any one of claims 1-8.

Images