Block-based text difference recognition method, device and computer equipment

Abstract

The present application relates to a block-based text difference recognition method, device and computer equipment. The method comprises: using the Myers algorithm to process the files to be compared to obtain the deleted, newly added and unchanged line sets, constructing the changed block set based on this and identifying the split and merged lines therein and then deleting them; traversing the deleted lines and unchanged lines based on the newly added lines, and forming a many-to-many mapping set containing corresponding line update operations according to the block movement and copy characteristics; traversing the changed block set and obtaining the final line update mapping pair through weighted calculation and loop processing; using the KM algorithm to perform multiple loop optimal matching on the above mapping set according to specific weight rules; and generating the final editing script with the split and merged mapping set, the final matching set and the unmatched lines. The present method can more accurately identify text differences.

Description

Technical Field

[0001] The present application relates to the technical field of text processing, and in particular to a block-based text difference recognition method, device and computer equipment. Background Art

[0002] With the rapid development of information technology, the importance of text data processing in many fields is increasing day by day, and text difference detection technology is also evolving. At present, this technology is widely used in many fields such as software development and document management. Its application scenarios are constantly expanding, and the requirements for technology are also increasing.

[0003] Most of the existing mainstream text difference detection technologies are based on line-level operations, which identify editing actions by comparing two text files line by line (such as different versions of the same file). In practical applications, such technologies are often integrated into version control systems and document editing software, and can meet basic text comparison needs to a certain extent, such as helping developers view the addition, deletion, and modification of code lines, and assisting document authors and collaborators in tracking revision records. However, this technology has obvious defects. On the one hand, actual editing is often based on statement blocks, such as moving or copying function modules in the code, but line-level technology cannot accurately identify block-level operations, and will split the processing, resulting in complicated results and loss of semantic logic information. On the other hand, it only compares at the line level and lacks the ability to understand the deep semantics and logical structure of the text. When processing documents containing formula references or function logical relationships in the code, which involve multiple lines of editing operations, it is impossible to accurately judge their impact, and it is difficult to meet the needs of modern complex text processing for high-precision and intelligent difference analysis. Summary of the invention

[0004] Based on this, it is necessary to provide a block-based text difference recognition method, device and computer equipment to address the above technical problems.

[0005] A block-based text difference recognition method, the method comprising:

[0006] Use the Git diff algorithm to process the two files to be compared to obtain the deleted row set, the newly added row set, and the unchanged row set;

[0007] Constructing a changed block set according to the deleted row set, the newly added row set and the changed block information;

[0008] Traversing the change block set, identifying the split and merge rows of the row split and row merge operations in the change block set, and deleting the identified split and merge rows from the change block set, the deleted row set, and the added row set;

[0009] Traversing the change block set to identify row update operations, calculating update similarity values ​​by combining context similarity and row similarity weighting method to obtain multiple pairs of row update mapping pairs, and determining a final row update mapping set by cyclically deleting mapping pairs containing the most intersections;

[0010] According to the newly added row set, the deleted rows are traversed, and according to the block movement information, all block movements are identified to form a many-to-many block movement mapping set including the row update operation after the block movement; according to the newly added row set, the deleted rows and the unchanged rows are traversed, and according to the block replication information, block replication is detected and a many-to-many block replication mapping set including the row update operation after the block replication is formed;

[0011] The weighted bipartite graph optimal matching KM algorithm is used to optimally match the row update mapping set, many-to-many block move mapping set and many-to-many block copy mapping set. The KM algorithm is cycled multiple times to process the remaining matching sets until the optimal matching set is obtained; the split and merged rows, the optimal matching set, and the unmatched deleted and added rows are taken as input, and the final editing script is generated according to the row order.

[0012] In one of the embodiments, the changed block information indicates a block consisting of consecutive row deletions and row additions.

[0013] In one embodiment, the block movement and block copy information indicates that the line position or indentation position of continuous text lines is moved or copied as a whole, and the relative indentation of each line in the block remains unchanged before and after the editing operation.

[0014] In one embodiment, the block copy information indicates that the line position or indentation position of the continuous text lines is copied as a whole, and the relative indentation of each line in the block remains unchanged before and after the copy.

[0015] In one embodiment, the method further includes: setting weights of the row update mapping set, the many-to-many block move mapping set, and the many-to-many block copy mapping set, and calculating edge weights according to the weights and the context similarity:

[0016] ;

[0017] Among them, MappingWeight represents mapping weight, EditWeight represents editing weight, BlockLength represents the number of block lines, and ContextSimilarity represents block context similarity;

[0018] The optimal matching is performed using the mapping weight as the KM algorithm weight value.

[0019] In one embodiment, the context similarity is calculated based on the ratio of equal lines of character strings in the four lines of character strings before and after the two blocks.

[0020] A block-based text difference recognition device, the device comprising:

[0021] The data processing module is used to process two files to be compared using the Git diff algorithm to obtain a deleted row set, a newly added row set and an unchanged row set; construct a changed block set according to the deleted row set, the newly added row set and the changed block information; traverse the changed block set, identify the split and merged rows of the row split and row merge operations in the changed block set, and delete the identified split and merged rows from the changed block set, the deleted row set and the newly added row set;

[0022] A mapping module, traversing the changed block set to identify row update operations, calculating update similarity values ​​by combining context similarity and row similarity weighting method to obtain multiple-to-multi row update mapping pairs, and determining a final row update mapping set by cyclically deleting mapping pairs containing the most intersections; traversing deleted rows according to the newly added row set, identifying all block movements according to block movement information to form a multiple-to-multi block movement mapping set containing row update operations after block movement; traversing deleted rows and unchanged rows according to the newly added row set, detecting block replication according to block replication information and forming a multiple-to-multi block replication mapping set containing row update operations after block replication;

[0023] The matching module is used to use the weighted bipartite graph optimal matching KM algorithm to optimally match the row update mapping set, the many-to-many block move mapping set and the many-to-many block copy mapping set, and to process the remaining matching sets through multiple cycles of the KM algorithm until the optimal matching set is obtained; the split and merged rows, the optimal matching set, and the unmatched deleted and added rows are taken as input, and the final editing script is generated according to the row order.

[0024] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0025] Use the Git diff algorithm to process the two files to be compared to obtain the deleted row set, the newly added row set, and the unchanged row set;

[0026] Constructing a changed block set according to the deleted row set, the newly added row set and the changed block information;

[0027] Traversing the change block set, identifying the split and merge rows of the row split and row merge operations in the change block set, and deleting the identified split and merge rows from the change block set, the deleted row set, and the added row set;

[0028] Traversing the change block set to identify row update operations, calculating update similarity values ​​by combining context similarity and row similarity weighting method to obtain multiple pairs of row update mapping pairs, and determining a final row update mapping set by cyclically deleting mapping pairs containing the most intersections;

[0029] According to the newly added row set, the deleted rows are traversed, and according to the block movement information, all block movements are identified to form a many-to-many block movement mapping set including the row update operation after the block movement; according to the newly added row set, the deleted rows and the unchanged rows are traversed, and according to the block replication information, block replication is detected and a many-to-many block replication mapping set including the row update operation after the block replication is formed;

[0030] The weighted bipartite graph optimal matching KM algorithm is used to optimally match the row update mapping set, many-to-many block move mapping set and many-to-many block copy mapping set. The KM algorithm is cycled multiple times to process the remaining matching sets until the optimal matching set is obtained; the split and merged rows, the optimal matching set, and the unmatched deleted and added rows are taken as input, and the final editing script is generated according to the row order.

[0031] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0032] Use the Git diff algorithm to process the two files to be compared to obtain the deleted row set, the newly added row set, and the unchanged row set;

[0033] Constructing a changed block set according to the deleted row set, the newly added row set and the changed block information;

[0034] Traversing the change block set, identifying the split and merge rows of the row split and row merge operations in the change block set, and deleting the identified split and merge rows from the change block set, the deleted row set, and the added row set;

[0035] Traversing the change block set to identify row update operations, calculating update similarity values ​​by combining context similarity and row similarity weighting method to obtain multiple pairs of row update mapping pairs, and determining a final row update mapping set by cyclically deleting mapping pairs containing the most intersections;

[0036] According to the newly added row set, the deleted rows are traversed, and according to the block movement information, all block movements are identified to form a many-to-many block movement mapping set including the row update operation after the block movement; according to the newly added row set, the deleted rows and the unchanged rows are traversed, and according to the block replication information, block replication is detected and a many-to-many block replication mapping set including the row update operation after the block replication is formed;

[0037] The weighted bipartite graph optimal matching KM algorithm is used to optimally match the row update mapping set, many-to-many block move mapping set and many-to-many block copy mapping set. The KM algorithm is cycled multiple times to process the remaining matching sets until the optimal matching set is obtained; the split and merged rows, the optimal matching set, and the unmatched deleted and added rows are taken as input, and the final editing script is generated according to the row order.

[0038] The block-based text difference recognition method, device, computer equipment and storage medium first use the Gitdiff algorithm to obtain the deleted, added and unchanged line sets, which provide basic data for subsequent analysis, help to preliminarily locate the approximate range of text changes, and lay the foundation for accurate identification of differences. Then, the change block set is constructed and the split and merge operations are detected, which can integrate scattered line changes into more logical change blocks, eliminate special operation interference, and make the subsequent detection of major editing operations (such as block movement, copying and line updates) more focused and accurate. In the block movement and copying mapping steps, the relevant line sets are traversed according to the newly added lines, and the detection operations are strictly defined to form a many-to-many mapping set containing line updates, which makes the recognition of the overall operation of the text block more accurate, and can better capture the common and important editing behaviors such as code block movement and copying in actual programming or document editing and their accompanying line updates. When mapping line updates, a weighted method is used to calculate the similarity and cyclically process the intersection to ensure the uniqueness and rationality of the final line update mapping pair, thereby accurately presenting the modification details at the line level. Based on the optimal matching of the KM algorithm, the edge weights are calculated considering the edit type weight, the number of block lines and the context similarity, and the remaining matching sets are processed in multiple cycles. This can effectively solve the many-to-many relationship problem of the mapping set, obtain the matching results that best match the actual editing operations, and enable the entire text difference recognition to reach the optimal state under different edit types and complex relationships. The edit script finally generated records all the editing operations from the original file to the modified file in a clear and orderly manner, including various block-level and line-level operations and their location and content changes, which is conducive to software developers to understand code evolution, document editors to track modification history, and to coordinate the editing content of all parties in team collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a block-based text difference identification method in one embodiment;

[0040] Figure 2 A schematic diagram of updating a many-to-many mapping pair in a row according to an embodiment;

[0041] Figure 3 A schematic diagram of another embodiment in which a many-to-many relationship exists between the overall row and block mapping sets;

[0042] Figure 4 A flowchart of a block-based text difference identification method in one embodiment;

[0043] Figure 5 is a structural block diagram of a block-based text difference identification device in one embodiment;

[0044] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In one embodiment, Figure 1 As shown, a block-based text difference recognition method is provided, comprising the following steps:

[0047] Step 102: Use the Git diff algorithm to process the two files to be compared to obtain a deleted row set, a newly added row set, and an unchanged row set.

[0048] When Git diff algorithms (including Myers, Minimal, Histogram, Patience) compare two files, the lines that exist in the original file (before modification) but are missing in the modified file are classified as deleted line sets. These lines represent the parts that were deleted during the file editing process. In contrast to the deleted line set, the added line set contains the lines that appear in the modified file but do not exist in the original file (before modification). These lines are newly added during the file editing process. The unchanged line set refers to the lines with exactly the same content in the original file (before modification) and the modified file. These lines have not changed during the file editing process.

[0049] Step 104: construct a changed block set according to the deleted row set, the newly added row set and the changed block information.

[0050] A change block set consists of hunks of consecutive line deletions and line additions, and there are no unchanged lines in between these hunks. In other words, when a continuous area is found in the text, where some lines are deleted, followed by some lines being added, and there are no unchanged lines between these two parts, the entire area is defined as a change block, and all such change blocks are grouped together to form a change block set. For example, in a code file, if three consecutive lines of code are deleted, followed by two consecutive lines of code being added, and there are no other unchanged lines of code between the deleted and added five lines of code, then these five lines of code constitute a change block and will be included in the change block set.

[0051] Step 106, traverse the change block set, identify the split and merge rows of the row split and row merge operations in the change block set, and delete the identified split and merge rows from the change block set, the deleted row set, and the added row set.

[0052] Line segmentation refers to splitting the originally continuous text lines logically or in format to make them into multiple lines. Line merging is the opposite of line segmentation. It merges the originally separated text lines into one continuous text line.

[0053] By correctly identifying and processing split rows from the change block set, deleted row set, and newly added row set (such as deleting them after constructing the change block set), subsequent misjudgment of these rows can be avoided, allowing block-based text difference analysis to more accurately focus on real block moves, block copies, and row updates. Similarly, correctly processing merged rows from related sets (such as deleting them) can avoid misinterpretations of text changes and ensure more accurate identification of subsequent block-level and row-level editing operations.

[0054] Step 108, traverse the changed block set to identify row update operations, calculate update similarity values ​​by combining context similarity and row similarity weighting method to obtain multiple-to-multiple row update mapping pairs, and determine the final row update mapping set by cyclically deleting the mapping pairs containing the most intersections.

[0055] The formed many-to-many block movement mapping set not only records the block movement itself, but also includes the row update operations that may accompany the block movement. The update similarity value is calculated by combining the context similarity and row similarity weighted method, which fully considers the complexity of the row update operation. The similarity of the row itself focuses on the direct changes in the text content, such as the replacement of keywords, the adjustment of sentence structure, etc.; while the context similarity measures the relationship between the row update operation and the surrounding text environment from a more macro perspective.

[0056] Step 110, traverse the deleted rows according to the newly added row set, identify all block moves according to the block move information, and form a many-to-many block move mapping set including the row update operations after the block move.

[0057] Through this traversal method, all block replication sources (original blocks from deleted or unchanged rows) and targets (copies in newly added rows) can be fully identified, ensuring that no block replication behavior is missed, thereby accurately constructing a complete picture of block replication in the text. In addition, the formed many-to-many block replication mapping set not only records the basic information of block replication, but also includes the row update operations that may occur after block replication.

[0058] Step 112, traverse the deleted rows and unchanged rows according to the newly added row set, detect block replication according to the block replication information, and form a many-to-many block replication mapping set including row update operations after block replication.

[0059] Through this targeted traversal, these row update operations can be captured comprehensively and carefully, avoiding missing important text change details.

[0060] After obtaining the many-to-many row update mapping pair, such as Figure 2As shown in the figure, since the initial mapping pairs may be crossed, it is a critical step to determine the final row update mapping set by cyclically deleting the mapping pairs with the most intersections. In actual editing scenarios, crossed mapping pairs may lead to a misinterpretation of the text editing logic, for example, two row updates with different intentions may be mistakenly associated together. By cyclically deleting the mapping pairs with the most intersections, these unreasonable associations can be gradually eliminated, so that the final row update mapping set can more clearly and accurately reflect the true update relationship between rows.

[0061] Step 114, use the weighted bipartite graph optimal matching KM algorithm to optimally match the row-by-row update mapping set, the many-to-many block move mapping set, and the many-to-many block copy mapping set, and process the remaining matching sets by multiple cycles of the KM algorithm until the optimal matching set is obtained.

[0062] The row update mapping set, block move mapping set, and block copy mapping set may have complex many-to-many relationships in the initial stage. For example, a newly added row may have a potential mapping relationship with multiple deleted rows or unchanged rows (such as in block copy or block move scenarios). The KM algorithm converts these mapping relationships into a weighted bipartite graph, where the start and end rows of the mapping row or block respectively constitute the start and end nodes of the bipartite graph (if there is overlap, they are merged), the mapping relationship constitutes the node connection, and the edge weight is calculated according to the edit type weight, the number of block rows, and the context similarity, so as to find the maximum weight matching in this weighted bipartite graph. Through multiple iterative calculations, the KM algorithm can determine a set of optimal matching subsets.

[0063] Step 116, taking the split and merged rows, the optimal matching set, and the unmatched deleted and added rows as input, and generating a final editing script in row order.

[0064] The purpose of generating editing scripts in line order is to provide a clear and orderly way to present the editing process of the text. Starting from the beginning of the text, the editing operations experienced by each line are recorded in sequence, whether it is line deletion (using specific symbols or marks to indicate that the line has been deleted), line addition (clearly marking the insertion position and content of the new line), or line update (detailed description of the original content of the line, the updated content, and the reason or basis for the update, which may involve the relationship with the context and other lines), as well as block movement (recording the original position of the block, the position after the move, and the line update during the move) and block copying (also recording the source block position, the target block position, and the line update after the copy). This line-by-line recording method makes the editing script highly readable and traceable. Users can understand the text change process line by line like reading ordinary text, which is convenient for code debugging and review in software development, and tracking content evolution and reviewing modified content in document editing.

[0065] In the block-based text difference recognition method, first, the Myers algorithm is used to obtain the deleted, added and unchanged line sets, which provides basic data for subsequent analysis, helps to preliminarily locate the approximate range of text changes, and lays the foundation for accurate identification of differences. Then, the change block set is constructed and the split and merge operations are detected, which can integrate scattered line changes into more logical change blocks, eliminate special operation interference, and make the subsequent detection of major editing operations (such as block movement, copying and line updates) more focused and accurate. In the block movement and copying mapping steps, the relevant line sets are traversed according to the newly added lines, and the operations are detected according to strict definitions to form a many-to-many mapping set containing line updates, which makes the recognition of the overall operation of the text block more accurate, and can better capture the common and important editing behaviors such as code block movement and copying in actual programming or document editing and their accompanying line updates. When mapping line updates, the weighted method is used to calculate the similarity and cyclically process the intersection to ensure the uniqueness and rationality of the final line update mapping pair, so as to accurately present the modification details at the line level. Based on the optimal matching of the KM algorithm, the edge weights are calculated considering the edit type weight, the number of block lines and the context similarity, and the remaining matching sets are processed in multiple cycles. This can effectively solve the many-to-many relationship problem of the mapping set, and obtain the matching results that best meet the actual editing operations, so that the entire text difference recognition can reach the optimal state under different edit types and complex relationships. The edit script finally generated records all the editing operations from the original file to the modified file in a clear and orderly manner, including various block-level and line-level operations and their location and content changes, which is conducive to software developers to understand code evolution, document editors to track modification history, and to coordinate the editing content of all parties in team collaboration.

[0066] In one embodiment, the changed block information indicates a block consisting of consecutive row deletions and row additions.

[0067] In addition, the block move and block copy information indicates that the line position or indentation position of the entire continuous text line is moved or copied, and the relative indentation of each line in the block remains unchanged before and after the editing operation.

[0068] In yet another embodiment, the block copy information indicates that the line position or indentation position of continuous text lines is copied as a whole, and the relative indentation of each line in the block remains unchanged before and after the copy.

[0069] In one embodiment, since the overall row and block mapping sets composed of the row update mapping set, the block move mapping set and the block copy mapping set may have a many-to-many relationship, such as Figure 3As shown, row updates and block moves only allow one-to-one relationships, and block copies only allow one-to-one and many-to-one relationships. In this regard, the weighted bipartite graph optimal matching KM algorithm (Kuhn-Munkres Algorithm) is used to identify the optimal matching subsets of the input row update mapping set, block move mapping set, and block copy mapping set. Among them, the starting row and the ending row of each group of mapping rows or blocks constitute the starting and ending nodes of the bipartite graph respectively (if the two nodes overlap, they are merged into one node), and the mapping relationship constitutes the node connection. Set the weights of the row update mapping set, many-to-many block move mapping set, and many-to-many block copy mapping set (update mapping set: 1, many-to-many block move mapping set: 2, many-to-many block copy mapping set: 3), and calculate the edge weight according to the editing weight, the number of block rows, and the block context similarity:

[0070] ;

[0071] MappingWeight represents mapping weight, EditWeight represents editing weight, BlockLength represents the number of block lines, and ContextSimilarity represents the block context similarity; the optimal match is performed using the mapping weight as the weight value of the KM algorithm.

[0072] After one KM algorithm, a set of optimal matching sets is obtained and added to the final matching set. There is also a set of remaining matching sets that overlap with the optimal matching set. The matching sets that are not added to the optimal matching set and the optimal matching set are "complemented" by removing the overlapping parts at the start and end points to form the remaining matching sets. Then the KM algorithm is used on the remaining matching sets to form a new set of optimal matching sets and add them to the final matching set. This cycle continues until there are no remaining matching sets.

[0073] Specifically, the KM algorithm is used to find the optimal match in the constructed weighted bipartite graph. The KM algorithm gradually determines the optimal matching scheme by continuously adjusting the label values ​​of the nodes and finding augmenting paths, so that the sum of the weights of the matched edges is maximized (or minimized in some cases, depending on the specific optimization goal setting). In this process, the algorithm will prioritize the edges with higher weights for matching based on the edge weights, because these edges represent a better mapping relationship in terms of editing operation type, text length, and context similarity. For example, a mapping relationship with a higher editing type weight (such as a block copy weight of 3) and a higher context similarity (that is, a better Levenshtein similarity calculation result) and a moderate number of block rows is more likely to be preferred as the optimal match.

[0074] In one embodiment, the context similarity is calculated based on the ratio of equal lines of character strings in the four lines of character strings before and after the two blocks.

[0075] Finally, according to the above embodiment of the present invention, the following is formed: Figure 4 A flowchart of the block-based text difference identification process.

[0076] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0077] In one embodiment, Figure 5 As shown, a block-based text difference recognition device is provided, comprising: a data processing module 502, a mapping module 504 and a matching module 506, wherein:

[0078] The data processing module 502 is used to process the two files to be compared using the Git diff algorithm to obtain a deleted row set, a newly added row set and an unchanged row set; construct a changed block set according to the deleted row set, the newly added row set and the changed block information; traverse the changed block set, identify the split and merged rows of the row split and row merge operations in the changed block set, and delete the identified split and merged rows from the changed block set, the deleted row set and the newly added row set;

[0079] A mapping module 504 is used to traverse the changed block set to identify row update operations, calculate update similarity values ​​by combining context similarity and row similarity weighting method to obtain multiple-to-multi row update mapping pairs, and determine the final row update mapping set by cyclically deleting mapping pairs containing the most intersections; traverse deleted rows according to the newly added row set, identify all block movements according to block movement information to form a multiple-to-multi block movement mapping set containing row update operations after block movement; traverse deleted rows and unchanged rows according to the newly added row set, detect block replication according to block replication information and form a multiple-to-multi block replication mapping set containing row update operations after block replication;

[0080] The matching module 506 is used to use the weighted bipartite graph optimal matching KM algorithm to optimally match the row update mapping set, the multi-block move mapping set and the multi-block copy mapping set, and to process the remaining matching sets by multiple cycles of the KM algorithm until the optimal matching set is obtained; the split and merged rows, the optimal matching set, and the unmatched deleted and added rows are taken as input, and the final editing script is generated in row order.

[0081] In one of the embodiments, the changed block information indicates a block consisting of consecutive row deletions and row additions.

[0082] In one embodiment, the block movement information indicates that the line position or indentation position of the continuous text lines is moved as a whole, and the relative indentation of each line in the block remains unchanged before and after the movement.

[0083] In one embodiment, the block copy information indicates that the line position or indentation position of the continuous text lines is copied as a whole, and the relative indentation of each line in the block remains unchanged before and after the copy.

[0084] In one embodiment, the matching module 506 is further configured to

[0085] The weights of the many-to-many block move mapping set and the many-to-many block copy mapping set are set, and the edge weights are calculated according to the edit weight, the number of block rows, and the block context similarity as follows:

[0086] ;

[0087] Among them, MappingWeight represents mapping weight, EditWeight represents editing weight, BlockLength represents the number of block lines, and ContextSimilarity represents block context similarity;

[0088] The optimal matching is performed using the mapping weight as the KM algorithm weight value.

[0089] In one embodiment, the matching module 506 is further configured to calculate the context similarity according to the ratio of the equal lines of character strings in the four lines of character strings in the two blocks.

[0090] For the specific definition of the block-based text difference identification device, please refer to the definition of the block-based text difference identification method above, which will not be repeated here. Each module in the above-mentioned block-based text difference identification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0091] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a block-based text difference recognition method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0092] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0093] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0094] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0095] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0096] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A block-based text difference recognition method, characterized in that: The method comprises: Use the Git diff algorithm to process the two files to be compared to obtain the deleted row set, the newly added row set, and the unchanged row set; Constructing a changed block set according to the deleted row set, the newly added row set and the changed block information; Traversing the change block set, identifying the split and merge rows of the row split and row merge operations in the change block set, and deleting the identified split and merge rows from the change block set, the deleted row set, and the added row set; Traversing the change block set to identify row update operations, calculating update similarity values ​​by combining context similarity and row similarity weighting method to obtain multiple pairs of row update mapping pairs, and determining a final row update mapping set by cyclically deleting mapping pairs containing the most intersections; Traversing the deleted rows according to the newly added row set, identifying all block moves according to the block move information to form a many-to-many block move mapping set including the row update operations after the block moves; According to the newly added row set, the deleted rows and unchanged rows are traversed, block replication is detected according to the block replication information, and a many-to-many block replication mapping set including row update operations after block replication is formed; The weighted bipartite graph optimal matching KM algorithm is used to optimally match the row update mapping set, many-to-many block move mapping set and many-to-many block copy mapping set. The KM algorithm is cycled multiple times to process the remaining matching sets until the optimal matching set is obtained; the split and merged rows, the optimal matching set, and the unmatched deleted and added rows are taken as input, and the final editing script is generated according to the row order.

2. The method according to claim 1, characterized in that The changed block information indicates a block consisting of consecutive row deletions and row additions.

3. The method according to claim 1, characterized in that The block movement and block copy information indicates that the line position or indentation position of the continuous text lines is moved or copied as a whole, and the relative indentation of each line in the block remains unchanged before and after the editing operation.

4. The method according to claim 1, characterized in that The block copy information indicates that the line position or indentation position of the continuous text lines is copied as a whole, and the relative indentation of each line in the block remains unchanged before and after the copy.

5. The method according to any one of claims 1 to 4, characterized in that: The method of using the weighted bipartite graph optimal matching KM algorithm to optimally match the row update mapping set, the many-to-many block move mapping set, and the many-to-many block copy mapping set includes: The weights of the many-to-many block move mapping set and the many-to-many block copy mapping set are set, and the edge weights are calculated according to the edit weight, the number of block rows, and the block context similarity as follows: Among them, MappingWeight represents mapping weight, EditWeight represents editing weight, BlockLength represents the number of block lines, and ContextSimilarity represents block context similarity; The optimal matching is performed using the mapping weight as the KM algorithm weight value.

6. The method according to claim 5, characterized in that The steps to obtain context similarity include: The context similarity is calculated based on the ratio of equal lines of strings in the four lines of strings before and after the two blocks.

7. A block-based text difference recognition device, characterized in that: The device comprises: The data processing module is used to process two files to be compared using the Git diff algorithm to obtain a deleted row set, a newly added row set and an unchanged row set; construct a changed block set according to the deleted row set, the newly added row set and the changed block information; traverse the changed block set, identify the split and merged rows of the row split and row merge operations in the changed block set, and delete the identified split and merged rows from the changed block set, the deleted row set and the newly added row set; A mapping module is used to traverse the changed block set to identify row update operations, calculate update similarity values ​​by combining context similarity and row similarity weighting method to obtain multiple-to-multi row update mapping pairs, and determine the final row update mapping set by cyclically deleting mapping pairs containing the most intersections; traverse deleted rows according to the newly added row set, identify all block movements according to block movement information to form a multiple-to-multi block movement mapping set containing row update operations after block movement; traverse deleted rows and unchanged rows according to the newly added row set, detect block replication according to block replication information and form a multiple-to-multi block replication mapping set containing row update operations after block replication; The matching module is used to use the weighted bipartite graph optimal matching KM algorithm to optimally match the row update mapping set, the many-to-many block move mapping set and the many-to-many block copy mapping set, and to process the remaining matching sets through multiple cycles of the KM algorithm until the optimal matching set is obtained; the split and merged rows, the optimal matching set, and the unmatched deleted and added rows are taken as input, and the final editing script is generated according to the row order.

8. The device according to claim 7, characterized in that The changed block information indicates a block consisting of consecutive row deletions and row additions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images