Source code similarity detection method and system based on deep learning
By processing source code into code trees and sequence images using a deep learning-based method, the problem of balancing efficiency and accuracy in traditional methods is solved, achieving efficient and intuitive comparison of code similarity detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NINGXIA KAIXINTE INFORMATION TECH CO LTD
- Filing Date
- 2026-01-19
- Publication Date
- 2026-05-12
AI Technical Summary
Traditional code similarity detection methods struggle to balance efficiency and accuracy, and lack the ability to aggregate and abstract code logic at a high level, resulting in a cumbersome comparison process that makes it difficult to intuitively present the similarity relationships between codes.
The deep learning-based method processes the source code into a code tree and decomposes it into subtree sequences. These sequences are then uploaded to the data operation pool for processing via a transmission path channel. After being converted into sequence images, they are compared, and the similarity is determined by using statement identifiers and image overlap rates.
It improves the efficiency and accuracy of code similarity recognition, and makes code similarity judgment more intuitive and accurate.
Smart Images
Figure FT_1 
Figure FT_2
Abstract
Description
Technical Field
[0001] This invention relates to the field of similarity detection technology, specifically to a source code similarity detection method and system based on deep learning. Background Technology
[0002] In the field of software engineering, code similarity detection is of great significance for identifying code clones, detecting software plagiarism, assisting in program understanding, and promoting code reuse. Traditional code similarity detection methods are usually based on text comparison or token sequence matching. These methods can reflect the similarity characteristics of code to a certain extent, but they often have limitations in balancing efficiency and accuracy. In addition, existing methods mostly rely on directly matching code symbols, lacking the ability to aggregate and abstract code logic at a high level, making the comparison process cumbersome and not easy to intuitively present the overall similarity relationship between two code segments. Summary of the Invention
[0003] The purpose of this invention is to provide a source code similarity detection method and system based on deep learning to address the shortcomings in the prior art.
[0004] To achieve the above objectives, the present invention provides the following technical solution: a source code similarity detection method based on deep learning, comprising the following steps: Step S1: Obtain two pieces of source code to be detected, and construct the code tree corresponding to each source code. Decompose the code tree into several subtree sequences based on statement identifiers, and construct the transmission path channel for each subtree sequence. Step S2: Establish a data operation pool, connect the transmission path channel corresponding to each subtree sequence to the data operation pool, and have the data operation pool process each subtree sequence into a corresponding sequence image; Step S3: Compare images from different sequences. Based on the comparison results of all sequence images corresponding to the same code tree, obtain the similarity between the two source code segments.
[0005] In a preferred embodiment, the process of acquiring two pieces of source code to be detected and constructing the code tree corresponding to each source code includes: Select two code snippets as the target code for which similarity detection is required; Set the starting and ending reading points of the target code. Each code segment of the target code is a code item. Set the code extraction characters. Build a root node for each of the two target code segments. Assign the code item at the starting reading point to the root node. Traverse the code segment of the target code from the root node until each code extraction character is reached to obtain a corresponding tree path. The tree path is split at the position corresponding to the extracted character in the code. The split position is marked as the secondary root node. The current tree path is processed into tree paths under two branches that form the same root relationship. This process continues until the extracted character is traversed to the termination point of the target code. This completes the construction of a code tree consisting of a root node, several secondary root nodes, and several tree paths.
[0006] In a preferred embodiment, the process of decomposing the code tree into several subtree sequences based on statement identifiers and constructing a separate transmission path channel for each subtree sequence includes: Statement identifiers represent different types of statement fragments. By processing the target code through several types of statement identifiers, the code tree of the code tree is obtained, which corresponds to the code subtree represented by each of the statement identifiers. The code subtrees of statement identifiers of the same type in two target code segments are classified into a subtree sequence. Several subtree sequences are obtained based on the classification of several types of statement identifiers. The corresponding number of transmission channels are set based on the subtree sequences. Several routing points and parsing points are set on the transmission channel of each subtree sequence. At each routing point, the behavior events on the transmission channel of the corresponding subtree sequence are captured and handed over to the parsing point at the corresponding position to parse the behavior events, determine whether the behavior events are abnormal events, and obtain a safe path based on the judgment result. When each segment of the transmission channel formed between routing points is marked as a secure path, all secure paths corresponding to the same transmission channel are integrated, and the corresponding transmission channel is converted into a transmission path channel. Each transmission path channel is used to perform secure transmission of a corresponding subtree sequence.
[0007] In a preferred embodiment, the process of determining whether a behavioral event is an abnormal event and obtaining a safe path based on the determination result includes: If so, the decision is made to obtain the defense measures for the abnormal event, and the abnormal event and the corresponding defense measures are broadcast at the current routing point to other same or different transmission channels, and received by the routing points on the transmission channels that have not been processed. When a receiving routing point captures a behavioral event, it compares the captured behavioral event with the previously received behavioral event. If the comparison matches, it directly invokes the defense measures corresponding to the compared behavioral event. If the comparison does not match, it hands over the data parsing of the corresponding behavioral event to the parsing point located at the same position as the current routing point, and decides on the corresponding defense measures. If not, the transmission channel formed between the current routing point and the next adjacent routing point is segmented and marked as a safe path.
[0008] In a preferred embodiment, the process of establishing a data operation pool includes: Deploy a resource pool and connect it to the cloud server. Create several operation partitions within the resource pool. Set data operation points and data index points in each operation partition. Data operation points are used to obtain a type of code-visualization architecture. Based on the obtained code-visualization architecture, perform visualization processing on the code data stored in the current operation partition. Data index points are used to establish an architecture reuse grid between different data operation points. The architecture reuse grid includes fully reused subgrids and partially iterative subgrids. Fully reused subgrids are used to call the corresponding processing steps of the image processing in different operation partitions that are completely identical in architecture. Partially iterative subgrids are used to call the processing steps of the image processing in different operation partitions that are partially identical in architecture. For the different parts of the architecture, it is re-iterated based on the current code-image processing architecture. When each operation partition within the resource pool completes the acquisition of the corresponding code for the corresponding data operation point—the graphical processing architecture—the entire resource pool is used as the data operation pool.
[0009] In a preferred embodiment, the process of connecting the transmission path channel corresponding to each subtree sequence to the data operation pool via a port includes: Set a port receiving address for each operation partition, obtain the channel address of the transmission path channel corresponding to each subtree sequence as the port upload address, and each transmission path channel initiates its own data upload request to the data operation pool; After receiving a data upload request, the data operation pool associates the receiving port address of one of its idle operation partitions with the upload port address that initiated the data upload request, establishes a communication association between the corresponding transmission path channel and the data operation pool, and sends the subtree sequence to the corresponding communication-associated operation partition for storage through the transmission path channel.
[0010] In a preferred embodiment, the process of processing each subtree sequence into a corresponding sequence image by the data operation pool includes: Within each operation partition, the subtree sequence is broken down into several subsequence objects; Set up compilation units, notification units, and scheduling units at each subsequence object; The compilation unit is used to complete the compilation and parsing of the corresponding sub-sequence objects by the code-image processing architecture, and to convert the code fragments corresponding to the sub-sequence objects into sequence images; The notification unit is used to notify other subsequence objects that have not been compiled and parsed of the result data of the compilation and parsing of any subsequence object under a subtree sequence. The scheduling unit is used to obtain the result data of sub-sequence objects that have not been compiled and parsed, and to copy the corresponding sequence images.
[0011] In a preferred embodiment, the process of comparing different sequence images and obtaining the similarity between two source code segments based on the comparison results of all sequence images corresponding to the same code tree includes: Several sequence images are grouped according to their respective code trees to obtain two image sets for each code tree. Each image set includes several sequence images that are advanced in the order of code compilation time. Set up a processing layer for two sequence images that are in the same code compilation sequence, stack the sequence images within the processing layer, and set the minimum segmentation area to process the stacked sequence images into several sub-stacked objects; Based on the sub-overlay objects and the preset first overlap threshold, similar sub-objects and dissimilar sub-objects are obtained. Based on the similar sub-objects and dissimilar sub-objects, the layer overlap rate of the processed layer is obtained. Based on the layer overlap rate and the preset second overlap threshold, similar layers and dissimilar layers are obtained. The tree similarity rate between two code trees is obtained based on similar and dissimilar layers. The tree similarity rate is used as the code similarity between the two target code segments. The similarity between the two target code segments is used to determine the degree of similarity of the source code represented by each segment.
[0012] This invention also provides a deep learning-based source code similarity detection system, the system comprising: The code tree module is used to acquire two pieces of source code to be detected, and to build the code tree corresponding to each source code. The code tree is decomposed into several subtree sequences based on statement identifiers, and a transmission path channel is built for each subtree sequence. The sequence image module is used to establish a data operation pool, connect the transmission path channel corresponding to each subtree sequence to the data operation pool, and process each subtree sequence into a corresponding sequence image by the data operation pool. The similarity comparison module is used to compare images from different sequences. Based on the comparison results of all sequence images corresponding to the same code tree, it obtains the degree of similarity between two pieces of source code.
[0013] In the above technical solution, the technical effects and advantages provided by the present invention are as follows: The present invention processes the source code into a code tree that embodies its core logic, and further decomposes the code tree into several subtree sequences. A corresponding transmission path channel is constructed for each subtree sequence to be uploaded to the data operation pool for processing. Simultaneously, several subtree sequences are converted into corresponding sequence images. By comparing the similarity between the sequence images represented by the two source code segments, the similarity between the source code segments is obtained. On the one hand, the subtree sequences are obtained based on the classification of statement identifiers. Statement identifiers represent a type of statement fragment, realizing the similarity aggregation of a certain amount of code data, laying the foundation for subsequent data analysis and processing, and improving the efficiency of code similarity recognition to a certain extent. On the other hand, the images have the same specifications and size, and the comparison between the images is more intuitive and convenient to judge by the overlap rate between the two, which helps to improve the accuracy and intuitiveness of code similarity judgment. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.
[0015] Figure 1 This is a flowchart of the method of the present invention.
[0016] Figure 2 This is a system block diagram of the present invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Example 1, please refer to Figure 1 As shown in this embodiment, a source code similarity detection method based on deep learning includes the following steps: Step S1: Obtain two pieces of source code to be detected, and construct the code tree corresponding to each source code. Decompose the code tree into several subtree sequences based on statement identifiers, and construct the transmission path channel for each subtree sequence. Step S2: Establish a data operation pool, connect the transmission path channel corresponding to each subtree sequence to the data operation pool, and have the data operation pool process each subtree sequence into a corresponding sequence image; Step S3: Compare images from different sequences. Based on the comparison results of all sequence images corresponding to the same code tree, obtain the similarity between the two source code segments.
[0019] It should be further explained that, in the specific implementation process, the process of obtaining two pieces of source code to be tested and constructing the code tree corresponding to each source code includes: Select two code snippets as the target code for which similarity detection is required; Set the start and end points of the target code. The start point corresponds to the first code item of the code segment, and the end point corresponds to the last code item of the code segment. Set the code extraction character, which is used as a filter identifier for several code items between the start and end points. Construct a root node for each of the two target code segments, map the code item at the starting reading point of the target code to the root node, and traverse the code extraction character from the root node along the code segment of the target code until every code extraction character is traversed to obtain a corresponding tree path; The tree path is split at the position corresponding to the extracted character in the code. The split position is marked as the secondary root node. The current tree path is processed into tree paths under two branches that form the same root relationship. This process continues until the extracted character in the code is traversed to the termination reading point corresponding to the target code. In this way, a code tree consisting of a root node, several secondary root nodes and several tree paths is constructed. Among them, the code extraction characters are feature identifier characters obtained after analyzing the source code of a certain amount of historical data. The code extraction characters are used to divide the source code into different specific execution function types. For example, when the code extraction character is else, the code corresponding to another execution condition opposite to the current execution condition is selected, indicating that the code segment corresponding to the current source code undergoes a condition change at the corresponding code extraction character. When the code extraction character is if, the corresponding condition of the code segment corresponding to several selectable selection conditions is executed at the current position. When the code extraction character is Switch, the corresponding condition of several selectable selection conditions is executed at the position corresponding to the current code extraction character. The code extraction characters include, but are not limited to, the above types. It should be noted that the code extraction characters used for code feature extraction are obtained based on deep learning technology. Specifically, the code extraction characters are implemented and optimized by a pre-trained deep neural network model. This deep neural network model is built based on deep learning technology and is trained on a large amount of historical source code data (such as open source project libraries) to learn the syntactic structure and semantic logic of the source code. It automatically identifies and outputs key logical segmentation points or statement type identifiers in the source code. Code items are each specific code word in the source code, specifically represented by words in the source code. By comparing the code extraction characters with each code item in the code segment in turn, it is determined whether there are logical or functional choices or changes at a certain reading position in the code segment, and then the source code is split into different refined sub-segments.
[0020] It should be further explained that, in the specific implementation process, the process of decomposing the code tree into several subtree sequences based on statement identifiers and constructing a separate transmission path channel for each subtree sequence includes: By analyzing historical code data using big data and the bag-of-words model, statement identifiers are obtained based on the analysis results. These statement identifiers represent different types of statement fragments. The historical code data covers several types of code fragments corresponding to several types of code logic. The statement identifiers obtained by processing the historical code data also cover common types of statement fragments. For example, when there are several selectable code items included in the Switch statement in the source code, the selectable code items are in the same code structure. A corresponding statement identifier is assigned to the code items in the same code structure, and different statement identifiers are assigned to the code items in different code structures. For example, in the selection judgment statement corresponding to if-else, the code items in the code structure corresponding to if are assigned the corresponding statement identifier based on the code function to be executed, while the code items in the code structure corresponding to else are assigned a different type of statement identifier. By processing the code trees corresponding to the two target code segments using several types of statement identifiers, the code subtrees represented by all statement identifiers corresponding to each code tree are obtained. The code subtrees of the same type of statement identifier in the two target code segments are classified into a subtree sequence. Then, based on the statement identifiers of several types, the two target code segments are classified into several subtree sequences accordingly. Based on the subtree sequence, a corresponding number of transmission channels are set. Several routing points and parsing points are set on the transmission channel corresponding to each subtree sequence. At each routing point, the behavior events that occur on the transmission channel where the corresponding subtree sequence is located are captured, and the corresponding parsing point is used to parse the behavior events and determine whether the behavior events are abnormal events. If so, the decision is made to obtain the defense measures corresponding to the abnormal event, and the abnormal event and the corresponding defense measures are broadcast at the current routing point to other same or different transmission channels, so that the unprocessed routing points on the transmission channels can receive them; When a receiving routing point captures a behavioral event, it compares the captured behavioral event with the previously received behavioral event. If they match, it directly calls the execution result corresponding to the compared behavioral event, that is, it executes the corresponding defensive measures. If they do not match, it hands over the data parsing of the corresponding behavioral event to the parsing point located at the same position as the current routing point, and decides on the corresponding defensive measures. If not, the transmission channel formed between the current routing point and the next adjacent routing point is segmented and marked as a safe path; Repeat the above operation until each transmission channel segment formed between routing points is marked as a secure path. Integrate all secure paths corresponding to the same transmission channel, and then convert the corresponding transmission channel into a transmission path channel. Each transmission path channel is used to perform secure transmission of a corresponding subtree sequence.
[0021] It should be further explained that, in the specific implementation process, the process of establishing a data operation pool and connecting the transmission path channel corresponding to each subtree sequence to the data operation pool includes: Deploy a resource pool, connect the resource pool to the cloud server, create several operation partitions within the resource pool, and set corresponding data operation points and data index points in each operation partition. The data operation point is used to obtain a type of code-graphic processing architecture, and based on the obtained code-graphic processing architecture, to perform corresponding graphical processing on the code data stored in the current operation partition. The data index points are used to establish an architecture reuse grid between different data operation points. The architecture reuse grid includes fully reused sub-grids and partially iterative sub-grids. By fully reusing subgrids to call the corresponding processing steps of the image processing in different operation partitions with the same architecture, by partially iterating subgrids to call the processing steps of the image processing in different operation partitions with the partially the same architecture, and by iterating the different architecture parts again based on the current code-image processing architecture. When each operation partition within the resource pool completes the acquisition of the corresponding code-graphical processing architecture for the corresponding data operation point, the entire resource pool is used as the corresponding data operation pool.
[0022] In this process, a digital digest of architectural features is constructed for each data operation point using deep learning technology. The digital digest of architectural features includes the identifier of the core algorithm or tool used by the data operation point, the hash of configuration parameters (such as the hash value used in the source code to represent the theme, font, background or margin settings), and the code input / output format. The digital digest of architectural features is used as the identity of the data operation point. An architecture reuse grid is constructed for adjacent data operation points at the data index point connection position. The fully reused subgrids and some iterative subgrids included in the architecture reuse grid correspond to different use cases. When the architecture type required by a certain code data is exactly the same as the code-graphic processing architecture corresponding to the existing operation partition, a new operation partition is not created. Instead, the corresponding identical data operation point is called through the architecture reuse grid as a fully reused subgrid. The code-graphic processing architecture of the fully reused subgrid is used as the usage condition of the current operation partition. When the required architecture type for a certain code data cannot be completely identical to the existing code-graphics processing architecture corresponding to the operation partition, but there are some similar architecture features (for example, the existing code snippet being analyzed represents a dark theme, while the architecture types of all operation partitions only include light themes, but all parameters except for the contrast between light and dark themes are exactly the same), the most similar data operation point is located through the architecture reuse grid for invocation, and used as a partial iterative subgrid. The processing steps of the operation partition corresponding to the partial iterative subgrid in the same part are reused, and the code-graphics processing architecture of the corresponding operation partition is called for local iteration for different parts (theme contrast). This avoids building the entire processing flow from scratch and saves computation and initialization time.
[0023] Set a corresponding port receiving address for each operation partition, obtain the channel address of the transmission path channel corresponding to each subtree sequence as the port upload address, and each transmission path channel initiates its own data upload request to the data operation pool. After receiving a data upload request, the data operation pool associates the port receiving address of one of its idle operation partitions with the port uploading address that initiated the data upload request. It then establishes a communication association between the corresponding transmission path channel and the data operation pool, thereby completing end-to-end communication between the transmission path channel and the data operation pool. Finally, it sends the subtree sequence through the transmission path channel to a specific operation partition in the corresponding communication-associated data operation pool for storage.
[0024] It should be further explained that, in the specific implementation process, the process by which the data operation pool processes each subtree sequence into a corresponding sequence image includes: Set the processing order for each operation partition in the data operation pool. Based on the processing order, process the subtree sequence stored in each operation partition in turn. Within each operation partition, decompose the subtree sequence into several subsequence objects. Set up compilation units, notification units, and scheduling units at each subsequence object; The compilation unit is used to complete the compilation and parsing of the corresponding sub-sequence objects by the code-image processing architecture, converting the partial code fragments corresponding to the sub-sequence objects into sequence images. Specifically, the code-image processing architecture generates sequence images as follows: the compilation unit receives sub-sequence objects allocated from the data operation pool. Each sub-sequence object contains partial code fragments to be converted, along with corresponding metadata (such as programming language type and subtree sequence identifier). The compilation unit preprocesses the partial code fragments to remove irrelevant whitespace characters, standardize newline character format, and perform initial syntax validity checks. It then calls the syntax parser integrated into the code-image processing architecture to perform lexical and syntactic analysis on the preprocessed partial code fragments, constructing the corresponding abstract syntax tree. Simultaneously, it performs enhanced analysis, including identifying and annotating keywords, operators, literals, comments, and identifiers in the code, and analyzing code blocks (such as functions, loops, and conditional statements). Based on the nesting hierarchy and scope, and according to preset rules or context, semantic type inference and classification are performed on specific identifiers (such as variable names and function names) to provide a data foundation for subsequent image rendering. Based on the abstract syntax tree and enhanced analysis results, combined with the visual theme configuration (including color scheme, font family, font size, line height, margin, shadow effect, and window decoration style) preloaded by the code-image processing architecture, a stylized rendering model is generated. Each syntax element node in the abstract syntax tree is mapped to the corresponding style rule defined in the visual theme configuration. According to the nesting hierarchy of code blocks, progressive indentation is calculated and applied, and line numbers, syntax highlight colors, and background area color blocks are integrated to form an intermediate data model that completely describes the position, color, and content of each pixel or vector element in the sequence of images. The stylized rendering model is then passed to the corresponding image generation engine deployed in the code-image processing architecture to perform the final transformation, including raster image paths and vector image paths. If the output format is a bitmap such as PNG or JPG, the image generation engine creates a virtual canvas in memory and accurately draws all elements according to the stylized rendering model, including text anti-aliasing, background gradient rendering, and shadow effect compositing. The canvas content is then rasterized and encoded into the specified bitmap format file to generate the final sequence of images. If the output format is a vector graphic such as SVG, the image generation engine directly converts the stylized rendering model into the corresponding SVG XML document structure, retains the vector description information of all graphic elements, and generates the corresponding sequence of images. The generated sequence of image files are assigned a unique identifier and associated with the identifier of the original sub-sequence object. The compilation unit also encapsulates a metadata file, recording the version type of the code-image processing architecture used, the visual theme configuration hash value, the generation timestamp, the source code fragment hash value, and the storage path or binary data index of the sequence images. Finally, the sequence images and their metadata serve as the result data of the compilation and parsing of the sub-sequence object.
[0025] Among them, the image generation engine and stylized rendering model are deep neural network models trained on a large scale of code-image pairs. By integrating deep learning technology into the generation stage of sequential images, it realizes the transformation from rule-based code to image to intelligent visualization based on semantic understanding.
[0026] The notification unit is used to notify other subsequence objects that have not been compiled and parsed of the result data of any subsequence object under a subtree sequence, and to set the index address of the subsequence object on the cloud server, and to store the result data obtained from the notification to their respective index addresses; The scheduling unit is used to obtain the index address of the sub-sequence object that has not been compiled and parsed. When the sub-sequence object needs to generate the corresponding sequence image of the corresponding code segment, the index address is called to obtain the result data of the sub-sequence object that has been compiled and parsed in the same subtree sequence, and the corresponding sequence image is directly copied.
[0027] It should be further explained that, in the specific implementation process, the process of comparing different sequence images and obtaining the similarity between two pieces of source code based on the comparison results of all sequence images corresponding to the same code tree includes: Several sequence images are grouped according to the code tree to which they belong, thereby obtaining image sets corresponding to the two code trees. Each image set includes several sequence images that are advanced in the order of code compilation time. Set up a processing layer for two image sequences in the same code compilation sequence from two image sets. Stack the image sequences within the processing layer, set a minimum segmentation region, and process the stacked image sequences into several sub-stacked objects based on the minimum segmentation region. Each sub-overlay object is compared sequentially to obtain the overlap rate of each sub-overlay object. When the overlap rate exceeds the preset first overlap threshold, the corresponding sub-overlay object is marked as a similar sub-object; otherwise, the sub-overlay object is marked as a dissimilar sub-object. The process of obtaining the overlap rate of each sub-overlay object is as follows: Define the boundary range (the area range of the minimum segmentation region) for each sub-overlay object. The boundary range is uniquely determined by the position of the minimum segmentation region within the processing layer. Extract the pixel regions corresponding to the boundary range from the two stacked image sequences, denoted as region A (from image set one) and region B (from image set two). Convert region A and region B into pixel value matrices of the same dimension (image set one and image set two correspond to the image sets formed by the two target code segments, respectively). Calculate the overlap rate between region A and region B by iterating and comparing the pixel values at each corresponding position in the two pixel matrices. When two pixel values are completely equal (or the difference is within a preset tolerance threshold), they are counted as one overlapping pixel. The overlap rate of the sub-overlay object is obtained based on the following formula: Overlap rate = (Total number of overlapping pixels / Total number of pixels within the pixel area of the sub-overlapping object) × 100%; The overlap rate is identified as The first overlap threshold is denoted as ; When the child stacked objects satisfy ≥ If the sub-overlay object is highly similar in the two image sequences, then the corresponding sub-overlay object is identified as a similar sub-object. When the child stacked objects satisfy < If the sub-overlay object has a significant visual difference in the two image sequences, then the corresponding sub-overlay object is identified as a dissimilar sub-object. The layer overlap rate of the corresponding processing layer is obtained based on the overlap rate of all sub-overlay objects under the same processing layer, where the number of similar sub-objects is denoted as... The number of dissimilar sub-objects is denoted as The layer overlap rate is denoted as The layer overlap rate can be expressed by the following formula: ; Set a second overlap threshold, and denote the second overlap threshold as... ; Will ≥ The processed layer is identified as a similar layer, and < The processed layers are marked as dissimilar layers; Based on the identifier status of all processing layers under two code trees (i.e., whether the processing layer is a similar layer or a dissimilar layer), the tree similarity rate between the two code trees is obtained, and the tree similarity rate is denoted as... The number of similar layers and the number of dissimilar layers under the two code trees are respectively denoted as . as well as The tree similarity rate can be expressed by the following formula: ; The tree similarity rate corresponding to the two code trees is used as the code similarity between the two target code segments. Then, the similarity between the source code represented by the two target code segments is determined by the code similarity.
[0028] Example 2, please refer to Figure 2 As shown, the present invention also provides a source code similarity detection system based on deep learning, the system comprising: The code tree module is used to acquire two pieces of source code to be detected, and to build the code tree corresponding to each source code. The code tree is decomposed into several subtree sequences based on statement identifiers, and a transmission path channel is built for each subtree sequence. The sequence image module is used to establish a data operation pool, connect the transmission path channel corresponding to each subtree sequence to the data operation pool, and process each subtree sequence into a corresponding sequence image by the data operation pool. The similarity comparison module is used to compare images from different sequences. Based on the comparison results of all sequence images corresponding to the same code tree, it obtains the degree of similarity between two pieces of source code.
[0029] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A source code similarity detection method based on deep learning, characterized in that, Includes the following steps: Step S1: Obtain two pieces of source code to be detected, and construct the code tree corresponding to each source code. Decompose the code tree into several subtree sequences based on statement identifiers, and construct the transmission path channel for each subtree sequence. Step S2: Establish a data operation pool, connect the transmission path channel corresponding to each subtree sequence to the data operation pool, and have the data operation pool process each subtree sequence into a corresponding sequence image; Step S3: Compare images from different sequences. Based on the comparison results of all sequence images corresponding to the same code tree, obtain the similarity between the two source code segments.
2. The source code similarity detection method based on deep learning according to claim 1, characterized in that, The process of obtaining two pieces of source code to be tested and constructing the code tree corresponding to each source code includes: Select two code snippets as the target code for which similarity detection is required; Set the starting and ending reading points of the target code. Each code segment of the target code is a code item. Set the code extraction characters. Build a root node for each of the two target code segments. Assign the code item at the starting reading point to the root node. Traverse the code segment of the target code from the root node until each code extraction character is reached to obtain a corresponding tree path. The tree path is split at the position corresponding to the extracted character in the code. The split position is marked as the secondary root node. The current tree path is processed into tree paths under two branches that form the same root relationship. This process continues until the extracted character is traversed to the termination point of the target code. This completes the construction of a code tree consisting of a root node, several secondary root nodes, and several tree paths.
3. The source code similarity detection method based on deep learning according to claim 2, characterized in that, The process of decomposing the code tree into several subtree sequences based on statement identifiers and constructing a separate transmission path channel for each subtree sequence includes: Statement identifiers represent different types of statement fragments. By processing the target code through several types of statement identifiers, the code tree of the code tree is obtained, which corresponds to the code subtree represented by each of the statement identifiers. The code subtrees of statement identifiers of the same type in two target code segments are classified into a subtree sequence. Several subtree sequences are obtained based on the classification of several types of statement identifiers. The corresponding number of transmission channels are set based on the subtree sequences. Several routing points and parsing points are set on the transmission channel of each subtree sequence. At each routing point, the behavior events on the transmission channel of the corresponding subtree sequence are captured and handed over to the parsing point at the corresponding position to parse the behavior events, determine whether the behavior events are abnormal events, and obtain a safe path based on the judgment result. When each segment of the transmission channel formed between routing points is marked as a secure path, all secure paths corresponding to the same transmission channel are integrated, and the corresponding transmission channel is converted into a transmission path channel. Each transmission path channel is used to perform secure transmission of a corresponding subtree sequence.
4. The source code similarity detection method based on deep learning according to claim 3, characterized in that, The process of determining whether a behavioral event is an abnormal event and obtaining a safe path based on the determination result includes: If so, the decision is made to obtain the defense measures for the abnormal event, and the abnormal event and the corresponding defense measures are broadcast at the current routing point to other same or different transmission channels, and received by the routing points on the transmission channels that have not been processed. When a receiving routing point captures a behavioral event, it compares the captured behavioral event with the previously received behavioral event. If the comparison matches, it directly invokes the defense measures corresponding to the compared behavioral event. If the comparison does not match, it hands over the data parsing of the corresponding behavioral event to the parsing point located at the same position as the current routing point, and decides on the corresponding defense measures. If not, the transmission channel formed between the current routing point and the next adjacent routing point is segmented and marked as a safe path.
5. The source code similarity detection method based on deep learning according to claim 4, characterized in that, The process of establishing a data operation pool includes: Deploy a resource pool and connect it to the cloud server. Create several operation partitions within the resource pool. Set data operation points and data index points in each operation partition. Data operation points are used to obtain a type of code-visualization architecture. Based on the obtained code-visualization architecture, perform visualization processing on the code data stored in the current operation partition. Data index points are used to establish an architecture reuse grid between different data operation points. The architecture reuse grid includes fully reused subgrids and partially iterative subgrids. Fully reused subgrids are used to call the corresponding processing steps of the image processing in different operation partitions that are completely identical in architecture. Partially iterative subgrids are used to call the processing steps of the image processing in different operation partitions that are partially identical in architecture. For the different parts of the architecture, it is re-iterated based on the current code-image processing architecture. When each operation partition within the resource pool completes the acquisition of the corresponding code for the corresponding data operation point—the graphical processing architecture—the entire resource pool is used as the data operation pool.
6. The source code similarity detection method based on deep learning according to claim 5, characterized in that, The process of connecting the transmission path channel corresponding to each subtree sequence to the data operation pool via ports includes: Set a port receiving address for each operation partition, obtain the channel address of the transmission path channel corresponding to each subtree sequence as the port upload address, and each transmission path channel initiates its own data upload request to the data operation pool; After receiving a data upload request, the data operation pool associates the receiving port address of one of its idle operation partitions with the upload port address that initiated the data upload request, establishes a communication association between the corresponding transmission path channel and the data operation pool, and sends the subtree sequence to the corresponding communication-associated operation partition for storage through the transmission path channel.
7. The source code similarity detection method based on deep learning according to claim 6, characterized in that, The process by which the data manipulation pool processes each subtree sequence into a corresponding sequence image includes: Within each operation partition, the subtree sequence is broken down into several subsequence objects; Set up compilation units, notification units, and scheduling units at each subsequence object; The compilation unit is used to complete the compilation and parsing of the corresponding sub-sequence objects by the code-image processing architecture, and to convert the code fragments corresponding to the sub-sequence objects into sequence images; The notification unit is used to notify other subsequence objects that have not been compiled and parsed of the result data of the compilation and parsing of any subsequence object under a subtree sequence. The scheduling unit is used to obtain the result data of sub-sequence objects that have not been compiled and parsed, and to copy the corresponding sequence images.
8. The source code similarity detection method based on deep learning according to claim 7, characterized in that, The process of comparing different image sequences and obtaining the similarity between two pieces of source code based on the comparison results of all image sequences corresponding to the same code tree includes: Several sequence images are grouped according to their respective code trees to obtain two image sets for each code tree. Each image set includes several sequence images that are advanced in the order of code compilation time. Set up a processing layer for two sequence images that are in the same code compilation sequence, stack the sequence images in the processing layer, and set the minimum segmentation area to process the stacked sequence images into several sub-stacked objects; Based on the sub-overlay objects and the preset first overlap threshold, similar sub-objects and dissimilar sub-objects are obtained. Based on the similar sub-objects and dissimilar sub-objects, the layer overlap rate of the processed layer is obtained. Based on the layer overlap rate and the preset second overlap threshold, similar layers and dissimilar layers are obtained. The tree similarity rate between two code trees is obtained based on similar and dissimilar layers. The tree similarity rate is used as the code similarity between the two target code segments. The similarity between the two target code segments is used to determine the degree of similarity of the source code represented by each segment.
9. A deep learning-based source code similarity detection system, used to implement the source code similarity detection method according to any one of claims 1 to 8, characterized in that, The system includes: The code tree module is used to acquire two pieces of source code to be detected, and to build the code tree corresponding to each source code. The code tree is decomposed into several subtree sequences based on statement identifiers, and a transmission path channel is built for each subtree sequence. The sequence image module is used to establish a data operation pool, connect the transmission path channel corresponding to each subtree sequence to the data operation pool, and process each subtree sequence into a corresponding sequence image by the data operation pool. The similarity comparison module is used to compare images from different sequences. Based on the comparison results of all sequence images corresponding to the same code tree, it obtains the degree of similarity between two pieces of source code.