Code data processing method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a code data processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The code data processing method and device are applied to various code detection scenes such as cloud technology, artificial intelligence, intelligent traffic and games. The code data processing method comprises the following steps: extracting a to-be-detected semantic slice from a to-be-detected code; performing character feature extraction on the code character sequence of the semantic slice to be detected to obtain a character feature sequence; performing semantic feature extraction on the character feature sequence based on the first sequence direction to obtain a first semantic feature sequence; semantic feature extraction is carried out on the character feature sequence based on a second sequence direction, a second semantic feature sequence is obtained, and the first sequence direction is opposite to the second sequence direction; and performing code detection in combination with the first semantic feature sequence and the second semantic feature sequence to obtain a code detection result. Through the method, the code detection accuracy can be improved.

Description

Technical Field

[0001] This application relates to data processing technology in the field of computer applications, and more particularly to a code data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] After the code is written, in order to improve the quality and security of the code, it is usually tested to detect whether there are any code vulnerabilities or other code anomalies.

[0003] To perform code detection, code is typically segmented into fixed-length segments, and detection is then performed based on these segments. However, in this process, segmenting the code into fixed-length segments affects the accuracy of code feature acquisition, which in turn impacts the accuracy of code detection. Summary of the Invention

[0004] This application provides a code data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the accuracy of code detection.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a code data processing method, the code data processing method including:

[0007] Extract semantic slices to be detected from the code to be detected;

[0008] Character features are extracted from the code character sequence of the semantic slice to be detected to obtain a character feature sequence;

[0009] Semantic features are extracted from the character feature sequence based on the first sequence direction to obtain the first semantic feature sequence;

[0010] Based on the second sequence direction, semantic features are extracted from the character feature sequence to obtain a second semantic feature sequence, wherein the first sequence direction is opposite to the second sequence direction.

[0011] Code detection is performed by combining the first semantic feature sequence and the second semantic feature sequence to obtain the code detection result.

[0012] This application provides a code data processing apparatus, the code data processing apparatus comprising:

[0013] The slice extraction module is used to extract semantic slices to be detected from the code to be detected;

[0014] The feature extraction module is used to extract character features from the code character sequence of the semantic slice to be detected, and obtain a character feature sequence.

[0015] The feature extraction module is further configured to perform semantic feature extraction on the character feature sequence based on the first sequence direction to obtain a first semantic feature sequence;

[0016] The feature extraction module is further configured to perform semantic feature extraction on the character feature sequence based on the second sequence direction to obtain a second semantic feature sequence, wherein the first sequence direction is opposite to the second sequence direction;

[0017] The code detection module is used to perform code detection by combining the first semantic feature sequence and the second semantic feature sequence to obtain the code detection result.

[0018] In this embodiment of the application, the feature extraction module is further configured to traverse the character feature sequence based on the first sequence direction, and perform the following processing on the t-th character feature traversed, where T≥t>1, and t is an integer, and T represents the number of character features in the character feature sequence: extract semantic features from the t-th character feature based on the (t-1)-th first semantic feature to obtain the t-th first semantic feature; and determine the T first semantic features obtained by traversing the character feature sequence based on the first sequence direction as the first semantic feature sequence.

[0019] In this embodiment of the application, the feature extraction module is further configured to perform semantic feature extraction by combining the (t-1)th first semantic feature and the tth character feature of the (t-1)th character feature to obtain first reset gate information and first update gate information; calculate the state of the first candidate memory unit by combining the first reset gate information, the (t-1)th semantic feature and the tth character feature; and determine the tth first semantic feature by combining the first update gate information, the state of the first candidate memory unit and the (t-1)th first semantic feature.

[0020] In this embodiment of the application, the feature extraction module is further configured to traverse the character feature sequence based on the second sequence direction, and perform the following processing on the s-th character feature traversed, where T≥s>1 and s is an integer: extract semantic features from the s-th character feature based on the (s-1)-th second semantic feature of the (s-1)-th character feature to obtain the s-th second semantic feature; and determine the T second semantic features obtained by traversing the character feature sequence based on the second sequence direction as the second semantic feature sequence.

[0021] In this embodiment of the application, the feature extraction module is further configured to perform semantic feature extraction by combining the (s-1)th second semantic feature of the (s-1)th character feature and the sth character feature to obtain second reset gate information and second update gate information; calculate the state of the second candidate memory unit by combining the second reset gate information, the (s-1)th semantic feature and the sth character feature; and determine the sth second semantic feature by combining the second update gate information, the state of the second candidate memory unit and the (s-1)th second semantic feature.

[0022] In this embodiment of the application, the code detection module is further configured to: determine a corresponding target first semantic feature from the first semantic feature sequence and a corresponding target second semantic feature from the second semantic feature sequence for a target character feature in the character feature sequence, wherein the target character feature is any character feature in the character feature sequence; combine the target first semantic feature and the target second semantic feature to determine a target semantic feature; obtain a target semantic feature sequence corresponding to the character feature sequence from the target semantic feature of the target character feature; and perform code detection based on the target semantic feature sequence to obtain the code detection result.

[0023] In this embodiment of the application, the code detection module is further configured to obtain a first fusion result of the first state weight in the first sequence direction and the first semantic feature of the target; obtain a second fusion result of the second state weight in the second sequence direction and the second semantic feature of the target; and determine the target semantic feature by combining the first fusion result and the second fusion result.

[0024] In this embodiment of the application, the feature extraction module is further configured to perform embedding representation on the code character sequence of the semantic slice to be detected to obtain an embedding representation sequence; perform attention processing on the embedding representation sequence to obtain a feature sequence to be transformed; and perform nonlinear transformation on the feature sequence to be transformed to obtain the character feature sequence.

[0025] In this embodiment of the application, the feature extraction module is further configured to perform character embedding representation on the code character sequence of the semantic slice to be detected to obtain a character embedding sequence; perform segment embedding representation on the code character sequence to obtain a segment embedding sequence; perform position embedding representation on the code character sequence to obtain a position embedding sequence; and combine the character embedding sequence, the segment embedding sequence and the position embedding sequence to obtain the embedding representation sequence.

[0026] In this embodiment, the feature extraction module is further configured to perform matrix transformation on the embedded representation sequence to obtain query features, key features, and value features; perform attention processing on the query features, key features, and value features based on the linear transformation matrix of the specified mapping space to obtain attention features; obtain at least one attention feature corresponding to at least one specified mapping space from the attention features of the specified mapping space; and obtain the feature sequence to be transformed based on at least one attention feature.

[0027] In this embodiment of the application, the code data processing device further includes a character segmentation module, used to denoise the semantic slice to be detected to obtain a semantic slice to be mapped; map the custom information in the semantic slice to be mapped to a specified symbol representation to obtain a semantic slice to be segmented, wherein the custom information includes at least one of the following: string content, function name, and variable name; and perform code character segmentation on the semantic slice to be segmented to obtain the code character sequence.

[0028] In this embodiment of the application, the slice extraction module is further configured to perform forward slice extraction on the code to be detected to obtain a forward semantic slice; perform backward slice extraction on the code to be detected to obtain a backward semantic slice; and integrate the forward semantic slice and the backward semantic slice into the semantic slice to be detected.

[0029] In this embodiment, the code detection result is obtained through a code detection model; the code data processing device further includes a model training module for acquiring training data, the training data including semantic slice samples and code detection annotation information of the semantic slice samples; code detection is performed on the semantic slice samples using a model to be trained to obtain code detection prediction information, the model to be trained being a neural network model to be trained for code detection; based on the difference between the code detection prediction information and the code detection annotation information, the model to be trained is trained to obtain the code detection model.

[0030] In this embodiment of the application, the code data processing device further includes a code repair module, which is used to determine the semantic slice to be detected as code to be repaired when the code detection result indicates that the semantic slice to be detected contains abnormal code; to display repair prompt information for the code to be repaired; and to perform code repair on the code to be repaired in response to the repair operation of the repair prompt information.

[0031] This application provides an electronic device for code data processing, the electronic device comprising:

[0032] Memory is used to store executable instructions or computer programs.

[0033] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the code data processing method provided in the embodiments of this application.

[0034] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the code data processing method provided in this application.

[0035] This application provides a computer program product, including computer-executable instructions or a computer program, which, when executed by a processor, implements the code data processing method provided in this application.

[0036] The embodiments of this application have at least the following beneficial effects: When performing code detection on the code to be detected, the semantic slice to be detected is first extracted from the code to be detected, and the character feature sequence of the code character sequence of the semantic slice to be detected is extracted, thereby realizing the extraction of the features of the code characters; then, based on the character feature sequence, semantic features are extracted from two opposite sequence directions respectively, thereby realizing the extraction of global semantic features of the code characters from the entire semantic slice to be detected, improving the accuracy of feature extraction of the semantic slice to be detected; thus, by comprehensively performing code detection on the semantic feature sequences extracted separately, the accuracy of code detection can be improved. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the architecture of the code data processing system provided in the embodiments of this application;

[0038] Figure 2 This is one of the embodiments provided in this application. Figure 1 A schematic diagram of the terminal structure in the diagram;

[0039] Figure 3 This is a flowchart illustrating the code data processing method provided in the embodiments of this application. Figure 1 ;

[0040] Figure 4 This is a flowchart illustrating the code data processing method provided in the embodiments of this application. Figure 2 ;

[0041] Figure 5 This is a flowchart illustrating the code data processing method provided in the embodiments of this application. Figure 3 ;

[0042] Figure 6 This is a schematic diagram of an exemplary model training process provided in an embodiment of this application;

[0043] Figure 7 This is a flowchart illustrating the code data processing method provided in the embodiments of this application. Figure 4 ;

[0044] Figure 8 This is an exemplary code detection diagram provided in an embodiment of this application;

[0045] Figure 9 This is a schematic diagram of the structure of a bidirectional encoder model of an exemplary converter provided in an embodiment of this application;

[0046] Figure 10 This is a schematic diagram of the output of an exemplary embedding layer provided in an embodiment of this application;

[0047] Figure 11 This is a schematic diagram of an exemplary single encoder provided in an embodiment of this application;

[0048] Figure 12 This is an exemplary matrix transformation diagram provided in an embodiment of this application;

[0049] Figure 13 This is a schematic diagram of the structure of an exemplary bidirectional gated loop unit model provided in an embodiment of this application;

[0050] Figure 14 This is a schematic diagram of the structure of an example gated loop unit provided in an embodiment of this application;

[0051] Figure 15 This is a flowchart illustrating the training process of an example hybrid neural network model provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] In the following description, the terms “first, second, third, fourth” are used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first, second, third, fourth” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0055] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0056] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0057] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0058] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0059] 1) Artificial neural networks are mathematical models that mimic the structure and function of biological neural networks. Exemplary structures of artificial neural networks in this application include Graph Convolutional Networks (GCNs, a type of neural network for processing graph-structured data), Deep Neural Networks (DNNs), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Neural State Machines (NSMs), and Phase-Functioned Neural Networks (PFNNs). In this application embodiment, code detection can be achieved using the neural network model corresponding to the artificial neural network.

[0060] 2) Semantic slicing refers to the code segments extracted from the code that can fully represent the semantics of the code; for example, code segments that represent function call flow, code segments that represent loop control, and so on.

[0061] It should be noted that, for code detection, the code is typically segmented into fixed-length segments, and detection is performed based on these segments. However, segmenting the code into fixed-length segments in the above-mentioned code detection process affects the accuracy of code feature acquisition, and thus affects the accuracy of code detection.

[0062] In addition, deep learning models (such as FastText, TextCNN, TextRNN, and BERT) can be used for code detection. However, when using deep learning models for code detection, they often process fixed-length code snippets, which are independent of each other. This affects the acquisition of contextual information in the code. Furthermore, the fact that the deep learning model is only used to process fixed-length code snippets affects the accuracy of code feature acquisition, and consequently, the accuracy of code detection.

[0063] Based on this, embodiments of this application provide a code data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the accuracy of code detection. The following describes exemplary applications of the electronic device for code data processing (hereinafter referred to as the code data processing device) provided in embodiments of this application. The code data processing device provided in embodiments of this application can be implemented as various types of terminals such as robots, smartphones, smartwatches, laptops, tablets, desktop computers, smart home appliances, set-top boxes, smart vehicle devices, portable music players, personal digital assistants, dedicated messaging devices, intelligent voice interaction devices, portable gaming devices, and smart speakers. It can also be implemented as a server, or as a combination of a terminal and a server. The following will describe exemplary applications when the code data processing device is implemented as a terminal.

[0064] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the code data processing system provided in the embodiments of this application; as shown... Figure 1 As shown, to support a code data processing application, in the code data processing system 100, a terminal 400 (terminals 400-1 and 400-2 are shown as examples) connects to a server 200 via a network 300. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. The server 200 provides code detection services to the terminal 400 via the network 300. This code detection service can be sending code detection results to the terminal 400 via the network 300, or it can be distributing code detection models to the terminal 400 via the network 300. This embodiment does not limit the specific implementation of this service. Furthermore, the code data processing system 100 also includes a database 500 for providing data support to the server 200. Figure 1 The example shown illustrates a scenario where the database 500 is independent of the server 200. However, the database 500 can also be integrated into the server 200, and this embodiment does not limit this to any particular case.

[0065] Terminal 400 is used to extract a semantic slice to be detected from the code to be detected; to extract character features from the character sequence of the code of the semantic slice to be detected, to obtain a character feature sequence; to extract semantic features from the character feature sequence based on a first sequence direction, to obtain a first semantic feature sequence; to extract semantic features from the character feature sequence based on a second sequence direction, to obtain a second semantic feature sequence, wherein the first sequence direction is opposite to the second sequence direction; to perform code detection by combining the first semantic feature sequence and the second semantic feature sequence, to obtain the code detection result, and to display the code detection result (graphical interfaces 400-11 and 400-21 are shown as examples).

[0066] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0067] See Figure 2 , Figure 2 This is one of the embodiments provided in this application. Figure 1 A schematic diagram of the terminal structure in the diagram; such as Figure 2 As shown, terminal 400 includes at least one processor 410, memory 450, at least one network interface 420, and user interface 430. The various components in terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0068] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0069] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0070] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0071] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0072] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0073] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0074] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, Wi-Fi, and Universal Serial Bus (USB), etc.

[0075] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0076] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0077] In some embodiments, the code data processing apparatus provided in this application can be implemented in software. Figure 2 A code data processing device 455 stored in memory 450 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a slice extraction module 4551, a feature extraction module 4552, a code detection module 4553, a character segmentation module 4554, a model training module 4555, and a code repair module 4556. These modules are logically connected and can therefore be arbitrarily combined or further split according to their implemented functions. The functions of each module will be described below.

[0078] In some embodiments, the code data processing apparatus provided in this application can be implemented in hardware. As an example, the code data processing apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the code data processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0079] In some embodiments, the terminal or server can implement the code data processing method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as code detection APPs; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0080] The code data processing method provided in this application will be described below with reference to exemplary applications and implementations of the code data processing device provided in the embodiments of this application. Furthermore, the code data processing method provided in the embodiments of this application is applicable to various code detection scenarios such as cloud technology, artificial intelligence, smart transportation, and games.

[0081] See Figure 3 , Figure 3 This is a flowchart illustrating the code data processing method provided in the embodiments of this application. Figure 1 ,in, Figure 3 The main body executing each step is the code and data processing device; the following will combine... Figure 3 The steps shown are explained.

[0082] Step 101: Extract the semantic slices to be detected from the code to be detected.

[0083] In this embodiment, when the code is completed or a code detection request is received, the code data processing device takes the completed code or the code requesting code detection as the code to be detected, thus obtaining the code to be detected. To perform code detection on the code to be detected, the code data processing device extracts slices from the code to be detected, and the extracted slices are called semantic slices to be detected. Here, the code data processing device may extract a single semantic slice from the code to be detected and determine that single semantic slice as the semantic slice to be detected; it may also divide the entire code to be detected into multiple semantic slices and call each of the divided semantic slices the semantic slice to be detected; or it may divide a portion of the code to be detected into multiple semantic slices and call each of the divided semantic slices the semantic slice to be detected; and so on. This embodiment does not limit the specific implementation.

[0084] It should be noted that the code to be detected is the code to be inspected. This can be code written in real-time, code written historically, or a combination of both, etc. This application embodiment does not limit this. Code inspection, also known as code review, is a code quality control method used to identify defects, errors, information deviating from coding standards, and performance issues in the code. Code anomalies refer to situations that affect the normal operation of the code, such as buffer overflows, running efficiency lower than a specified running efficiency, and code syntax errors. The semantic slice to be detected is the semantic slice to be inspected. This can be a semantic slice obtained through forward slice extraction, a semantic slice obtained through backward slice extraction, or a semantic slice obtained through both forward and backward slice extraction. This application embodiment does not limit this.

[0085] See Figure 4 , Figure 4 This is a flowchart illustrating the code data processing method provided in the embodiments of this application. Figure 2 ,in, Figure 4 The execution entity for each step is the code and data processing device; such as Figure 4 As shown in the embodiment of this application, step 101 can be implemented by steps 1011 to 1013; that is, the code data processing device extracts the semantic slice to be detected from the code to be detected, including steps 1011 to 1013. Each step is described below.

[0086] Step 1011: Extract forward slices from the code to be detected to obtain forward semantic slices.

[0087] In this embodiment of the application, the code data processing device first determines the target information to be extracted for semantic slicing in the code to be detected. Then, starting from the target information, it determines the set of all variables and code statements that affect the target information, and the set of variables and code statements that affect the target information is called the forward semantic slice.

[0088] It should be noted that forward slice extraction refers to the process of identifying the set of variables and code statements that affect the target information, starting from the target information in the code to be detected; in other words, forward slice extraction is the process of identifying the set of variables and code statements that affect the target information before it is detected. Forward semantic slicing refers to the semantic slices obtained by performing forward slice extraction on the target information in the code to be detected; where the target information is the information to be sliced, such as variables, code statements, etc.

[0089] Step 1012: Extract backward slices from the code to be detected to obtain backward semantic slices.

[0090] In this embodiment of the application, when the target information to be extracted for semantic slicing is determined in the code to be detected, starting from the target information, a set of all variables and code statements affected by the target information is determined, and the set of variables and code statements affected by the target information is called backward semantic slice.

[0091] It should be noted that backward slice extraction refers to the process of identifying the set of variables and code statements affected by the target information in the code to be detected; in other words, backward slice extraction is the process of identifying the set of variables and code statements affected by the target information after it has been determined. Backward semantic slicing refers to the semantic slices obtained by backward slice extraction of the target information in the code to be detected.

[0092] In the embodiments of this application, steps 1011 and 1012 are not sequential in execution; here, steps 1011 and 1012 can be executed synchronously, sequentially, or alternately, etc., and the embodiments of this application do not limit this.

[0093] Step 1013: Integrate the forward semantic slice and the backward semantic slice into a semantic slice to be detected.

[0094] In this embodiment of the application, after the code data processing device obtains the forward semantic slice and the backward semantic slice, it integrates the forward semantic slice and the backward semantic slice, and the integrated result is the semantic slice to be detected; thus, the semantic slice to be detected represents the integrated result of the forward semantic slice and the backward semantic slice.

[0095] It is understandable that by employing both forward slice extraction and backward slice extraction methods, a complete semantic slice to be detected can be obtained; and by performing code detection on the complete semantic slice to be detected, the accuracy of code detection can be improved.

[0096] Step 102: Extract character features from the code character sequence of the semantic slice to be detected to obtain a character feature sequence.

[0097] In this embodiment, in order to perform code detection on the semantic slice to be detected, the code data processing device first represents the semantic slice to be detected as a sequence of code characters, and then performs feature extraction on the sequence of code characters to extract the character-level features of each code character in the sequence of code characters, thus obtaining the character features corresponding to each code character; after the code data processing device extracts the character features for each code character in the sequence of code characters, it constructs a sequence based on the extracted character features of the sequence of code characters, thus obtaining the character feature sequence.

[0098] It's important to note that the code character sequence is obtained by segmenting the semantic slice to be detected into code characters. Code characters are character representations used in code writing; a code character is a character representation within the semantic slice to be detected, such as a variable, function, punctuation mark, data type field, etc. Character feature extraction refers to extracting character-level features. There is a one-to-one correspondence between the character feature sequence and the code character sequence; that is, one character feature in the character feature sequence corresponds to one code character in the code character sequence, or one code character in the code character sequence corresponds to one character feature in the character feature sequence. Furthermore, the character features are the character-level features of the code characters.

[0099] See also Figure 4 In this embodiment of the application, step 102 can be implemented by steps 1021 to 1023; that is, the code data processing device extracts character features from the code character sequence of the semantic slice to be detected to obtain a character feature sequence, including steps 1021 to 1023. Each step is described below.

[0100] Step 1021: Embed the code character sequence of the semantic slice to be detected to obtain the embedding representation sequence.

[0101] In this embodiment, after obtaining the code character sequence of the semantic slice to be detected, the code data processing device first performs an embedding representation on each code character in the code character sequence to obtain a data representation for each code character to extract character features. The data representation for character feature extraction corresponding to each code character is called the embedding representation. Thus, the code data processing device can obtain an embedding representation sequence that corresponds one-to-one with the code character sequence for the code character sequence.

[0102] It should be noted that there is a one-to-one correspondence between the embedding representation sequence and the code character sequence. That is, one embedding representation in the embedding representation sequence corresponds to one code character in the code character sequence, or one code character in the code character sequence corresponds to one embedding representation in the embedding representation sequence.

[0103] In this embodiment of the application, the code data processing device performs an embedding representation on the code character sequence of the semantic slice to be detected to obtain an embedding representation sequence, including: the code data processing device performs a character embedding representation on the code character sequence of the semantic slice to be detected to obtain a character embedding sequence; performs a segment embedding representation on the code character sequence to obtain a segment embedding sequence; performs a position embedding representation on the code character sequence to obtain a position embedding sequence; and combines the character embedding sequence, the segment embedding sequence, and the position embedding sequence to obtain an embedding representation sequence.

[0104] It should be noted that the embedding representation may include character embedding representation, and may also include at least one of character embedding representation, separator embedding representation, and positional embedding representation; the embodiments of this application do not limit this. Specifically, character embedding representation refers to embedding code characters, and the code data processing device refers to the embedding representation of code characters as character embedding; separator embedding representation refers to segmenting code statements in a code character sequence, and the code data processing device refers to the segmentation representation of code statements in a code character sequence as separator embedding; positional embedding representation refers to embedding the position of code characters in a code character sequence, and the code data processing device refers to the embedding representation of the position of code characters in a code character sequence as positional embedding. Therefore, when the embedding representation includes character embedding representation, the code data processing device can use the character embedding sequence as the embedding representation sequence; when the embedding representation includes character embedding representation and separator embedding representation, the code data processing device can use the character embedding sequence and separator embedding sequence as the embedding representation sequence; when the embedding representation includes character embedding representation and positional embedding representation, the code data processing device can use the character embedding sequence and positional embedding sequence as the embedding representation sequence; when the embedding representation includes character embedding representation, separator embedding representation, and positional embedding representation, the code data processing device can use the character embedding sequence, separator embedding sequence, and positional embedding sequence as the embedding representation sequence.

[0105] It should also be noted that there is a one-to-one correspondence between character embedding sequences and code character sequences. That is, one character embedded in a character embedding sequence corresponds to one code character in the code character sequence, or one code character in the code character sequence corresponds to one character embedding in a character embedding sequence. Similarly, there is a one-to-one correspondence between delimiter embedding sequences and code character sequences. That is, one delimiter embedded in a delimiter embedding sequence corresponds to one code character in the code character sequence, or one code character in the code character sequence corresponds to one delimiter embedding in a delimiter embedding sequence. Finally, there is a one-to-one correspondence between positional embedding sequences and code character sequences. That is, one positional embedding in a positional embedding sequence corresponds to one code character in the code character sequence, or one code character in the code character sequence corresponds to one positional embedding in a positional embedding sequence.

[0106] It is understandable that when embedding representation of code character sequences, character embedding representation is used as the basis, combined with segmentation embedding representation and position embedding representation, to achieve the segmentation of code statements and the combination of the positions of code characters in the code character sequence, thereby improving the accuracy of character feature extraction of code character sequences.

[0107] Step 1022: Perform attention processing on the embedded representation sequence to obtain the feature sequence to be transformed.

[0108] In this embodiment, after obtaining the embedded representation sequence, the code data processing device performs attention processing on the embedded representation sequence to extract features related to code detection and remove features unrelated to code detection. The feature sequence to be transformed is the attention processing result of the embedded representation sequence.

[0109] It should be noted that attention processing, also known as an attention mechanism, refers to a method that simulates human attention concentration. It assigns different weights based on the importance of different parts of the embedded representation sequence, thus focusing more intently on the features used for code detection compared to the embedded representation sequence itself. In other words, attention processing assigns different weights to different parts of the embedded representation sequence for targeted processing. Here, attention processing can be self-attention, multi-head attention, or a combination of both, etc., and this application does not limit this approach. Here, when performing attention processing, the code data processing device can combine the attention processing result of the embedded representation sequence with the embedded representation sequence itself to form a feature sequence to be transformed.

[0110] In this embodiment of the application, the code data processing device performs attention processing on the embedded representation sequence to obtain a feature sequence to be transformed, including: the code data processing device first performs matrix transformation on the embedded representation sequence to obtain query features, key features, and value features; then, based on the linear transformation matrix of a specified mapping space, it performs attention processing on the query features, key features, and value features to obtain attention features; finally, from the attention features of the specified mapping space, it obtains at least one attention feature corresponding to at least one specified mapping space; and based on the at least one attention feature, it obtains the feature sequence to be transformed.

[0111] It should be noted that the code data processing device performs matrix transformations on the embedded representation sequence using specified transformation parameters, including query transformation parameters, key transformation parameters, and value transformation parameters. Therefore, the code data processing device performs matrix transformations on the embedded representation sequence using query transformation parameters, and the fusion result of the query transformation parameters and the embedded representation sequence is called the query feature; it performs matrix transformations on the embedded representation sequence using key transformation parameters, and the fusion result of the key transformation parameters and the embedded representation sequence is called the key feature; and it performs matrix transformations on the embedded representation sequence using value transformation parameters, and the fusion result of the value transformation parameters and the embedded representation sequence is called the value feature. When performing attention processing on the query features, key features, and value features, the code data processing device calculates attention scores based on the query features, key features, and value features. The attention score represents the association strength between the query feature and each key feature. Then, scaling processing is performed based on the attention score, and the scaling result is normalized to obtain the attention weight, thus completing the weight allocation. Finally, the fusion result of the attention weight and the embedded representation sequence is called the attention feature. When the code data processing device uses self-attention processing, it obtains one attention feature, while when the code data processing device uses multi-head attention processing, it obtains multiple attention features. Thus, the code data processing device can obtain at least one attention feature and an integrated result of the feature sequence to be transformed into at least one attention feature.

[0112] It is understandable that by applying attention processing to the embedding representation sequence, dynamic weight allocation is achieved for different parts of the embedding representation sequence, which enhances the importance of features related to code detection in the embedding representation sequence and reduces the importance of features unrelated to clipping detection in the embedding representation sequence, thereby increasing the importance of features in the embedding representation sequence that are helpful for code detection.

[0113] Step 1023: Perform a nonlinear transformation on the feature sequence to be transformed to obtain the character feature sequence.

[0114] In this embodiment of the application, the code data processing device is also used to perform a nonlinear transformation on the feature sequence to be transformed, so as to extract an accurate character feature sequence from the feature sequence to be transformed.

[0115] It should be noted that the code data processing device can use a feedforward neural network layer to perform a nonlinear transformation on the feature sequence to be transformed, or it can use specified nonlinear transformation parameters to perform a nonlinear transformation on the feature sequence to be transformed, etc., and this application embodiment does not limit this. Here, when performing a nonlinear transformation, the code data processing device can combine the feature sequence to be transformed and the nonlinear transformation result of the feature sequence to be transformed into a character feature sequence.

[0116] In the embodiments of this application, attention processing and nonlinear transformation can be performed once or iteratively, and this application does not limit the specific steps. When attention processing and nonlinear transformation are performed iteratively, after obtaining the nonlinear transformation result of the feature sequence to be transformed, the code data processing device performs attention processing and nonlinear transformation on the nonlinear transformation result of the feature sequence to be transformed until the iteration ends, at which point the final nonlinear transformation result is determined as the character feature sequence. The iteration end condition can be a specified iteration duration, a specified number of iterations, or a combination of the above, etc., and this application does not limit the specific steps.

[0117] Step 103: Extract semantic features from the character feature sequence based on the first sequence direction to obtain the first semantic feature sequence.

[0118] In this embodiment, the code data processing device is further configured to extract semantic features from the character feature sequence using different sequence directions. These different sequence directions include a first sequence direction and a second sequence direction, with the first sequence direction being opposite to the second sequence direction. Here, when the code data processing device extracts semantic features from the character feature sequence using the first sequence direction, the semantic feature corresponding to each character feature in the extracted character feature sequence is called the first semantic feature, thus obtaining a first semantic feature sequence that corresponds one-to-one with the character feature sequence.

[0119] It should be noted that semantic feature extraction is used to extract semantic information between code statements in the semantic slice to be detected. There is a one-to-one correspondence between the character feature sequence and the first semantic feature sequence. That is, one character feature in the character feature sequence corresponds to one first semantic feature in the first semantic feature sequence, or one first semantic feature in the first semantic feature sequence corresponds to one character feature in the character feature sequence.

[0120] See also Figure 4In this embodiment of the application, step 103 can be implemented by steps 1031 to 1033; that is, the code data processing device extracts semantic features from the character feature sequence based on the first sequence direction to obtain the first semantic feature sequence, including steps 1031 to 1033. Each step is described below.

[0121] Step 1031: Traverse the character feature sequence based on the first sequence direction, and execute step 1032 for the t-th character feature encountered.

[0122] In this embodiment of the application, the code data processing device performs semantic feature extraction for each character feature in the character feature sequence, and the semantic feature extraction is based on the semantic features transmitted from the first sequence direction; thus, the code data processing device traverses the character feature sequence based on the first sequence direction and performs semantic feature extraction for the t-th character feature traversed.

[0123] It should be noted that T ≥ t > 1, and t is an integer, where T represents the number of character features in the character feature sequence; therefore, the acquisition of the corresponding semantic feature is only performed in step 1032 when the traversed character feature is a character feature after the first character feature. Here, when the traversed character feature is the first character feature, the semantic feature extracted from the first character feature is used as the first semantic feature of the first character feature.

[0124] Step 1032: Extract semantic features from the t-th character feature based on the (t-1)-th first semantic feature to obtain the t-th first semantic feature.

[0125] It should be noted that when the code data processing device traverses to the t-th character feature, it has already obtained the (t-1)-th first semantic feature of the (t-1)-th character feature. Therefore, the code data processing device extracts semantic features of the t-th character feature based on the (t-1)-th first semantic feature, and the obtained semantic feature of the t-th character feature is called the t-th first semantic feature.

[0126] In this embodiment of the application, the code data processing device extracts semantic features from the t-th character feature based on the (t-1)-th first semantic feature of the (t-1)-th character feature to obtain the t-th first semantic feature. This includes: the code data processing device first extracts semantic features by combining the (t-1)-th first semantic feature of the (t-1)-th character feature and the t-th character feature to obtain first reset gate information and first update gate information; then, it calculates the state of the first candidate memory unit by combining the first reset gate information, the (t-1)-th first semantic feature, and the t-th character feature; finally, it determines the t-th first semantic feature by combining the first update gate information, the state of the first candidate memory unit, and the (t-1)-th first semantic feature.

[0127] It should be noted that the code data processing device integrates the (t-1)th first semantic feature and the tth character feature using reset gate parameters, and processes the integration result using a reset gate activation function, thus obtaining the first reset gate information. The code data processing device integrates the (t-1)th first semantic feature and the tth character feature using update gate parameters, and processes the integration result using an update gate activation function, thus obtaining the first update gate information. The code data processing device first obtains the fusion result of the first reset gate information and the (t-1)th first semantic feature, then integrates the fusion result of the first reset gate information and the (t-1)th first semantic feature, along with the tth character feature, using candidate memory unit state parameters, and processes the integration result using a first candidate memory unit state activation function, thus obtaining the first candidate memory unit state. The code data processing device acquires information negatively correlated with the first update gate information, and acquires the third fusion result of the information negatively correlated with the first update gate information and the (t-1)th first semantic feature, and acquires the fourth fusion result of the first update gate information and the state of the first candidate memory unit. By integrating the third fusion result and the fourth fusion result, the tth first semantic feature is obtained.

[0128] Step 1033: Determine the T first semantic features obtained by traversing the character feature sequence based on the first sequence direction as the first semantic feature sequence.

[0129] It should be noted that when the code data processing device traverses the character feature sequence based on the first sequence direction, it can obtain T-1 first semantic features corresponding to the T-1 character features after the first traversed character feature using step 1032, and it can also obtain the first semantic feature of the first traversed character feature. Therefore, when traversing the character feature sequence based on the first sequence direction, it can obtain T first semantic features. Here, the code data processing device constructs a first semantic feature sequence that corresponds one-to-one with the character feature sequence based on the sequence order in the character feature sequence.

[0130] Step 104: Extract semantic features from the character feature sequence based on the second sequence direction to obtain the second semantic feature sequence.

[0131] In this embodiment of the application, when the code data processing device performs semantic feature extraction on the character feature sequence using the second sequence direction, the semantic feature corresponding to each character feature in the extracted character feature sequence is called the second semantic feature, thus obtaining the second semantic feature sequence that corresponds one-to-one with the character feature sequence.

[0132] It should be noted that there is a one-to-one correspondence between the character feature sequence and the second semantic feature sequence. That is, one character feature in the character feature sequence corresponds to one second semantic feature in the second semantic feature sequence, or one second semantic feature in the second semantic feature sequence corresponds to one character feature in the character feature sequence. Furthermore, steps 103 and 104 are not sequential in execution; they can be executed synchronously, sequentially, or alternately, etc., and this embodiment does not limit the execution order.

[0133] See also Figure 4 In this embodiment of the application, step 104 can be implemented by steps 1041 to 1043; that is, the code data processing device extracts semantic features from the character feature sequence based on the second sequence direction to obtain the second semantic feature sequence, including steps 1041 to 1043. Each step is described below.

[0134] Step 1041: Traverse the character feature sequence based on the second sequence direction, and execute step 1042 for the s-th character feature encountered.

[0135] In this embodiment of the application, the code data processing device performs semantic feature extraction for each character feature in the character feature sequence, and the semantic feature extraction is based on the semantic features transmitted from the second sequence direction; thus, the code data processing device traverses the character feature sequence based on the second sequence direction and performs semantic feature extraction for the s-th character feature traversed.

[0136] It should be noted that T ≥ s > 1, and s is an integer; therefore, the acquisition of the corresponding semantic feature is only performed in step 1042 when the traversed character feature is a character feature after the first character feature. Here, when the traversed character feature is the first character feature, the semantic feature extracted from the first character feature is used as the second semantic feature of the first character feature.

[0137] Step 1042: Extract semantic features from the s-th character feature based on the (s-1)-th second semantic feature of the (s-1)-th character feature to obtain the s-th second semantic feature.

[0138] It should be noted that when the code data processing device traverses to the s-th character feature, it has already obtained the s-1-th second semantic feature of the (s-1)-th character feature. Therefore, the code data processing device extracts semantic features from the s-th character feature based on the (s-1)-th second semantic feature, and the obtained semantic feature of the s-th character feature is called the s-th second semantic feature.

[0139] In this embodiment of the application, the code data processing device extracts semantic features from the s-th character feature based on the (s-1)-th second semantic feature of the (s-1)-th character feature to obtain the s-th second semantic feature. This includes: the code data processing device first extracts semantic features by combining the (s-1)-th second semantic feature of the (s-1)-th character feature and the s-th character feature to obtain second reset gate information and second update gate information; then, it calculates the state of the second candidate memory unit by combining the second reset gate information, the (s-1)-th semantic feature, and the s-th character feature; finally, it determines the s-th second semantic feature by combining the second update gate information, the state of the second candidate memory unit, and the (s-1)-th second semantic feature.

[0140] It should be noted that the code data processing device integrates the (s-1)th second semantic feature and the sth character feature using reset gate parameters, and processes the integration result using a reset gate activation function, thus obtaining the second reset gate information. The code data processing device integrates the (s-1)th second semantic feature and the sth character feature using update gate parameters, and processes the integration result using an update gate activation function, thus obtaining the second update gate information. The code data processing device first obtains the fusion result of the second reset gate information and the (s-1)th second semantic feature, then integrates the fusion result of the second reset gate information and the (s-1)th second semantic feature, along with the sth character feature, using candidate memory unit state parameters, and processes the integration result using a second candidate memory unit state activation function, thus obtaining the second candidate memory unit state. The code data processing device acquires information negatively correlated with the second update gate information, and acquires the third fusion result of the information negatively correlated with the second update gate information and the (s-1)th second semantic feature, as well as the fourth fusion result of the second update gate information and the state of the second candidate memory unit. By integrating the third fusion result and the fourth fusion result, the sth second semantic feature is obtained.

[0141] It should be noted that both the first and second update gate information are update gate information, used to determine the extent to which semantic features from the previous time step are passed to the current time step. Both the first and second reset gate information are reset gate information, used to determine the degree of integration between the semantic features from the previous time step and the semantic features of the character features at the current time step. Both the first and second candidate memory unit states are candidate memory unit states, referring to the intermediate states used to generate the semantic features at the current time step.

[0142] Step 1043: Determine the T second semantic features obtained by traversing the character feature sequence based on the second sequence direction as the second semantic feature sequence.

[0143] It should be noted that when the code data processing device traverses the character feature sequence based on the second sequence direction, it can obtain T-1 second semantic features corresponding to the T-1 character features after the first traversed character feature using step 1042, and it can also obtain the second semantic feature of the first traversed character feature. Therefore, when traversing the character feature sequence based on the second sequence direction, it can obtain T second semantic features. Here, the code data processing device constructs a second semantic feature sequence that corresponds one-to-one with the character feature sequence based on the sequence order in the character feature sequence.

[0144] Step 105: Combine the first semantic feature sequence and the second semantic feature sequence to perform code detection and obtain the code detection result.

[0145] It should be noted that the code data processing device combines the first semantic feature sequence and the second semantic feature sequence to perform code detection. The obtained code detection result can indicate that there is code anomaly in the semantic slice sample to be detected, or it can indicate that the semantic slice sample to be detected is normal code.

[0146] See also Figure 4 In this embodiment of the application, step 105 can be implemented by steps 1051 to 1054; that is, the code data processing device combines the first semantic feature sequence and the second semantic feature sequence to perform code detection and obtain code detection results, including steps 1051 to 1054. Each step is described below.

[0147] Step 1051: For the target character features in the character feature sequence, determine the corresponding target first semantic feature from the first semantic feature sequence, and determine the corresponding target second semantic feature from the second semantic feature sequence.

[0148] It should be noted that the target character feature is any character feature in the character feature sequence; for the target character feature, the code data processing device can determine a first semantic feature based on the sequence correspondence from the first semantic feature sequence, and the determined first semantic feature is called the target first semantic feature; and can determine a second semantic feature based on the sequence correspondence from the second semantic feature sequence, and the determined second semantic feature is called the target second semantic feature.

[0149] Step 1052: Combine the first semantic feature of the target and the second semantic feature of the target to determine the semantic feature of the target.

[0150] It should be noted that the code data processing device can combine the first and second semantic features of the target by concatenating them, or by performing a weighted summation of the first and second semantic features, etc. This application embodiment does not limit this approach. Here, the code data processing device refers to the result of combining the first and second semantic features of the target as the target semantic feature.

[0151] In this embodiment of the application, the code data processing device determines the target semantic features by combining the target first semantic features and the target second semantic features, including: the code data processing device first obtains a first fusion result of the first state weight in the first sequence direction and the target first semantic features; and obtains a second fusion result of the second state weight in the second sequence direction and the target second semantic features; and finally, combines the first fusion result and the second fusion result to determine the target semantic features.

[0152] It should be noted that a corresponding fusion weight, namely the first state weight, is pre-set for the first sequence direction. Therefore, the code data processing device fuses the first state weight with the target first semantic feature, and the fusion result of the first state weight and the target first semantic feature is called the first fusion result. Similarly, a corresponding fusion weight, namely the second state weight, is pre-set for the second sequence direction. Therefore, the code data processing device fuses the second state weight with the target second semantic feature, and the fusion result of the second state weight and the target second semantic feature is called the second fusion result. Here, the code data processing device can accumulate the first fusion result and the second fusion result to achieve a combination of the first fusion result and the second fusion result to obtain the target semantic feature. It can also combine the accumulated result of the first fusion result and the second fusion result with a preset bias term to obtain the target semantic feature, etc. This application embodiment does not limit this.

[0153] Step 1053: Obtain the target semantic feature sequence corresponding to the character feature sequence from the target semantic features of the target character features.

[0154] It should be noted that when the corresponding target semantic features are obtained for each target character feature in the character feature sequence, the target semantic feature sequence corresponding to the character feature sequence is also obtained.

[0155] Step 1054: Perform code detection based on the target semantic feature sequence to obtain the code detection results.

[0156] In this embodiment of the application, after the code detection device obtains the target semantic feature sequence by combining the first semantic feature sequence and the second semantic feature sequence, code detection is performed based on the target semantic feature sequence.

[0157] Understandably, when performing code detection on the code to be detected, the semantic slice to be detected is first extracted from the code, and the character feature sequence of the code character sequence of the semantic slice to be detected is extracted, thus realizing the extraction of the features of the code characters. Then, based on the character feature sequence, semantic features are extracted from two opposite sequence directions, thus realizing the extraction of global semantic features of the code characters from the entire semantic slice to be detected, improving the accuracy of feature extraction of the semantic slice to be detected. Therefore, by combining the extracted semantic feature sequences for code detection, the accuracy of code detection can be improved.

[0158] See Figure 5 , Figure 5 This is a flowchart illustrating the code data processing method provided in the embodiments of this application. Figure 3 ,in, Figure 5 The execution entity for each step is the code and data processing device; such as Figure 5 As shown in the embodiment of this application, steps 106 to 108 are included before step 102; that is, before the code data processing device extracts character features from the code character sequence of the semantic slice to be detected and obtains the character feature sequence, the code data processing device further includes steps 106 to 108. Each step will be described below.

[0159] Step 106: Denoise the semantic slice to be detected to obtain the semantic slice to be mapped.

[0160] It should be noted that denoising includes at least one of the following processes: deletion of characters of non-ASCII type, and deletion of comments. Here, the code data processing device refers to the denoised semantic slice to be detected as the semantic slice to be mapped.

[0161] In this embodiment of the application, when denoising includes the deletion of characters of non-Information Interchange Standard Code type, the code data processing device can use regular expressions to identify characters of non-Information Interchange Standard Code type in the semantic slice to be detected, and delete the identified characters of non-Information Interchange Standard Code type; when denoising includes the deletion of comments, the code data processing device can use comment identifiers to identify comments in the semantic slice to be detected, and delete the identified comments.

[0162] Step 107: Map the custom information in the semantic slice to be mapped to the specified symbol representation to obtain the semantic slice to be divided.

[0163] It should be noted that the custom information includes at least one of the following: string content, function name, and variable name; the code data processing device refers to the semantic slice to be mapped after the custom information mapping is completed as the semantic slice to be divided.

[0164] In the embodiments of this application, when the custom information includes string content, the specified symbol represents the specified string; when the custom information includes function name, the specified symbol represents the specified function name and function number; when the custom information includes variable name, the specified symbol represents the specified variable name and variable number.

[0165] Step 108: Divide the semantic slice to be divided into code characters to obtain a code character sequence.

[0166] It should be noted that, in order to represent the semantic slice to be divided as sequence data, the code data processing device divides the semantic slice to be divided into code characters, and constructs a sequence based on the order in which each code character appears in the semantic slice to be divided, thus obtaining the code character sequence.

[0167] It is understandable that by denoising the semantic slice to be detected and mapping the custom information in the semantic slice to be mapped to a specified symbol representation, the interference information in the semantic slice to be detected is reduced, thereby improving the accuracy of feature extraction and code detection.

[0168] In this embodiment, the code detection result can be obtained through a code detection model, which is a neural network model used for code detection; see also Figure 6 , Figure 6 This is a schematic diagram of an exemplary model training process provided in an embodiment of this application; as shown... Figure 6 As shown, the code detection model is trained through steps 109 to 111. Each step is explained below.

[0169] Step 109: Obtain training data, which includes semantic slice samples and code detection annotation information for semantic slice samples.

[0170] It should be noted that the semantic slice sample is the sample to be detected by code detection. The semantic slice sample is similar to the semantic slice to be detected, and will not be described again in this embodiment. The code detection annotation information is the actual result of code detection of the semantic slice sample, which can indicate that there is code anomaly in the semantic slice sample, or it can indicate that the semantic slice sample is normal code.

[0171] Step 110: Use the model to be trained to perform code detection on the semantic slice samples to obtain code detection prediction information.

[0172] It should be noted that the model to be trained is a neural network model used for code detection. It can be the original neural network model that has been constructed, a pre-trained neural network model, or a large model, etc. This application embodiment does not limit this. The process of the code data processing device using the model to be trained to perform code detection on the semantic slice samples is similar to the process of obtaining the code detection results of the semantic slice to be detected, and will not be described again in this application embodiment.

[0173] Step 111: Based on the difference between the code detection prediction information and the code detection annotation information, train the model to be trained to obtain the code detection model.

[0174] In this embodiment, since the model to be trained is a neural network model, the accuracy of the code detection prediction information obtained by the model to be trained needs to be compared and confirmed with the code detection annotation information. Therefore, the code data processing device compares the code detection prediction information with the code detection annotation information and calculates a loss function value based on the difference between the two. Then, backpropagation is performed on the model to be trained based on this loss function value to adjust the model parameters and achieve training. Finally, the trained model is called the code detection model. Backpropagation is a technique used to train neural network models. In a supervised learning environment, backpropagation calculates the gradient of the loss function value with respect to the model parameters and updates the model parameters through gradient descent to minimize the loss function value.

[0175] It should be noted that the training process of the model to be trained can be single-step or iterative, and this application embodiment does not limit this. When the training process of the model to be trained is iterative, the iterative training ends when the iteration termination condition is met, and the model to be trained in the last iteration is determined as the code detection model. The iteration termination condition can be reaching an accuracy threshold, reaching an iteration count threshold, reaching an iteration duration threshold, or a combination of the above, etc., and this application embodiment does not limit this.

[0176] In this embodiment, the loss function value can be calculated using a loss function; the loss function is a function that calculates the loss function value between the code detection prediction information and the code detection annotation information.

[0177] It should be noted that loss functions in machine learning and deep learning are used to quantify the difference between model predictions and actual values; loss functions include mean squared error (MSE), mean absolute error (MAE), cross-entropy loss, and log loss.

[0178] See Figure 7 , Figure 7 This is a flowchart illustrating the code data processing method provided in the embodiments of this application. Figure 4 ,in, Figure 7 The execution entity for each step is the code and data processing device; such as Figure 7 As shown in the embodiment of this application, after step 105, steps 112 to 114 are also included; that is, after the code data processing device performs code detection by combining the first semantic feature sequence and the second semantic feature sequence and obtains the code detection result, the code data processing device further includes steps 112 to 114. Each step is described below.

[0179] Step 112: When the code detection result indicates that the semantic slice to be detected contains abnormal code, the semantic slice to be detected is identified as code to be repaired.

[0180] It should be noted that when the code detection result indicates that the semantic slice to be detected contains abnormal code, it means that the semantic slice to be detected needs to be repaired. Therefore, the code data processing device will identify the semantic slice to be detected as code to be repaired.

[0181] Step 113: Display repair prompts for the code to be repaired.

[0182] It should be noted that when the code detection result indicates that the semantic slice to be detected contains abnormal code, the code data processing device is also used to display repair prompts for the code to be repaired, indicating that the code to be repaired needs to be repaired.

[0183] Step 114: In response to the repair prompt message, perform code repair on the code to be repaired.

[0184] It should be noted that when the displayed repair prompt message is used to repair the code to be repaired, the code data processing device also receives the repair operation in response to the repair prompt message; at this time, the code data processing device responds to the repair operation and performs code repair on the code to be repaired.

[0185] It is understandable that improving the accuracy of code detection can improve the accuracy and efficiency of code repair.

[0186] The following describes an exemplary application of the embodiments of this application in a practical application scenario. This exemplary application describes the process of security detection of code based on a hybrid neural network model, wherein the hybrid neural network model includes character feature extraction and multi-sequence direction semantic feature extraction functions. It is readily understood that the code data processing method provided in this application is applicable to any scenario combining character feature extraction and multi-sequence direction semantic feature extraction for code detection. Here, the process of code detection using a hybrid neural network model including character feature extraction and multi-sequence direction semantic feature extraction functions is used as an example for illustration.

[0187] It should be noted that in the method for code security detection based on a hybrid neural network model provided in this application embodiment, the hybrid neural network model is exemplarily a Bidirectional Encoder Representation from Transformers (BERT) model and a Bidirectional Gated Recurrent Unit (BGRU) model. The BERT model is a context-based pre-trained model that learns the embedding vector of each character through batch corpora, i.e., it learns a character vector representation related to the context; and the BERT model is used for character feature extraction. The BGRU model is a Bidirectional Gated Recurrent Unit (GRU), and GRU is a neural network model commonly used for sequence data modeling, a variant of recurrent neural networks, used to solve long-term dependencies and gradient problems in backpropagation; and the BGRU model is used for semantic feature extraction based on multiple sequence directions.

[0188] In this embodiment of the application, when performing security detection on code based on a hybrid neural network model, the process includes four steps: data preprocessing, feature extraction using the BERT model, feature extraction using the BGRU model, and acquisition of detection results. Each step is described below.

[0189] For example, see Figure 8 , Figure 8 This is an exemplary code detection diagram provided in an embodiment of this application; as shown... Figure 8As shown, when performing code detection based on the hybrid neural network model 8-1 (referred to as the code detection model), the input layer 8-2 is used to obtain the input data corresponding to the code to be detected based on data preprocessing; the bidirectional encoder model 8-11 of the converter in the hybrid neural network model 8-1 is used to implement the feature extraction of the BERT model (referred to as character feature extraction) to obtain a character vector sequence representing the semantic features between characters; the bidirectional gated recurrent unit model 8-12 in the hybrid neural network model 8-1 is used to implement the feature extraction of the BGRU model (referred to as speech feature extraction); the classification output layer 8-3 is used to classify the output results of the bidirectional gated recurrent unit model 8-12 to obtain the code detection results.

[0190] It should be noted that the program source code in the software system is used as the code to be detected. Data preprocessing is performed on the code to be detected to obtain the corresponding input data. In other words, the input layer 8-2 is used to preprocess the code to be detected to obtain the input of the hybrid neural network model 8-1, so that the input of the hybrid neural network model 8-1 is the code to be detected after data preprocessing.

[0191] For example, the input S = {S1, S2, ..., S...} of the hybrid neural network model 8-1 n1}, where S i1 (i1∈[1,n1], where n1 represents the total number of code statements) represents the vector of code statements; and code statements are composed of statement elements such as keywords, variables, and symbols, hence S i ={s1,s2,…,s n2}, where s i2 (i2∈[1,n2], where n2 represents the total number of statement elements in the code statement) represents keywords, variables, or symbols in the code statement.

[0192] It should also be noted that data preprocessing refers to preprocessing the code to be detected before code detection based on the hybrid neural network model, so as to meet the input requirements of the hybrid neural network model.

[0193] In the embodiments of this application, data preprocessing includes slice acquisition, word segmentation, string replacement, symbolization of functions and variables, and slice vectorization. Each process is described below.

[0194] Slice acquisition refers to the process of obtaining bidirectional slices of the code to be detected. Here, forward slice extraction is performed on the code to be detected to obtain the extracted forward slice; backward slice extraction is performed on the code to be detected to obtain the extracted backward slice; finally, the extracted forward slice and the extracted backward slice are concatenated to form a bidirectional slice, which yields the semantic slice (SC) (called the semantic slice to be detected), thus completing the slice acquisition of the code to be detected.

[0195] Word segmentation refers to dividing a semantic slice into a sequence of words to facilitate subsequent word vectorization. During word segmentation, the pre-segmented semantic slice is read, and each symbol or other statement element in the semantic slice (SC) is separated by spaces. For example, the code statement "if(!ptr&&!size)" in the semantic slice is segmented into {"if", "(", "!", "ptr", "&", "&", "!", "size", ")"}.

[0196] String replacement refers to uniformly replacing all strings in a semantic slice with the same string. For example, replacing all strings in a semantic slice with "STRFORMAT". For instance, replacing "helloworld" with "STRFORMAT" in the semantic slice "string str = "helloworld""" would result in "string str = "STRFORMAT""". String replacement also includes the removal of comments and characters of non-standard information exchange code types. Here, comments can be removed by recognizing comment formats such as "* / ", " / *", and " / / ", and non-ASCII characters can be removed using regular expressions.

[0197] Symbolization of functions and variables refers to replacing custom function names and variable names with unified symbolic representations. For example, replacing a custom function name with "fun_i3" indicates that the corresponding function is the i3rd function; replacing a custom variable name with "var_i4" indicates that the corresponding variable is the i4th variable. For instance, replacing "ff_png_zalloc" with "fun_1", "opaque" with "v_1", "items" with "v_2", and "size" with "v_3" in the semantic slice "void*ff_png_zalloc(void*opaque,unsigned int items,unsigned int size)" results in "void*fun_1(void*v_1,unsigned int v_2,unsigned int v_3)".

[0198] Understandably, by replacing strings and symbolizing functions and variables, the influence of irrelevant information in the code to be detected on the detection results can be reduced, thereby improving the accuracy of code detection.

[0199] Slice vectorization refers to the vector representation of the code to be detected after slicing, word segmentation, string replacement, and symbolization of functions and variables. Vectorization methods (such as Word2Vec, Fasttext, and Glove) can be used for slice vectorization. For example, since the Skip-gram model in Word2Vec is superior to the Continuous Bag-of-Words (CBOW) model for rare words, this embodiment can use the Skip-gram model to achieve slice vectorization, thereby obtaining the word vector sequence to be input into the hybrid neural network model.

[0200] Understandably, data preprocessing can preserve the semantic information of the source code to the greatest extent possible, thereby improving the accuracy of the detection results.

[0201] See also Figure 8 The input layer 8-2 inputs the word vector sequence 8-41 corresponding to the semantic slices in the code to be detected to the bidirectional encoder model 8-11 of the converter. The bidirectional encoder model 8-11 processes the word vector sequence 8-41 to obtain the semantic features between words, resulting in a word vector sequence 8-42 (called the character feature sequence) that corresponds one-to-one with the word vector sequence 8-41. Each word vector in the word vector sequence 8-42 is a vector representation of a word. The processing of the bidirectional encoder model 8-11 and the encoder model 8-11 of the converter is described below.

[0202] It should be noted that the BERT model is an improvement on the converter model, which is composed of layers of encoders stacked on top of each other. Therefore, the BERT model consists of two parts: the embedding layer and the encoder layer.

[0203] See Figure 9 , Figure 9 This is a schematic diagram of the structure of a bidirectional encoder model of an exemplary converter provided in an embodiment of this application; as shown... Figure 9As shown, the embedding layer 9-1 is used to input the word vector sequence (represented as word vector 9-11 to word vector 9-1N). The encoder layer 9-2 exemplarily illustrates a two-layer encoder, where each encoder in each layer can receive the outputs of the modules in the previous layer; for example, each encoder in the first layer can receive word vectors 9-11 to 9-1N. Thus, through the attention mechanism in the encoder, each word vector in the word vector sequence can include information before and after that word vector, achieving bidirectional information extraction. In addition, the bidirectional encoder model of the converter also includes an output layer 9-3 for outputting the word vector sequence (represented as word vector 9-31 to word vector 9-3N).

[0204] The embedding layer and encoder of the BERT model will be explained below.

[0205] It should be noted that the output of the embedding layer consists of three parts: token embedding, segment embedding, and position embedding.

[0206] For example, see Figure 10 , Figure 10 This is an exemplary output diagram of an embedded layer provided in an embodiment of this application; as shown... Figure 10 As shown, for input 10-1 ("[CLS]void*v_4=NULL[SEP]return NULL[SEP]"), the corresponding word embedding 10-2 ("E") is shown. [] E void E * E v_ E ＝ E NULL E [EP] E return E NULL E [EP] "), segmentation embedding 10-3 ("E") A E A E A E A E A E A E A E B E B E B ") and position embedding 10-4 ("E0E1E2E3E4E5E6E7E8E9").

[0207] It's important to note that word embeddings represent character-level word vectors. The starting position is indicated by adding a start marker (CLS) to the beginning of each word, and a segmentation marker (SEP) is used to segment the code statement. The BERT model uses a WordPiece for word segmentation, ensuring that each input word receives a correct representation. Segmentation embeddings are used to distinguish between code statements. When two code statements are input, each word in the first code statement is marked with "E". A In the second code statement, each word will be marked as "E". B Position embedding is used to represent the positional information of words in a sequence, which helps to address the problem of ignoring word order in attention mechanisms.

[0208] The encoder in the BERT model extracts semantic information between words from the input vector, such as grammatical dependencies, feature data dependencies, control dependencies, and relationships between words. The encoder consists of N stacked multi-head self-attention layers and feedforward neural network layers; this structure enables the BERT model to capture the semantics between words. When the input sequence is X = {X1, X2, ..., X...} n4 When n4 represents the total number of elements in the sequence, the output formula of each sublayer in the Encoder is shown in formula (1), so that the vector output by the previous sublayer is passed to the next sublayer through the summation normalization layer (LayerNorm), as shown in formula (1).

[0209] sub_layer_output=LayerNorm(X+(SubLayer(X)))(1);

[0210] Where SubLayer represents a multi-head self-attention layer or a feedforward neural network layer, and sub_layer_output represents the output of the summation and normalization layer for SubLayer.

[0211] See Figure 11 , Figure 11 This is a schematic diagram of an exemplary single encoder provided in an embodiment of this application; as shown... Figure 11 As shown, an encoder 11-1 is generated by stacking a multi-head attention layer 11-11, an add-normalization layer 11-12, a feed-forward neural network layer 11-13, and another add-normalization layer 11-14. Each layer of BERT is composed of such an encoder 11-1.

[0212] It's important to note that in the BERT model, if it includes 24 encoder layers, with each layer containing 16 self-attention layers, the word vector dimension is 1024. In the BERT model, if it includes 12 encoder layers, with each layer containing 12 self-attention layers, the word vector dimension is 768. Regardless of the BERT model's size, the size of the feedforward neural network layer is always set to 4A, where A is the word vector dimension. That is, when A = 768, the size of the feedforward neural network layer is 3072, and when A = 1024, the size of the feedforward neural network layer is 4096.

[0213] It should also be noted that the multi-head self-attention layer is used to read the semantics of the sequence after reading each word vector in the input sequence; the acquisition of this semantics can be achieved through the following steps.

[0214] First, a matrix transformation is performed on the input sequence X. The matrix transformation process is shown in formula (2).

[0215] Q = XW Q K = XW K V = XW V (2);

[0216] Where Q represents the query feature, which is the output of the previous attention sublayer; K represents the key feature, V represents the value feature, and K and V are the set of attention vectors output by the Encoder. W Q W K and W V Both represent matrix transformation parameters.

[0217] Additionally, the process of performing matrix transformations on the input sequence X can be found in [link to documentation]. Figure 12 , Figure 12 This is an exemplary matrix transformation diagram provided in an embodiment of this application; as shown... Figure 12 As shown, the input sequence 12-2 is subjected to matrix transformation based on the first parameter 12-1 to obtain query feature 12-3; the input sequence 12-2 is subjected to matrix transformation based on the second parameter 12-4 to obtain key feature 12-5; and the input sequence 12-2 is subjected to matrix transformation based on the third parameter 12-6 to obtain value feature 12-7.

[0218] Next, calculate the self-attention (Attention(Q,K,V)). The self-attention calculation process is shown in formula (3).

[0219]

[0220] Among them, K T The transpose of represents , d represents the vector dimension of , and Softmax1 is the activation function.

[0221] Next, based on formula (3), through W1, W2, ..., W h Project Q, K, V onto h subspaces, W1, W2, ..., W... h Let h represent the independent linear transformation matrix, and h represent the total number of linear transformations, which is the total number of heads in the attention mechanism; the projection process is shown in formula (4).

[0222]

[0223] Among them, h j W represents the projection result (called the attention feature) in the j-th (k∈[1,h]) subspace. j include and The parameters are, in order: query feature projection parameters, key feature projection parameters, and value feature projection parameters. Here, This is based on the calculation process of Attention(Q,K,V) in formula (3).

[0224] Finally, for h different projection results h1,…,head h (referred to as at least one attention feature) are concatenated to generate the feature vector Z, as shown in formula (5).

[0225] Z=MultiHead(Q,K,V)=Concat(h1,…,head h W O (5);

[0226] Among them, W O The parameters represent the splicing parameters, and both Z and MultiHead(Q,K,V) represent the splicing results.

[0227] It should be noted that the feedforward neural network is essentially a simple two-layer network, where the first layer is an activation function (ReLU). The function of the feedforward neural network is to provide nonlinear learning capability. For the vector Z in the input sequence, the processing of the feedforward neural network FFN is shown in formula (6).

[0228] FFN(Z)=ax(0,ZW 11 +1)W 12 +2(6);

[0229] Where max represents the maximum value function, W 11 W 12 b1 and b2 represent the network parameters of the feedforward neural network, and FFN(Z) represents the output of the feedforward neural network.

[0230] It should be noted that the function of the summation and normalization layer is to pass the vector output by the previous sub-layer to the next sub-layer. The summation and normalization layer is ultimately used to transform the input into data with a mean of 0 and a variance of 1.

[0231] See also Figure 8 The bidirectional encoder model 8-11 of the converter inputs its output word vector sequence 8-42 into the bidirectional gated recurrent unit model 8-12. The bidirectional gated recurrent unit model 8-12 is a two-direction (referred to as the first sequence direction and the second sequence direction) GRU network structure composed of two individual GRUs, including a forward GRU network and a backward GRU network. The two individual GRUs do not share states; the forward GRU network only propagates states along the forward direction, and the backward GRU network only propagates states along the backward direction. Simultaneously, the output vectors of the two GRUs are concatenated, and the final output is used as the entire BERT-BGRU output 8-43. The following describes the bidirectional gated recurrent unit model 8-12 and its processing procedure.

[0232] It should be noted that the BGRU model is used to extract contextual semantic information from the source code statements of a software system program. The BGRU model combines two unidirectional GRU networks to extract features from two directions, fusing the sequence features in the source code into word vectors.

[0233] See Figure 13 , Figure 13 This is a schematic diagram of an exemplary bidirectional gated loop unit model provided in an embodiment of this application; as shown... Figure 13 As shown, the bidirectional gated recurrent unit model 13-1 includes an input layer 13-11, a forward network 13-12, a backward network 13-13, and an output layer 13-14. The input layer 13-11 is used to obtain the input word vector sequence 13-2; the forward network 13-12 is used to pass information based on the forward order of the word vector sequence 13-2; the backward network 13-13 is used to pass information based on the reverse order of the word vector sequence 13-2; and the output layer 13-14, for each word vector in the word vector sequence 13-2, concatenates the output obtained from the forward pass with the output obtained from the backward pass to form the final output 13-3.

[0234] See Figure 14 , Figure 14 This is a schematic diagram of the structure of an example gated loop unit provided in an embodiment of this application; as shown Figure 14 As shown, the gated loop unit 14-1 includes a reset gate 14-11, an update gate 14-12, and a candidate memory unit state function 14-13; the output 14-21 of the gated loop unit at the previous time step (denoted as h) u-1) and the current input 14-22 (denoted as x) u After passing through the reset gate 14-11, update gate 14-12, and candidate memory unit state function 14-13 in the gated loop unit 14-1, the output 14-23 (denoted as h) of the gated loop unit 14-1 at the current moment can be obtained. u ).

[0235] It should be noted that reset gate 14-11 is used to combine output 14-21 and input 14-22 to obtain reset gate information r. u Reset door information r u The process of obtaining is shown in Formula (7); Formula (7) is shown below.

[0236] r u = 1(W r *[h u-1 , u + r (7);

[0237] Where σ1 represents the activation function (e.g., the sigmoid function), W r and b r All of these are for resetting door parameters.

[0238] It should be noted that the candidate memory cell state function 14-13 is used to obtain the candidate memory cell state by combining the output 14-21, the input 14-22, and the reset gate information. Candidate memory cell state The process of obtaining is shown in formula (8); formula (8) is shown below.

[0239]

[0240] Where tanh represents the hyperbolic tangent function. and These are all parameters of the candidate memory cell state function.

[0241] It should be noted that update gate 14-12 is used to combine output 14-21 and input 14-22 to obtain update gate information z. u Update door information z u The process of obtaining is shown in formula (9); formula (9) is shown below.

[0242] z u = 2(W z *[h u-1 , u + z (9);

[0243] Where σ² represents the activation function (e.g., the sigmoid function), Wz and b z All of these are updates to the gate parameters.

[0244] It should be noted that outputs 14-23 are obtained by combining the states of candidate memory cells. Update door information z u The process of obtaining outputs 14-23 is shown in formula (10), which is obtained from outputs 14-21. Formula (10) is shown below.

[0245]

[0246] It should be noted that, for the input x at the current moment... u If the output of the GRU in the previous time step before forward propagation is And the output of the GRU at the previous moment before the reverse propagation is The output of the GRU during the forward propagation at the current moment. As shown in Equation (11), the output of the GRU in the reverse propagation at the current time... As shown in formula (12), and the final output at the current time. As shown in formula (13).

[0247]

[0248]

[0249]

[0250] GRU() can encode the output word vectors into the hidden state of the gated recurrent unit through a series of nonlinear transformations. The calculation process is shown in formulas (7) to (10). At time u, the weights of the hidden state of the forward gated recurrent unit (called the first state weights), the weights of the hidden state of the backward gated recurrent unit (called the second state weights), and the bias of the hidden state are W, respectively. u V u and b u .

[0251] See also Figure 8 The classification output layer 8-3 processes the output 8-43 to obtain the code detection result 8-44.

[0252] It should be noted that the classification output layer uses classifier Softmax2, and the calculation process of classifier Softmax2 is shown in formula (14).

[0253]

[0254] Where H represents the output 8-43, Softmax2(H) k Let H represent the probability that H belongs to the k-th category. This represents the score of H belonging to the k-th category. H represents the cumulative score across all categories, and y represents the y-th category among all categories.

[0255] The training process of the hybrid neural network model is explained below.

[0256] First, let's explain the parameter settings for the hybrid neural network model; see Table 1, which shows the parameter settings for the hybrid neural network model; Table 1 is shown below.

[0257] Table 1

[0258] Model parameters Settings Word vector dimension 768 GRU output dimensions 100 Learning rate 1e-5 Dropout rate 0.5 Optimizer Adaptive Optimizer (Adam) Activation function Linear rectified function (ReLU) Model iteration count 80 number of floors 12 Size of each batch of training datasets 128

[0259] The training process of the hybrid neural network model will be explained next.

[0260] It should be noted that during the training process of the hybrid neural network model, the input is a dataset T = {T1, T2, ..., T}. B}, where B represents the total number of data points in the dataset; the output is the trained hybrid neural network model.

[0261] See Figure 15 , Figure 15 This is a flowchart illustrating the training process of an example hybrid neural network model provided in an embodiment of this application; as shown... Figure 15 As shown, the training process of the hybrid neural network model in this example includes steps 201 to 210, and each step is explained below.

[0262] Step 201: Generate a training set based on the dataset.

[0263] It should be noted that a numerical probability algorithm was used to divide the dataset into a 20% test set and an 80% training set.

[0264] Step 202: Extract slices from the training set.

[0265] It should be noted that slice extraction includes forward slice extraction and backward slice extraction.

[0266] Step 203: Perform word segmentation, string replacement, symbolization of functions and variables, and vectorization of the extracted bidirectional slices.

[0267] It should be noted that the process described in step 203 is similar to the preprocessing process in the model application process, and will not be described again in the embodiments of this application.

[0268] Step 204: Initialize the hybrid neural network model to be trained.

[0269] In this embodiment of the application, the hybrid neural network model to be trained can be initialized based on the information in Table 1.

[0270] Step 205: The bidirectional encoder model of the converter in the hybrid neural network model to be trained is used to process the slice vectorization result to obtain the word vector sequence.

[0271] It should be noted that the process described in step 205 is similar to the feature extraction process of the BERT model in the model application process, and will not be described again in this embodiment of the application.

[0272] Step 206: The word vector sequence is processed using the bidirectional gated recurrent unit model in the hybrid neural network model to be trained to obtain the features to be detected.

[0273] It should be noted that the process described in step 206 is similar to the feature extraction process of the BGRU model in the model application process, and will not be described again in this embodiment of the application.

[0274] Step 207: Use the classifier in the hybrid neural network model to be trained to output the code detection result corresponding to the feature to be detected.

[0275] It should be noted that the process described in step 207 is similar to the processing of the classification output layer in the model application process, and will not be described again in this embodiment.

[0276] Step 208: Train the hybrid neural network model to be trained based on the code detection results and training set annotation information to obtain the hybrid neural network model to be tested.

[0277] In the embodiments of this application, backpropagation can be used for model training.

[0278] Step 209: Use a test set to validate the hybrid neural network model to be tested.

[0279] It should be noted that the test set is used to verify the generalization ability of the hybrid neural network model under test.

[0280] Step 210: Output the trained hybrid neural network model when the verification is successful.

[0281] In this embodiment, the BGRU model in the hybrid neural network model can be replaced with a Kepler algorithm (KOA) optimized convolutional bidirectional gated recurrent unit (CNN-BGRU).

[0282] Understandably, the BERT model is first used to obtain local semantic information between words to obtain the corresponding word vector sequence. Then, this word vector sequence is used as input to the BGRU model to obtain global semantic information between code statements. In this way, both the local semantics of the code to be detected and the global semantics of the code to be detected can be obtained during the code detection process, thereby improving the accuracy of code detection.

[0283] The following description continues to illustrate the exemplary structure of the code data processing device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the code data processing device 455 of the memory 450 may include:

[0284] The slice extraction module 4551 is used to extract semantic slices to be detected from the code to be detected.

[0285] Feature extraction module 4552 is used to extract character features from the code character sequence of the semantic slice to be detected, and obtain a character feature sequence;

[0286] The feature extraction module 4552 is further configured to perform semantic feature extraction on the character feature sequence based on the first sequence direction to obtain a first semantic feature sequence;

[0287] The feature extraction module 4552 is further configured to perform semantic feature extraction on the character feature sequence based on the second sequence direction to obtain a second semantic feature sequence, wherein the first sequence direction is opposite to the second sequence direction;

[0288] The code detection module 4553 is used to perform code detection by combining the first semantic feature sequence and the second semantic feature sequence to obtain the code detection result.

[0289] In this embodiment of the application, the feature extraction module 4552 is further configured to traverse the character feature sequence based on the first sequence direction, and perform the following processing on the t-th character feature traversed, where T≥t>1, and t is an integer, and T represents the number of character features in the character feature sequence: extract semantic features from the t-th character feature based on the (t-1)-th first semantic feature of the (t-1)-th character feature to obtain the t-th first semantic feature; and determine the T first semantic features obtained by traversing the character feature sequence based on the first sequence direction as the first semantic feature sequence.

[0290] In this embodiment of the application, the feature extraction module 4552 is further configured to perform semantic feature extraction by combining the (t-1)th first semantic feature and the tth character feature of the (t-1)th character feature to obtain first reset gate information and first update gate information; calculate the state of the first candidate memory unit by combining the first reset gate information, the (t-1)th semantic feature and the tth character feature; and determine the tth first semantic feature by combining the first update gate information, the state of the first candidate memory unit and the (t-1)th first semantic feature.

[0291] In this embodiment of the application, the feature extraction module 4552 is further configured to traverse the character feature sequence based on the second sequence direction, and perform the following processing on the s-th character feature traversed, where T≥s>1 and s is an integer: extract semantic features from the s-th character feature based on the (s-1)-th second semantic feature of the (s-1)-th character feature to obtain the s-th second semantic feature; and determine the T second semantic features obtained by traversing the character feature sequence based on the second sequence direction as the second semantic feature sequence.

[0292] In this embodiment of the application, the feature extraction module 4552 is further configured to perform semantic feature extraction by combining the (s-1)th second semantic feature of the (s-1)th character feature and the sth character feature to obtain second reset gate information and second update gate information; calculate the state of the second candidate memory unit by combining the second reset gate information, the (s-1)th semantic feature and the sth character feature; and determine the sth second semantic feature by combining the second update gate information, the state of the second candidate memory unit and the (s-1)th second semantic feature.

[0293] In this embodiment of the application, the code detection module 4553 is further configured to: determine a corresponding target first semantic feature from the first semantic feature sequence and a corresponding target second semantic feature from the second semantic feature sequence for a target character feature in the character feature sequence, wherein the target character feature is any character feature in the character feature sequence; combine the target first semantic feature and the target second semantic feature to determine a target semantic feature; obtain a target semantic feature sequence corresponding to the character feature sequence from the target semantic feature of the target character feature; and perform code detection based on the target semantic feature sequence to obtain the code detection result.

[0294] In this embodiment of the application, the code detection module 4553 is further configured to obtain a first fusion result of the first state weight in the first sequence direction and the first semantic feature of the target; obtain a second fusion result of the second state weight in the second sequence direction and the second semantic feature of the target; and determine the target semantic feature by combining the first fusion result and the second fusion result.

[0295] In this embodiment of the application, the feature extraction module 4552 is further configured to perform embedding representation on the code character sequence of the semantic slice to be detected to obtain an embedding representation sequence; perform attention processing on the embedding representation sequence to obtain a feature sequence to be transformed; and perform nonlinear transformation on the feature sequence to be transformed to obtain the character feature sequence.

[0296] In this embodiment of the application, the feature extraction module 4552 is further configured to perform character embedding representation on the code character sequence of the semantic slice to be detected to obtain a character embedding sequence; perform segment embedding representation on the code character sequence to obtain a segment embedding sequence; perform position embedding representation on the code character sequence to obtain a position embedding sequence; and combine the character embedding sequence, the segment embedding sequence and the position embedding sequence to obtain the embedding representation sequence.

[0297] In this embodiment of the application, the feature extraction module 4552 is further configured to perform matrix transformation on the embedded representation sequence to obtain query features, key features and value features; perform attention processing on the query features, key features and value features based on the linear transformation matrix of the specified mapping space to obtain attention features; obtain at least one attention feature corresponding to at least one specified mapping space from the attention features of the specified mapping space; and obtain the feature sequence to be transformed based on at least one attention feature.

[0298] In this embodiment of the application, the code data processing device 455 further includes a character segmentation module 4554, which is used to denoise the semantic slice to be detected to obtain a semantic slice to be mapped; map the custom information in the semantic slice to be mapped to a specified symbol representation to obtain a semantic slice to be segmented, wherein the custom information includes at least one of the following: string content, function name and variable name; and perform code character segmentation on the semantic slice to be segmented to obtain the code character sequence.

[0299] In this embodiment of the application, the slice extraction module 4551 is further configured to perform forward slice extraction on the code to be detected to obtain a forward semantic slice; perform backward slice extraction on the code to be detected to obtain a backward semantic slice; and integrate the forward semantic slice and the backward semantic slice into the semantic slice to be detected.

[0300] In this embodiment, the code detection result is obtained through a code detection model; the code data processing device 455 further includes a model training module 4555, used to acquire training data, the training data including semantic slice samples and code detection annotation information of the semantic slice samples; the semantic slice samples are subjected to code detection using a model to be trained to obtain code detection prediction information, the model to be trained is a neural network model to be trained for code detection; based on the difference between the code detection prediction information and the code detection annotation information, the model to be trained is trained to obtain the code detection model.

[0301] In this embodiment of the application, the code data processing device 455 further includes a code repair module 4556, which is used to determine the semantic slice to be detected as code to be repaired when the code detection result indicates that the semantic slice to be detected contains abnormal code; to display repair prompt information for the code to be repaired; and to perform code repair on the code to be repaired in response to the repair operation of the repair prompt information.

[0302] This application provides a computer program product comprising computer-executable instructions or a computer program stored in a computer-readable storage medium. A processor of a code data processing device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the code data processing device to perform the code data processing method described in this application.

[0303] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the code data processing method provided in this application, for example, such as... Figure 3 The code shows the data processing method.

[0304] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0305] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0306] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0307] As an example, computer-executable instructions can be deployed to execute on a single electronic device (in which case, this single electronic device is a code data processing device), or to execute on multiple electronic devices located in one location (in which case, the multiple electronic devices located in one location are code data processing devices), or to execute on multiple electronic devices distributed across multiple locations and interconnected via a communication network (in which case, the multiple electronic devices distributed across multiple locations and interconnected via a communication network are code data processing devices).

[0308] It is understood that in the embodiments of this application, data related to the code to be detected is involved. When the embodiments of this application are applied to specific products or technologies, the permission or consent of the information subject is required. The collection, use and processing of related data must comply with relevant laws, regulations and standards, and conform to the principles of legality, legitimacy and necessity. It does not involve obtaining data types prohibited or restricted by laws and regulations, and will not hinder the normal operation of the target website.

[0309] In summary, when performing code detection on the code to be detected, the embodiments of this application first extract the semantic slice to be detected from the code to be detected, and then extract the character feature sequence of the code character sequence of the semantic slice to be detected, thereby realizing the extraction of the features of the code characters; then, based on the character feature sequence, semantic features are extracted from two opposite sequence directions, thereby realizing the extraction of global semantic features of the code characters from the entire semantic slice to be detected, improving the accuracy of feature extraction of the semantic slice to be detected; thus, by combining the extracted semantic feature sequences for code detection, the accuracy of code detection can be improved; and consequently, the accuracy and efficiency of code repair can be improved.

[0310] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method of processing code data, characterized by, The code data processing method comprises: extracting a to-be-detected semantic slice from a to-be-detected code; character feature extraction is performed on a code character sequence of the to-be-detected semantic slice to obtain a character feature sequence; first semantic feature extraction is performed on the character feature sequence based on a first sequence direction to obtain a first semantic feature sequence; second semantic feature extraction is performed on the character feature sequence based on a second sequence direction to obtain a second semantic feature sequence, the first sequence direction being opposite to the second sequence direction; code detection is performed in combination with the first semantic feature sequence and the second semantic feature sequence to obtain a code detection result.

2. The method of claim 1, wherein, The first semantic feature extraction based on the first sequence direction comprises: traversing the character feature sequence based on the first sequence direction, and performing the following processing on the tth character feature in the sequence, T≥t>1, t being an integer, and T representing the number of character features in the character feature sequence: performing semantic feature extraction on the tth character feature based on the t-1th first semantic feature of the t-1th character feature to obtain the tth first semantic feature; determining the T first semantic features obtained by traversing the character feature sequence based on the first sequence direction as a first semantic feature sequence.

3. The method of claim 2, wherein, The semantic feature extraction based on the t-1th first semantic feature of the t-1th character feature and the tth character feature comprises: performing semantic feature extraction in combination with the t-1th first semantic feature of the t-1th character feature and the tth character feature to obtain first reset gate information and first update gate information; calculating a first candidate memory cell state in combination with the first reset gate information, the t-1th semantic feature, and the tth character feature; determining the tth first semantic feature in combination with the first update gate information, the first candidate memory cell state, and the t-1th first semantic feature.

4. The method of claim 1, wherein, The second semantic feature extraction based on the second sequence direction comprises: traversing the character feature sequence based on the second sequence direction, and performing the following processing on the s th character feature in the sequence, T≥s>1, s being an integer: performing semantic feature extraction on the s th character feature based on the s-1th second semantic feature of the s-1th character feature to obtain the s th second semantic feature; determining the T second semantic features obtained by traversing the character feature sequence based on the second sequence direction as a second semantic feature sequence.

5. The method of claim 4, wherein, The semantic feature extraction based on the s-1th second semantic feature of the s-1th character feature and the s th character feature comprises: performing semantic feature extraction in combination with the s-1th second semantic feature of the s-1th character feature and the s th character feature to obtain second reset gate information and second update gate information; In combination with the second reset gate information, the (s-1)th semantic feature, and the s th character feature, a second candidate memory cell state is calculated; In combination with the second update gate information, the second candidate memory cell state, and the (s-1)th second semantic feature, the s th second semantic feature is determined.

6. The method according to any one of claims 1 to 5, characterized in that, The code detection is performed in combination with the first semantic feature sequence and the second semantic feature sequence to obtain a code detection result, including: For a target character feature in the character feature sequence, a corresponding target first semantic feature is determined from the first semantic feature sequence, and a corresponding target second semantic feature is determined from the second semantic feature sequence, the target character feature being any character feature in the character feature sequence; In combination with the target first semantic feature and the target second semantic feature, a target semantic feature is determined; From the target semantic feature of the target character feature, a target semantic feature sequence corresponding to the character feature sequence is obtained; Based on the target semantic feature sequence, code detection is performed to obtain the code detection result.

7. The method of claim 6, wherein, The combination of the target first semantic feature and the target second semantic feature to determine the target semantic feature includes: Obtaining a first fusion result of the first state weight of the first sequence direction and the target first semantic feature; Obtaining a second fusion result of the second state weight of the second sequence direction and the target second semantic feature; In combination with the first fusion result and the second fusion result, the target semantic feature is determined.

8. The method according to any one of claims 1 to 5, characterized in that, The character feature sequence is obtained by performing character feature extraction on the code character sequence of the semantic slice to be detected, including: The embedding representation sequence is obtained by performing embedding representation on the code character sequence of the semantic slice to be detected; The attention processing is performed on the embedding representation sequence to obtain a to-be-transformed feature sequence; The nonlinear transformation is performed on the to-be-transformed feature sequence to obtain the character feature sequence.

9. The method of claim 8, wherein, The embedding representation sequence is obtained by performing embedding representation on the code character sequence of the semantic slice to be detected, including: The character embedding sequence is obtained by performing character embedding representation on the code character sequence of the semantic slice to be detected; The separation embedding sequence is obtained by performing separation embedding representation on the code character sequence; The position embedding sequence is obtained by performing position embedding representation on the code character sequence; In combination with the character embedding sequence, the separation embedding sequence, and the position embedding sequence, the embedding representation sequence is obtained.

10. The method of claim 8, wherein, The attention processing is performed on the embedding representation sequence to obtain a to-be-transformed feature sequence, including: The matrix transformation is performed on the embedding representation sequence to obtain a query feature, a key feature, and a value feature; Based on the linear transformation matrix of the specified mapping space, the attention processing is performed on the query feature, the key feature, and the value feature to obtain an attention feature; From the attention feature of the specified mapping space, at least one attention feature corresponding to at least one specified mapping space is obtained; Based on at least one attention feature, the to-be-transformed feature sequence is obtained.

11. The method according to any one of claims 1 to 5, characterized in that, Before the character feature extraction on the code character sequence of the to-be-detected semantic slice is performed, the method further includes: denoising the to-be-detected semantic slice to obtain a to-be-mapped semantic slice; mapping custom information in the to-be-mapped semantic slice into a specified symbol representation to obtain a to-be-divided semantic slice, the custom information including at least one of a string content, a function name, and a variable name; dividing the to-be-divided semantic slice to obtain the code character sequence.

12. The method according to any one of claims 1 to 5, characterized in that, The to-be-detected semantic slice is extracted from the to-be-detected code, including: forward slicing the to-be-detected code to obtain a forward semantic slice; backward slicing the to-be-detected code to obtain a backward semantic slice; integrating the forward semantic slice and the backward semantic slice into the to-be-detected semantic slice.

13. The method according to any one of claims 1 to 5, characterized in that, The code detection result is obtained through a code detection model; The code detection model is obtained through training, including: obtaining training data, the training data including semantic slice samples and code detection annotation information of the semantic slice samples; performing code detection on the semantic slice samples by using a to-be-trained model to obtain code detection estimation information, the to-be-trained model being a to-be-trained neural network model used for code detection; training the to-be-trained model based on differences between the code detection estimation information and the code detection annotation information to obtain the code detection model.

14. The method according to any one of claims 1 to 5, characterized in that, After the code detection result is obtained by combining the first semantic feature sequence and the second semantic feature sequence, the method further includes: when the code detection result indicates that the to-be-detected semantic slice includes abnormal code, determining the to-be-detected semantic slice as to-be-repaired code; displaying repair prompt information for the to-be-repaired code; in response to a repair operation for the repair prompt information, performing code repair on the to-be-repaired code.

15. A code data processing apparatus, characterized by comprising: The code data processing apparatus includes: a slicing extraction module configured to extract a to-be-detected semantic slice from to-be-detected code; a feature extraction module configured to perform character feature extraction on a code character sequence of the to-be-detected semantic slice to obtain a character feature sequence; the feature extraction module is further configured to perform semantic feature extraction on the character feature sequence based on a first sequence direction to obtain a first semantic feature sequence; the feature extraction module is further configured to perform semantic feature extraction on the character feature sequence based on a second sequence direction to obtain a second semantic feature sequence, the first sequence direction being opposite to the second sequence direction; a code detection module configured to perform code detection by combining the first semantic feature sequence and the second semantic feature sequence to obtain a code detection result.

16. A code data processing electronic device, comprising: The electronic device includes: a memory configured to store computer executable instructions or computer programs; a processor configured to execute the computer executable instructions or computer programs stored in the memory to implement the code data processing method in any one of claims 1 to 14.

17. A computer-readable storage medium storing computer-executable instructions or a computer program, wherein the computer-executable instructions or the computer program comprise the steps of claim 16. The computer executable instructions or the computer program are executed by the processor to implement the code data processing method in any one of claims 1 to 14.

18. A computer program product comprising computer-executable instructions or a computer program, characterized in that, The computer executable instructions or the computer program are executed by the processor to implement the code data processing method in any one of claims 1 to 14.

Images