Code detection method and device for data sandbox

Through pre-trained code detection model and word segmentation processing technology, the data sandbox code is detected, which solves the problem of manual auditing in the existing technology that it is time-consuming and labor-intensive and difficult to distinguish code compliance, and realizes accurate detection of data sandbox code and data security improvement.

CN118940256APending Publication Date: 2024-11-12NEUSOFT CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411002024.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, the audit of data sandbox code mainly relies on manual auditing, which is time-consuming and labor-intensive and has poor real-time performance, making it difficult to detect dangerous APIs and malicious codes in a timely manner, and it is difficult to distinguish code compliance based on different sensitivity of the data.

Method used

Using pre-trained code detection model and word segmentation processing technology, the data sandbox code is detected. By running the code multiple times in the debugging environment, combining the detection results and running results, the code compliance is determined, word vectors are generated and the model is trained. , realize accurate detection of data sandbox code.

Benefits of technology

It improves the accuracy and efficiency of data sandbox code detection, can more accurately distinguish code types, ensure data security, reduce the dependence of manual audits, and improve the probability of discovering dangerous codes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940256A_ABST
    Figure CN118940256A_ABST
Patent Text Reader

Abstract

The invention relates to a code detection method and device for a data sandbox. The code detection method for the data sandbox comprises the steps that a data sandbox code is obtained, and the data sandbox code is used for processing data; according to a pre-trained code detection model and the data sandbox code, a first detection result is determined, and the first detection result is used for representing that the data sandbox code is a normal code or an abnormal code; running the data sandbox code in a preset debugging environment to obtain a code running result; and according to the first detection result and the code operation result, a second detection result is determined, and the second detection result is used for representing that the data sandbox code is a compliance code, a non-compliance code or a to-be-audited code. The method is used for realizing accurate detection of the data sandbox code, so that the security of the data is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of data sandboxes, and in particular to a code detection method and device for a data sandbox. Background Art

[0002] With the development of data processing technology, data sandbox technology has also developed. Data sandbox technology can achieve the separation of debugging environment and running environment, as well as the separation of data ownership and data usage rights. For example, data sandbox can mine and analyze raw data without exposing the raw data, and safely output the valuable data analysis results obtained through data mining and analysis, making the data available but invisible. Summary of the invention

[0003] The purpose of the present disclosure is to provide a data sandbox code detection method and device, which are used to accurately detect the data sandbox code and thus ensure the security of the data.

[0004] In order to achieve the above-mentioned objectives, in a first aspect, the present disclosure provides a code detection method for a data sandbox, comprising: obtaining a data sandbox code, wherein the data sandbox code is used to process data; determining a first detection result based on a pre-trained code detection model and the data sandbox code, wherein the first detection result is used to characterize that the data sandbox code is a normal code or an abnormal code; running the data sandbox code in a preset debugging environment to obtain a code running result; determining a second detection result based on the first detection result and the code running result, wherein the second detection result is used to characterize that the data sandbox code is a compliant code, a non-compliant code, or a code to be reviewed.

[0005] Optionally, determining the first detection result based on the pre-trained code detection model and the data sandbox code includes: performing word segmentation processing on the data sandbox code to obtain multiple code segmentations; generating a word vector corresponding to the data sandbox code based on the multiple code segmentations and a preset word vector length, the preset word vector length being a word vector length that matches the pre-trained code detection model; determining the first detection result based on the word vector corresponding to the data sandbox code and the pre-trained code detection model, the pre-trained code detection model being used to output an abnormal code probability and a normal code probability based on the word vector corresponding to the input data sandbox code.

[0006] Optionally, running the data sandbox code in a preset debugging environment to obtain a code running result includes: running the data sandbox code multiple times in the preset debugging environment to obtain multiple code running results; correspondingly, determining the second detection result based on the first detection result and the code running result includes: determining an abnormal code rate based on the multiple code running results, the abnormal code rate is used to characterize the proportion of the target code running result in the multiple code running results, and the target code running result is a non-compliant code running result; determining the second detection result based on the first detection result and the abnormal code rate.

[0007] Optionally, determining the second detection result based on the first detection result and the abnormal code rate includes: if the first detection result indicates that the data sandbox code is a normal code, and the abnormal code rate is greater than a preset threshold, or, the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is less than the preset threshold, then determining that the data sandbox code is a code to be reviewed, and the preset threshold is determined according to the sensitivity of the data processed by the data sandbox code; if the first detection result indicates that the data sandbox code is a normal code, and the abnormal code rate is less than or equal to the preset threshold, then determining that the data sandbox code is a compliant code; if the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is greater than or equal to the preset threshold, then determining that the data sandbox code is a non-compliant code.

[0008] Optionally, the code detection method further includes: obtaining multiple code samples and code sample running results corresponding to the multiple code samples, the multiple code samples respectively corresponding to category labels, the category labels including normal code labels and abnormal code labels; performing word segmentation processing on the multiple code samples respectively to obtain word lists corresponding to the multiple code samples respectively, each word list including multiple word lists; for each word list in the multiple word lists, determining the category score corresponding to the word list; determining the accuracy of the category labels corresponding to the multiple code samples respectively according to the code sample running results corresponding to the multiple code samples respectively, the category label accuracy is used to characterize the accuracy of the category label; generating training data according to the multiple word lists, the category scores corresponding to the multiple word lists respectively, the category labels corresponding to the multiple code samples respectively, and the category label accuracy corresponding to the multiple code samples respectively; and training the code detection model to be trained according to the training data to obtain the pre-trained code detection model.

[0009] Optionally, for each word segment in the multiple vocabularies, determining the category score corresponding to the word segment includes: determining the word frequency inverse document frequency corresponding to the word segment, the word frequency inverse document frequency is used to characterize the importance of the word segment in the corresponding code sample; determining a first entropy value of the word segment between a positive category code sample and a negative category code sample, the category label corresponding to the positive category code sample is a normal code label, and the category label corresponding to the negative category code sample is an abnormal code label; determining a second entropy value of the word segment in the positive category code sample and a third entropy value of the word segment in the negative category code sample; determining the positive category score corresponding to the word segment based on the first entropy value and the second entropy value; determining the negative category score corresponding to the word segment based on the first entropy value and the third entropy value; determining the category score corresponding to the word segment based on the word frequency inverse document frequency corresponding to the word segment, the positive category score corresponding to the word segment, and the negative category score corresponding to the word segment.

[0010] Optionally, performing word segmentation processing on the multiple code samples to obtain word lists corresponding to the multiple code samples respectively includes: performing word segmentation processing on the multiple code samples to obtain first word lists corresponding to the multiple code samples respectively; expanding the first word lists corresponding to the multiple code samples respectively according to the first word lists corresponding to the multiple code samples respectively and the context relationships corresponding to the multiple code samples respectively to obtain second word lists corresponding to the multiple code samples respectively; determining the word lists corresponding to the multiple code samples respectively according to the first word lists corresponding to the multiple code samples respectively and the second word lists corresponding to the multiple code samples respectively.

[0011] Optionally, generating training data according to the multiple vocabularies, the category scores corresponding to the multiple word segments, the category labels corresponding to the multiple code samples, and the accuracy of the category labels corresponding to the multiple code samples, includes: filtering the word segments in the multiple vocabularies according to the category scores corresponding to the multiple word segments to obtain multiple filtered word segments; generating multiple original word vectors according to the multiple filtered word segments and a preset word vector length, wherein each original word vector corresponds to a filtered word segment, and the preset word vector length is a word vector length that matches the code detection model to be trained; for each original word vector in the multiple original word vectors, determining the weights corresponding to the word segments included in the original word vector according to the category scores corresponding to the word segments included in the original word vector; generating sample word vectors according to the weights corresponding to the word segments included in the original word vector; generating the training data according to the multiple sample word vectors, the category labels corresponding to the multiple code samples, and the accuracy of the category labels corresponding to the multiple code samples.

[0012] Optionally, the loss function corresponding to the code detection model to be trained is used to perform calculations based on the predicted category probability, the category label and the category label accuracy.

[0013] In a second aspect, the present disclosure provides a code detection device for a data sandbox, comprising: an acquisition module, used to acquire a data sandbox code, wherein the data sandbox code is used to process data; a first determination module, used to determine a first detection result based on a pre-trained code detection model and the data sandbox code, wherein the first detection result is used to characterize that the data sandbox code is a normal code or an abnormal code; an operation module, used to run the data sandbox code in a preset debugging environment to obtain a code operation result; a second determination module, used to determine a second detection result based on the first detection result and the code operation result, wherein the second detection result is used to characterize that the data sandbox code is a compliant code, a non-compliant code, or a code to be reviewed.

[0014] Through the above technical solution, the code detection model is used to determine the first detection result that characterizes whether the data sandbox code is an abnormal code, and the data sandbox code is run in a debugging environment to obtain the code running result. Then, the second detection result that characterizes the type of the data sandbox code is determined through the first detection result and the code running result. Since the first detection result and the code running result belong to detection results of different dimensions, by integrating the detection results of different dimensions, the accuracy of the final detection result can be improved, and the code type can be more accurately distinguished. On the basis of more accurate detection results of the data sandbox code, the data code can be processed accordingly to improve the security of the data processed by the data sandbox code. Therefore, the technical solution can realize accurate detection of the data sandbox code, thereby ensuring the security of the data.

[0015] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0017] Figure 1 The present invention is a flowchart of a code detection method of a data sandbox according to an exemplary embodiment.

[0018] Figure 2 The figure is a flow chart of a code detection solution for a data sandbox according to an exemplary embodiment.

[0019] Figure 3The present invention is an application flow chart of a code detection solution for a data sandbox according to an exemplary embodiment.

[0020] Figure 4 The invention is a block diagram of a code detection device of a data sandbox according to an exemplary embodiment.

[0021] Figure 5 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0022] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0023] With the development of data processing technology, various data processing needs have also emerged. In some data processing needs, sensitive original data needs to be protected from leaving the domain and being invisible. Therefore, in order to enrich the data utilization scenarios, data sandboxes have emerged.

[0024] The core concept of the data sandbox is to separate data ownership and data usage rights by separating the debugging environment from the operating environment. The data sandbox can support desensitized sampling to obtain sample data, conduct data mining and analysis without exposing the original data, and output the valuable data results (models and calculation analysis results, etc.) obtained through data mining and analysis in a safe and open manner, making the data available but invisible.

[0025] However, the data sandbox faces some problems: if the data user uses the data sandbox platform to write data analysis code, after using the data analysis code to analyze the full amount of data (sensitive data), the output result file needs to be reviewed to ensure that the result file does not include sensitive or non-compliant data. Only result files that pass the review can be used by data users.

[0026] Therefore, the data sandbox code used by the data sandbox may output result files containing sensitive or non-compliant data. Based on this, it is necessary to review the result files of the data sandbox or review the data sandbox code.

[0027] In the related art, manual review is mainly used to determine whether the data sandbox code and the output result file are compliant. For example, the data administrator manually reviews and approves the result file.

[0028] This audit method not only consumes a lot of time and manpower, but also cannot guarantee real-time performance. In addition, there is a possibility of misjudgment and wrong judgment. If dangerous APIs (Application Programming Interfaces) in the code cannot be discovered in time, it may cause risks to subsequent data analysis.

[0029] In addition, with regard to the identification of malicious code, most of the identification is of malicious APIs that cause program vulnerabilities in the code. This type of malicious code is relatively easy to identify. For example, code that contains malicious APIs or may cause system vulnerabilities can be identified as malicious code.

[0030] However, in the data sandbox, the scenarios are different, and it is stipulated that the code that outputs non-compliant data is non-compliant code. First, there are many ways to determine whether the data is compliant. Secondly, due to the different levels of data, its sensitivity is also different, which makes the method of determining whether the code is compliant flexible. Even if the output data is not all compliant data, its corresponding code may not affect the normal use of the data sandbox. Generally speaking, in order to ensure the security of the data sandbox, all output data needs to be strictly reviewed.

[0031] It can be seen that how to target the specificity of the data sandbox, safely and effectively differentiate the code according to data compliance standards, and improve the efficiency of code review and result file review are technical issues that need to be solved.

[0032] Based on this, the embodiments of the present disclosure provide a technical solution, which performs comprehensive detection on the data sandbox code and accurately distinguishes the code types. On this basis, the code is processed accordingly based on the code detection results, which can improve the security of the data processed by the data sandbox code.

[0033] Figure 1 The following is a flow chart of a code detection method for a data sandbox according to an exemplary embodiment. The code detection method includes the following steps:

[0034] Step S11, obtaining the data sandbox code.

[0035] Step S12: determining a first detection result according to the pre-trained code detection model and the data sandbox code.

[0036] Step S13, running the data sandbox code in a preset debugging environment to obtain the code running result.

[0037] Step S14, determining a second detection result according to the first detection result and the code running result.

[0038] In step S11, the data sandbox code is used to process the data, and the processing here can be the aforementioned data analysis or mining. After the data sandbox code processes the data, the result file can be output. Therefore, by testing the data sandbox code, the data sandbox code can be reasonably applied according to the test results to ensure the security of data processing and avoid including non-compliant data or sensitive data in the result file.

[0039] In step S12, the first detection result can indicate whether the data sandbox code is normal code or abnormal code; in step S13, the code running result can be understood as the aforementioned result file, and the result file can also be used to determine whether the code is abnormal code. Therefore, step S12 and step S13 can be regarded as two different code detection methods, and the order of execution of these two steps is not limited.

[0040] In step S14, the second detection result can characterize the data sandbox code as a compliant code, a non-compliant code, or a code to be reviewed. Therefore, in the disclosed embodiment, the types defined for the data sandbox code include a compliant code type, a non-compliant code type, and a code to be reviewed. For different types, the final application or processing method may be different.

[0041] For example, for compliant codes, they can be directly applied. For non-compliant codes, they can be modified. For codes to be reviewed, they can be further reviewed manually. Among them, the processing methods of non-compliant codes and codes to be reviewed will be further introduced in subsequent embodiments.

[0042] It can be understood that the data sandbox code includes multiple fields. In order to facilitate the processing of the code detection model, the data sandbox code can be converted into corresponding word vectors, so that the code detection model can output the first detection result based on the input word vector.

[0043] Therefore, as an optional implementation, step S12 includes: performing word segmentation processing on the data sandbox code to obtain multiple code segmentations; generating a word vector corresponding to the data sandbox code based on the multiple code segmentations and a preset word vector length, wherein the preset word vector length is a word vector length that matches the pre-trained code detection model; determining a first detection result according to the word vector corresponding to the data sandbox code and the pre-trained code detection model, wherein the pre-trained code detection model is used to output an abnormal code probability and a normal code probability according to the word vector corresponding to the input data sandbox code.

[0044] In this implementation, the implementation of word segmentation processing can refer to mature technologies in the field.

[0045] The preset word vector length can be the word vector length matched by the pre-trained code detection model. That is, the word vector of the input model is usually specified to have a fixed length. Therefore, multiple codes can be tokenized and the corresponding word vectors can be generated according to the preset word vector length.

[0046] In some embodiments, the word vector length is used to represent the number of words included in the word vector. For example, if the word vector is: [w1, w2, w3, w4, w5], then the word vector length is 5.

[0047] In some embodiments, if the number of word segments just matches the word vector length, a word vector can be directly generated based on multiple word segments. If the number of word segments is less than the number of words represented by the word vector length, the word vector can be completed based on the existing word segments. If the number of word segments is greater than the number of words represented by the word vector length, the word segments can be deleted.

[0048] In some embodiments, before performing word segmentation processing on the data sandbox code, non-API codes in the data sandbox code, such as custom variables, constants, and strings, etc., may be removed.

[0049] Furthermore, the word vector corresponding to the data sandbox code is input into the pre-trained code detection model, and the pre-trained code detection model can output the abnormal code probability and the normal code probability, which can characterize whether the data sandbox code is a normal code or an abnormal code. For example, if the abnormal code probability is greater than the normal code probability, the data sandbox code is characterized as an abnormal code.

[0050] In order to ensure the model accuracy of the pre-trained code detection model, the embodiment of the present disclosure provides a training scheme for the code detection model, which is introduced below.

[0051] As an optional implementation, the training process of the code detection model includes: obtaining multiple code samples and code sample running results corresponding to the multiple code samples, respectively, the multiple code samples have category labels corresponding to them, and the category labels include normal code labels and abnormal code labels; performing word segmentation processing on the multiple code samples to obtain word lists corresponding to the multiple code samples, each word list includes multiple word lists; for each word list in the multiple word lists, determining the category score corresponding to the word list; according to the code sample running results corresponding to the multiple code samples, determining the accuracy of the category labels corresponding to the multiple code samples, the category label accuracy is used to characterize the accuracy of the category label; generating training data according to the multiple word lists, the category scores corresponding to the multiple word lists, the category labels corresponding to the multiple code samples, and the category label accuracy corresponding to the multiple code samples; according to the training data, training the code detection model to be trained to obtain a pre-trained code detection model.

[0052] The multiple code samples may be related codes recorded in the data sandbox. The multiple code samples may be run in a debugging environment to obtain code sample running results. Furthermore, the multiple code samples may have corresponding category labels, which may indicate whether the code samples are normal codes or abnormal codes.

[0053] For example, regarding the category label, if the code sample is a normal code, its corresponding category label may be "1" representing the positive class; if the code sample is an abnormal code, its corresponding category label may be "0" representing the negative class.

[0054] Regarding the determination of the category label of the code sample, manual labeling, algorithm or rule labeling can be used.

[0055] As an optional algorithm or rule, the category of the code sample is determined according to the preset data identifier and the generated data corresponding to the code sample, and the preset data identifier can be used to infer the data type. Specifically, the number of preset data identifiers included in the generated data corresponding to the code sample can be determined, and the category of the code sample is determined according to the number of preset data identifiers.

[0056] For example, the dataset D = {x1,x2,x3,…,x i ,…,x n ,s},{x1,x2,x3,…,x i ,…,x n} is a quasi-identifier, through which a specific piece of data, such as gender and zip code, may be inferred, and s is a sensitive attribute, such as name, phone number, and ID number.

[0057] For some open data, it is required that the data cannot be uniquely determined to belong to a certain category through features. Therefore, the following judgment method is adopted: first, the sensitive feature s is eliminated; second, several data with the same quasi-identifier features are regarded as an equivalence class, for example, divided into m equivalence classes C = {C1, C2, C3, ..., C j ,…,C m},|C j |≥q,1≤j≤m. The formula means: Equivalence class C j Each element in each quasi-identifier x i The values ​​on the same and meet the requirements of C j The number of data in the class needs to be greater than q. When judging whether the data is compliant, for example, when the q value is set to 2, if there are data in the equivalent class with less than 2 data, the output data is judged to be non-compliant, and the code that generates such data is marked as a negative sample, otherwise it is marked as a positive sample.

[0058] That is, if the class to which the data belongs can be determined through a quasi-identifier feature less than 2, the code sample corresponding to the data is a negative sample; otherwise, the code sample corresponding to the data is a positive sample.

[0059] As another identification method, the category of the code sample may be determined according to the result file corresponding to the code sample.

[0060] For example, for sensitive data, it is required that there should not be any data features of the original data. Therefore, when the result file of the code sample has the following situations, the code sample can be marked as a negative sample: the result file contains any data that is the same as the original data; the result file contains data generated by encoding or compression; the result file contains data with unclear meaning.

[0061] It is understandable that other sample annotation methods may be adopted according to specific application scenarios, which are not limited here.

[0062] For any code sample, the code sample running result corresponding to the code sample may include multiple code running results obtained by running the code multiple times in a code debugging environment, and each code running result may indicate whether the code is an abnormal code.

[0063] In some embodiments, the code debugging environment may be a simulated debugging environment configured for the code, which may characterize the actual operation of the code. Therefore, by running the code in the debugging environment, a code operation result characterizing whether the actual operation is abnormal may be obtained.

[0064] For multiple code samples, word segmentation processing is performed respectively to obtain word lists corresponding to the multiple code samples, each word list includes multiple word segments.

[0065] Before word segmentation, you can remove non-API codes in the code, such as custom variables, constants, and strings.

[0066] For example, for the following code, the word list after removing the API code can be [os, listdir, pd, DataFrame, …, concat].

[0067] files = os.listdir(path)

[0068] df = pd.DataFrame()

[0069] for file in files:

[0070] print(file)

[0071] df1=pd.read_csv(path+' / '+file)

[0072] df = pd.concat([df,df1])

[0073] In order to improve the richness of the vocabulary, as an optional implementation method, multiple code samples are respectively segmented to obtain vocabulary corresponding to the multiple code samples, including: performing word segmentation on the multiple code samples to obtain first vocabulary corresponding to the multiple code samples; expanding the first vocabulary corresponding to the multiple code samples according to the first vocabulary corresponding to the multiple code samples and the contextual relationship corresponding to the multiple code samples to obtain second vocabulary corresponding to the multiple code samples; determining the vocabulary corresponding to the multiple code samples according to the first vocabulary corresponding to the multiple code samples and the second vocabulary corresponding to the multiple code samples.

[0074] In this implementation, the first vocabulary can be regarded as the vocabulary obtained after word segmentation. Based on the first vocabulary, further vocabulary expansion can be performed to obtain a second vocabulary. Finally, the first vocabulary and the second vocabulary are integrated to obtain a final vocabulary.

[0075] In some embodiments, based on the contextual relationship, the second vocabulary may be generated by an inter-word n-gram method, which may include a 2-gram method or a 3-gram method.

[0076] For example, based on the word list [os, listdir, pd, DataFrame, …, concat], the word list using 2-gram can be expressed as [os_listdir, listdir_pd, …, pd_concat], and the word list using 3-gram can be expressed as [os_listdir_pd, listdir_pd_DataFrame, readcsv_pd_concat]. Adjacent words are connected by underscores or other identifiers and are regarded as one word segment.

[0077] Furthermore, the first vocabulary and the second vocabulary are synthesized, and the synthesized vocabulary is the vocabulary of the code sample, and the vocabulary of the code sample is a synthetic vocabulary of high-dimensional word segmentation and n-gram words.

[0078] It can be understood that the vocabulary of the code sample may include multiple participles, for example, "os", "listdir", "os_listdir", etc., which can be regarded as one participle respectively.

[0079] For each word segment in the multiple vocabularies, a category score corresponding to the word segment may be determined. The category score may be used as a characteristic degree of the vocabularies. The category score may be used to calculate a weight of each word segment.

[0080] The category score can represent the influence of the word in the category to which the corresponding code sample belongs. For example, if the code sample is a negative sample, and one of the words has a high negative category score, it means that the word is the key content in the code sample, and the code sample may be abnormal due to the word. Therefore, in subsequent code applications, you can focus on the word.

[0081] Therefore, in some embodiments, if the detection result of the aforementioned data sandbox code is code to be reviewed, the category score determination method herein can also be used to determine the category score of each word in the code to be reviewed, so as to further check the code according to the category score.

[0082] As an optional implementation, for each word segment in multiple vocabularies, determining the category score corresponding to the word segment, including: determining the word frequency inverse document frequency corresponding to the word segment, the word frequency inverse document frequency is used to characterize the importance of the word segment in the corresponding code sample; determining a first entropy value of the word segment between the positive category code sample and the negative category code sample, the category label corresponding to the positive category code sample is a normal code label, and the category label corresponding to the negative category code sample is an abnormal code label; determining a second entropy value of the word segment in the positive category code sample and a third entropy value of the word segment in the negative category code sample; determining the positive category score corresponding to the word segment based on the first entropy value and the second entropy value; determining the negative category score corresponding to the word segment based on the first entropy value and the third entropy value; determining the category score corresponding to the word segment based on the word frequency inverse document frequency corresponding to the word segment, the positive category score corresponding to the word segment, and the negative category score corresponding to the word segment.

[0083] When calculating the category score of a word segmentation, it is necessary to combine the frequency of occurrence of the word segmentation in other code samples, including the frequency of occurrence in positive category code samples and the frequency of occurrence in negative category code samples.

[0084] The word frequency inverse document frequency can represent the importance of the word segment in the corresponding code sample. Therefore, the word frequency inverse document frequency can be determined based on the word frequency and inverse document frequency of the word segment.

[0085] In addition, since the code sample usually exists in the form of a document, in the embodiment of the present disclosure, the frequency of occurrence of a segmentation word in the code sample is referred to as the document frequency.

[0086] For example, you can first calculate the code word frequency: tf(d,w) represents the frequency of word segmentation w, N wThe total number of times the word w appears in all codes, N is the total number of words in all codes. Then, calculate the inverse document frequency: ifd(w) represents the inverse document frequency, n represents the total number of code samples, and n(w) represents the number of code samples containing the word w. Finally, the term frequency inverse document frequency is: tf_idf(w) = tf(d,w) · idf(w), tf_idf(w) represents the term frequency inverse document frequency.

[0087] The first entropy value can be understood as an inter-class entropy value, which can be an entropy value that does not consider the document frequency, or an entropy value that considers the document frequency. Therefore, the first entropy value can be determined in two ways.

[0088] As a first determination method, the first entropy value can be expressed as:

[0089] Among them, H b (w) represents the first entropy value, w i Represents the number of word segmentation w in the i-th category. Since there are only two categories, positive and negative samples, the value of i here ranges from 1 to 2. N w Indicates the number of times word w appears in all codes. b The larger (w) is, the more evenly the word w is distributed in each category, and the smaller its ability to distinguish between categories is. b The smaller (w) is, the more unevenly the word w is distributed among the categories, and the greater its ability to distinguish the categories. Therefore, H b (w)The smaller the better.

[0090] As a second determination method, the first entropy value can be expressed as:

[0091] Among them, H b (w) represents the first entropy value, n is the total number of code samples, n(w) is the number of code samples containing word w, The larger it is, the lower the discrimination of the participle w is; The smaller it is, the higher the discrimination of the word w is, so H b (w) should be as small as possible.

[0092] It can be understood that in actual application, the above two determination methods can be selected according to needs.

[0093] The second entropy value can be understood as the entropy value within the positive category, and the third entropy value can be understood as the entropy value within the negative category. Similar to the first entropy value described above, the second entropy value and the third entropy value can also be determined using two optional determination methods.

[0094] As a first determination method, the second entropy value can be expressed as: The third entropy value can be expressed as:

[0095] Among them, H p (w) represents the second entropy value, H e (w) represents the third entropy value. n is the total number of code samples, w n For code d n The number of words in the middle part, N p is the total number of times word w appears in the positive code sample, N e H is the total number of times word w appears in negative samples. p (w) and H e The larger (w) is, the more evenly word w is distributed in the corresponding class, and the greater the contribution of word w to distinguishing this class. p (w) and H e The smaller (w) is, the more unevenly the word w is distributed in the corresponding class, and the smaller the contribution of word w to distinguishing this class is. Therefore, the smaller the intra-class difference of word w is, the higher the H p (w) and H e The larger (w) is, the more distinguishing the word w is within the class.

[0096] As a second determination method, the second entropy value can be expressed as: The third entropy value can be expressed as:

[0097] Among them, H p (w) represents the second entropy value, H e (w) represents the third entropy value. p Represents the number of positive class code samples, n p (w)+1 represents the number of positive class code samples containing word w. The larger the value is, the more evenly the word segmentation w is distributed in the positive class, and the better the distinction effect is. Otherwise, the distinction effect is worse. e Represents the number of negative class code samples, n e (w)+1 represents the number of negative code samples containing word w, The larger it is, the more evenly the word segmentation w is distributed in the negative class, and the better the distinction effect is. Otherwise, the distinction effect is worse.

[0098] It can be understood that in actual application, the above two determination methods can be selected according to needs.

[0099] Furthermore, based on the first entropy value and the second entropy value, a positive category score corresponding to the word segment may be determined.

[0100] For example, the positive category score corresponding to the word can be expressed as: The negative category score corresponding to this word can be expressed as:

[0101] After determining the positive and negative category scores corresponding to the word segmentation, the entire vocabulary can be sorted according to the scores to obtain a positive vocabulary list and a negative vocabulary list, which can be saved for further review of the data sandbox code, for example: for marking code keywords.

[0102] For example, the positive vocabulary list W P =[w1,w2,w3,…,w N ]; Negative vocabulary list W e =[w1,w2,w3,…,w N ]. From w1 to w N They are sorted from high to low by their positive and negative scores respectively.

[0103] Further, based on the term frequency inverse document frequency, the positive category score and the negative category score, a final category score may be determined.

[0104] For example, assume that the code dataset D = {d1, d2, d3, ..., d i ,…,d n}, d i =[w1,w2,w3,…,w j ,…,w P ]. Among them, w j For code sample d i The jth word in the code, there are P words in total.

[0105] Then, the category score of each word in the code can be expressed as: S(w) = max(tf idf(w) *S c (w)). Where S(w) represents the category score of word w, S c (w) represents the positive or negative category score of word w, tf idf(w) Represents the inverse document frequency of word frequency of word segmentation w. The calculation formula means: multiply the positive category score of word segmentation w by the inverse document frequency of word frequency to obtain the first product, and multiply the negative category score of word segmentation w by the inverse document frequency of word frequency to obtain the second product. The category score takes the larger value of the first product and the second product.

[0106] In the embodiment of the present disclosure, the category score can be used as a characteristic degree.

[0107] Regarding the accuracy of the category label, divide the test data into m parts or take m parts of data (with the same compliance judgment rule), run the code in the debugging environment and obtain the result data. Use the same compliance judgment rule to judge the compliance of m data. Suppose the number of non-compliant data is k, then the number of compliant data is mk. Then, the actual positive accuracy is The actual negative accuracy is Therefore, the label accuracy of the positive class code sample is The label accuracy of the negative class code sample is

[0108] Furthermore, training data is generated according to multiple vocabularies, category scores corresponding to multiple word segments, category labels corresponding to multiple code samples, and accuracy rates of category labels corresponding to multiple code samples.

[0109] As an optional implementation, training data is generated based on multiple vocabularies, category scores corresponding to multiple word segments, category labels corresponding to multiple code samples, and accuracy rates of category labels corresponding to multiple code samples, including: filtering word segments in multiple vocabularies based on category scores corresponding to multiple word segments to obtain multiple filtered word segments; generating multiple original word vectors based on the filtered multiple word segments and preset word vector lengths, wherein each original word vector corresponds to a filtered word segment, and the preset word vector length is a word vector length that matches a code detection model to be trained; for each original word vector in the multiple original word vectors, determining weights corresponding to the word segments included in the original word vector based on category scores corresponding to the word segments included in the original word vector; generating sample word vectors based on weights corresponding to the word segments included in the original word vector; generating training data based on multiple sample word vectors, category labels corresponding to multiple code samples, and accuracy rates of category labels corresponding to multiple code samples.

[0110] In this implementation, the category score is first used to screen the segmented words. For example, a category score threshold can be set to delete the segmented words whose category scores are less than the category score threshold. Alternatively, multiple segmented words can be sorted according to the category score, and the segmented words with category scores in the top 50% are retained, and the rest of the segmented words are deleted. Thus, the segmented words with low characteristic degrees are eliminated, so that the segmented words in the screened vocabulary are all high characteristic degree segmented words.

[0111] For example, the threshold is set to 0.5 (can be set as needed). For each code sample vocabulary, the first 50% of the words are extracted and retained. The other words in this code are considered as low-feature words and deleted. Finally, the vocabulary of the code sample after screening can be written as: i =[w1,w2,w3,…,w j,…,w Q , repeat n times (n is the number of code samples) to obtain the filtered vocabulary list for all code samples. The union of all vocabulary lists can form the filtered vocabulary list for all codes, denoted as: W = [w1, w2, w3, …, w F ; where F is the number of word segments in the filtered vocabulary list, and F < N (N is the number of word segments in the original vocabulary list).

[0112] In some embodiments, the preset word vector length can be the average of the word vector lengths of all code samples.

[0113] For example, assume that the filtered code dataset D = {d1, d2, d3, …, d i ,…, d n}, obtain the number of word segments D N = {N1, N2, N3, …, N i ,…, N n} of all code samples, and calculate the average length of all files where N i is the number of word segments in the vocabulary list of the i-th code sample.

[0114] Furthermore, when generating the original word vectors, if the length of the generated word vectors is less than the average length, then word vector completion is performed. If the length of the generated word vectors is greater than the average length, then the word vectors at the redundant positions are truncated.

[0115] For example, for the word vector length of each code sample, if the length is less than then the word vectors at the empty positions are completed, and the completion method is to repeat the word segments in the existing vocabulary list. For example if the size is 5, then [w1, w2, w3] can be completed to [w1, w2, w3, w1, w2]. If the length is greater than then the word vectors at the redundant positions are truncated, and the truncation method can be to select the last x word segments with the lowest category scores according to the category scores of the word segments. Finally, the original word vectors of all code samples are processed to have the same length.

[0116] For the model, when training, weights also need to be configured. Therefore, the weights can be determined according to the category scores.

[0117] For example, assume that the word vector of code sample D N is The weights of the word segments where S(w i ) represents the category score of the i-th word segment, P represents the total number of word segments, represents the length of the word vector. Furthermore, the sample word vector can be expressed as:

[0118] Based on multiple sample word vectors, a word embedding matrix can be randomly generated. The number of matrix rows can be the number of words in the word list L, and the number of columns is the length of the word vector. To prevent problems such as slow training speed caused by too high a dimension, the word vector dimension can be selected to be 50-200. Then, the word embedding matrix is ​​the training data for the final input model.

[0119] In the embodiment of the present disclosure, the code detection model to be trained can be divided into: an input layer, a hidden layer, an output layer and a loss function calculation.

[0120] The above process of generating training data from multiple filtered vocabulary lists can be understood as the data processing process of the input layer.

[0121] Regarding the hidden layer, assuming that the input fixed length of the code dataset is L, the input data dimension is L×200 and the hidden layer dimension is 200×1.

[0122] Regarding the output layer, it can be understood as a binary classification problem, and a function can be used to determine the category. After the hidden layer calculates the dot product sum, the classification probability of the positive and negative classes is calculated through the following function: x represents the category for which the classification probability needs to be calculated.

[0123] Regarding the loss function, in the embodiment of the present disclosure, the loss function is used to perform calculations based on the predicted category probability, category label, and category label accuracy.

[0124] Since the result data of data query or analysis generated by the code is affected by the original data, the same code will have different performance results when facing different data (the rules for judging data compliance need to be consistent). In other words, the result data generated by using the code with high probability of negative category for data analysis operation is not necessarily non-compliant data. Therefore, a loss function calculation method based on the actual probability of data generated by the data sandbox is proposed to optimize the output of the classification network.

[0125] The commonly used binary cross entropy loss function formula is as follows:

[0126]

[0127] Where n is the number of samples, is the predicted value, y i is the true label.

[0128] As a first optional implementation, the loss function can be expressed as:

[0129] Among them, E m Represents the model cross entropy, which is calculated as: Among them, n is the number of samples, y is the predicted value, and yi is the category label.

[0130] E t Represents the actual cross entropy, for positive class samples, Among them, m-km represents the accuracy of the category label corresponding to the positive category samples.

[0131] For negative class samples, in, Represents the accuracy of the category label corresponding to the negative category samples.

[0132] As a second optional implementation, the loss function can be expressed as:

[0133]

[0134] in, Represents the accuracy of the category label. For positive category samples, its value is For negative class samples, its value is

[0135] Through the above loss function, on the basis of category labels, the accuracy of category labels determined according to actual running results can be increased to better meet the actual code discrimination situation, thereby improving the accuracy of the model.

[0136] Further, in step S13, the data sandbox code is run in a preset debugging environment to obtain a code running result, which may be a result file of the code. By judging the result file according to the compliance rules, it can be determined whether the result file includes non-compliant data, and then it can be determined whether the code running result is compliant. For example, if non-compliant data is included, it is determined that the code running result is non-compliant.

[0137] Therefore, as an optional implementation, step S13 includes: running the data sandbox code multiple times in a preset debugging environment to obtain multiple code running results. Correspondingly, determining the second detection result based on the first detection result and the code running result includes: determining the abnormal code rate based on the multiple code running results, the abnormal code rate is used to characterize the proportion of the target code running result in the multiple code running results, and the target code running result is an incompliant code running result; determining the second detection result based on the first detection result and the abnormal code rate.

[0138] In this implementation, the data sandbox code may be run multiple times to obtain multiple result files. Then, it is determined whether the multiple result files are compliant result files. Next, the proportion of non-compliant result files in all result files is determined, and the proportion is determined as the abnormal code rate.

[0139] Further, according to the first detection result and the abnormal code rate, a second detection result may be determined.

[0140] As an optional implementation, a second detection result is determined based on the first detection result and the abnormal code rate, including: if the first detection result indicates that the data sandbox code is a normal code, and the abnormal code rate is greater than a preset threshold, or the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is less than the preset threshold, then the data sandbox code is determined to be a code to be reviewed, and the preset threshold is determined according to the sensitivity of the data processed by the data sandbox code; if the first detection result indicates that the data sandbox code is a normal code, and the abnormal code rate is less than or equal to the preset threshold, then the data sandbox code is determined to be a compliant code; if the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is greater than or equal to the preset threshold, then the data sandbox code is determined to be a non-compliant code.

[0141] In this embodiment, the preset threshold value may be related to the sensitivity of the data processed by the data sandbox code. For example, the greater the sensitivity of the data, the smaller the preset threshold value, or directly 0. The smaller the sensitivity of the data, the preset threshold value may be greater than 0.

[0142] The sensitivity of data can represent security requirements. If the sensitivity is high, it means that the security requirements are high and data misjudgment is not allowed. If the sensitivity is low, data misjudgment can be allowed. For example, if the sensitivity is 0.1, it means that a 10% misjudgment rate can occur.

[0143] Therefore, if the first detection result indicates that the data sandbox code is a normal code and the abnormal code rate is greater than a preset threshold, the data sandbox code can be determined as a code to be reviewed, indicating that further review is required.

[0144] If the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is less than a preset threshold, it also indicates that further review is required.

[0145] If the first detection result indicates that the data sandbox code is a normal code and the abnormal code rate is less than or equal to a preset threshold, then it can be determined that the data sandbox code is a compliant code.

[0146] If the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is greater than or equal to a preset threshold, then the data sandbox code is determined to be a non-compliant code.

[0147] Therefore, the data sandbox code can be ultimately determined as: compliant code, non-compliant code, or code to be reviewed.

[0148] In the embodiments of the present disclosure, different subsequent processing methods may be used for different types of data sandbox codes.

[0149] For example, compliant code can be directly reused in the production environment. Non-compliant code is prohibited from running in the production environment.

[0150] For the code to be reviewed, you can first mark the code to be reviewed according to the aforementioned positive word list and negative word list, for example: mark the negative words in it. Then, the marked code to be reviewed is fed back to the reviewer, who will review it again. Or, it can be returned to the developer for re-modification and verification of code compliance before use.

[0151] Figure 2 is a flow chart of a code detection solution for a data sandbox according to an exemplary embodiment. Figure 2 As shown, the code detection solution involves: code category labeling based on data compliance judgment, a fusion model based on vocabulary scoring and sandbox result features, and code compliance judgment and alarm review based on data sensitivity thresholds.

[0152] Among them, the code category labeling based on data compliance judgment can correspond to the aforementioned category label setting process.

[0153] The fusion model based on vocabulary scoring and sandbox result features corresponds to the aforementioned code detection model training process, which may include: original vocabulary generation, feature vocabulary generation, vocabulary filtering and word vector generation, and a code classification model based on sandbox result features.

[0154] Code compliance determination and alarm review based on data sensitivity thresholds correspond to the execution process of the aforementioned steps S13 and S14. In addition, the alarm review may correspond to the subsequent processing flow based on the final detection result of the code.

[0155] Figure 3 The present invention is an application flow chart of a code detection solution for a data sandbox according to an exemplary embodiment.

[0156] like Figure 3 As shown, in the original code vocabulary generation stage, the processing flow involves removing custom variables, word segmentation, n-gram processing and synthesizing a new vocabulary.

[0157] In the feature word list generation stage, processes such as tf-idf feature (i.e., term frequency inverse document frequency) calculation, positive class score calculation, and negative class score calculation are involved.

[0158] In the vocabulary filtering and word vector generation stage, low-featured vocabulary is removed and embedding vectors are randomly initialized. Thus, the model can be trained and applied to compliance identification.

[0159] Then, in the code compliance discrimination model stage based on data sensitivity threshold, the corresponding model processing is used to output the detection results. The final detection results may involve three types: compliant code, code to be reviewed, and non-compliant code.

[0160] Furthermore, the aforementioned negative category word list can be used to process the code to be reviewed, and further code and result data review can be performed.

[0161] Therefore, the technical solution of the embodiment of the present disclosure has the following technical effects:

[0162] The data compliance is judged and the code is marked. At the same time, a method for judging data compliance is proposed to achieve effective and simple data labeling.

[0163] Code features are used for word segmentation features, n-gram features, tf-idf features, and various information entropy features to calculate a code vocabulary with high-discrimination features to improve code discrimination.

[0164] The normalized code vocabulary scores of high-discrimination features are used as the model input weights, and the actual accuracy of the code running results in the sandbox debugging environment is used to calculate the model loss function to improve the accuracy of the model.

[0165] A code compliance judgment and warning review process based on data sensitivity thresholds is proposed, and a relatively flexible code compliance judgment rule is defined. According to the importance of data, a certain degree of misjudgment probability of the code is allowed, and it is further divided into three categories: compliant, non-compliant, and pending review. Therefore, auditors can focus on the code to be reviewed, which can effectively increase the probability of discovering dangerous code and improve work efficiency.

[0166] Figure 4 is a block diagram of a code detection device for a data sandbox according to an exemplary embodiment, the device comprising:

[0167] The acquisition module 401 is used to acquire a data sandbox code, where the data sandbox code is used to process data.

[0168] The first determination module 402 is used to determine a first detection result according to a pre-trained code detection model and the data sandbox code, wherein the first detection result is used to characterize whether the data sandbox code is a normal code or an abnormal code.

[0169] The running module 403 is used to run the data sandbox code in a preset debugging environment to obtain a code running result, and the code running result is used to characterize whether the data sandbox code is a normal code or an abnormal code.

[0170] The second determination module 404 is used to determine a second detection result according to the first detection result and the code running result, wherein the second detection result is used to characterize whether the data sandbox code is a compliant code, a non-compliant code, or a code to be reviewed.

[0171] Optionally, the first determination module 402 is also used to: perform word segmentation processing on the data sandbox code to obtain multiple code segmentations; generate a word vector corresponding to the data sandbox code based on the multiple code segmentations and a preset word vector length, wherein the preset word vector length is a word vector length that matches the pre-trained code detection model; determine the first detection result according to the word vector corresponding to the data sandbox code and the pre-trained code detection model, and the pre-trained code detection model is used to output an abnormal code probability and a normal code probability according to the word vector corresponding to the input data sandbox code.

[0172] Optionally, the running module 403 is also used to: run the data sandbox code multiple times in the preset debugging environment to obtain multiple code running results; the second determination module 404 is also used to: determine the abnormal code rate based on the multiple code running results, and the abnormal code rate is used to characterize the proportion of the target code running result in the multiple code running results, and the target code running result is a non-compliant code running result; determine the second detection result based on the first detection result and the abnormal code rate.

[0173] Optionally, the second determination module 404 is also used for: if the first detection result indicates that the data sandbox code is a normal code, and the abnormal code rate is greater than a preset threshold, or the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is less than the preset threshold, then the data sandbox code is determined to be a code to be reviewed, and the preset threshold is determined according to the sensitivity of the data processed by the data sandbox code; if the first detection result indicates that the data sandbox code is a normal code, and the abnormal code rate is less than or equal to the preset threshold, then the data sandbox code is determined to be a compliant code; if the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is greater than or equal to the preset threshold, then the data sandbox code is determined to be a non-compliant code.

[0174] Optionally, the device also includes a training module, which is used to: obtain multiple code samples and code sample running results corresponding to the multiple code samples, the multiple code samples respectively correspond to category labels, and the category labels include normal code labels and abnormal code labels; perform word segmentation processing on the multiple code samples respectively to obtain word lists corresponding to the multiple code samples, each word list includes multiple word lists; for each word list in the multiple word lists, determine the category score corresponding to the word list; according to the code sample running results corresponding to the multiple code samples, determine the accuracy of the category labels corresponding to the multiple code samples, and the category label accuracy is used to characterize the accuracy of the category label; generate training data according to the multiple word lists, the category scores corresponding to the multiple word lists, the category labels corresponding to the multiple code samples, and the category label accuracy corresponding to the multiple code samples; according to the training data, train the code detection model to be trained to obtain the pre-trained code detection model.

[0175] Optionally, the training module is also used to: determine the word frequency inverse document frequency corresponding to the segmentation, the word frequency inverse document frequency is used to characterize the importance of the segmentation in the corresponding code sample; determine the first entropy value of the segmentation between the positive category code sample and the negative category code sample, the category label corresponding to the positive category code sample is a normal code label, and the category label corresponding to the negative category code sample is an abnormal code label; determine the second entropy value of the segmentation in the positive category code sample and the third entropy value of the segmentation in the negative category code sample; determine the positive category score corresponding to the segmentation based on the first entropy value and the second entropy value; determine the negative category score corresponding to the segmentation based on the first entropy value and the third entropy value; determine the category score corresponding to the segmentation based on the word frequency inverse document frequency corresponding to the segmentation, the positive category score corresponding to the segmentation and the negative category score corresponding to the segmentation.

[0176] Optionally, the training module is also used to: perform word segmentation processing on the multiple code samples to obtain first vocabularies corresponding to the multiple code samples respectively; expand the first vocabularies corresponding to the multiple code samples respectively according to the first vocabularies corresponding to the multiple code samples respectively and the context relationships corresponding to the multiple code samples respectively to obtain second vocabularies corresponding to the multiple code samples respectively; determine the vocabularies corresponding to the multiple code samples respectively according to the first vocabularies corresponding to the multiple code samples respectively and the second vocabularies corresponding to the multiple code samples respectively.

[0177] Optionally, the training module is also used to: filter the word segments in the multiple word lists according to the category scores corresponding to the multiple word lists respectively, to obtain multiple filtered word lists; generate multiple original word vectors according to the multiple filtered word lists and a preset word vector length, wherein each original word vector corresponds to a filtered word list, and the preset word vector length is a word vector length that matches the code detection model to be trained; for each original word vector in the multiple original word vectors, determine the weights corresponding to the word segments included in the original word vector according to the category scores corresponding to the word segments included in the original word vector; generate a sample word vector according to the weights corresponding to the word segments included in the original word vector; generate the training data according to multiple sample word vectors, the category labels corresponding to the multiple code samples, and the accuracy of the category labels corresponding to the multiple code samples.

[0178] Optionally, the loss function corresponding to the code detection model to be trained is used to perform calculations based on the predicted category probability, the category label and the category label accuracy.

[0179] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0180] Figure 5 FIG. 5 is a block diagram of an electronic device 500 according to an exemplary embodiment. Figure 5 As shown, the electronic device 500 may include: a processor 501 and a memory 502. The electronic device 500 may also include one or more of a multimedia component 503, an input / output (I / O) interface 504, and a communication component 505.

[0181] The processor 501 is used to control the overall operation of the electronic device 500 to complete all or part of the steps in the code detection method of the data sandbox. The memory 502 is used to store various types of data to support the operation of the electronic device 500. For example, these data may include instructions for any application or method used to operate on the electronic device 500, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 502 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 503 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 502 or sent through the communication component 505. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 504 provides an interface between the processor 501 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 505 is used for wired or wireless communication between the electronic device 500 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 505 may include: Wi-Fi module, Bluetooth module, NFC module.

[0182] In an exemplary embodiment, the electronic device 500 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned data sandbox code detection method.

[0183] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned data sandbox code detection method are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 502 including program instructions, and the above-mentioned program instructions may be executed by the processor 501 of the electronic device 500 to complete the above-mentioned data sandbox code detection method.

[0184] In another exemplary embodiment, a computer-readable storage medium including program instructions is further provided, and when the program instructions are executed by a processor, the steps of the above-mentioned code detection method of the data sandbox are implemented.

[0185] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, the technical solution of the present disclosure can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the present disclosure.

[0186] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0187] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A code detection method for data sandbox, It is characterized in that include: Obtaining a data sandbox code, where the data sandbox code is used to process the data; Determine a first detection result according to the pre-trained code detection model and the data sandbox code, where the first detection result is used to characterize whether the data sandbox code is a normal code or an abnormal code; Run the data sandbox code in a preset debugging environment to obtain the code running result; A second detection result is determined based on the first detection result and the code running result, where the second detection result is used to characterize whether the data sandbox code is a compliant code, a non-compliant code, or a code to be reviewed.

2. The code detection method according to claim 1, It is characterized in that The determining a first detection result according to the pre-trained code detection model and the data sandbox code includes: Performing word segmentation processing on the data sandbox code to obtain multiple code word segments; Generate a word vector corresponding to the data sandbox code based on the multiple code word segmentations and a preset word vector length, wherein the preset word vector length is a word vector length that matches the pre-trained code detection model; The first detection result is determined according to the word vector corresponding to the data sandbox code and the pre-trained code detection model, and the pre-trained code detection model is used to output the abnormal code probability and the normal code probability according to the word vector corresponding to the input data sandbox code.

3. The code detection method according to claim 1, It is characterized in that The data sandbox code is run in a preset debugging environment to obtain the code running result, including: Running the data sandbox code multiple times in the preset debugging environment to obtain multiple code running results; Correspondingly, determining the second detection result according to the first detection result and the code running result includes: Determine an abnormal code rate according to the multiple code running results, where the abnormal code rate is used to characterize the proportion of the target code running result in the multiple code running results, and the target code running result is an incompliant code running result; The second detection result is determined according to the first detection result and the abnormal code rate.

4. The code detection method according to claim 3, It is characterized in that The determining the second detection result according to the first detection result and the abnormal code rate includes: If the first detection result indicates that the data sandbox code is a normal code and the abnormal code rate is greater than a preset threshold, or if the first detection result indicates that the data sandbox code is an abnormal code and the abnormal code rate is less than the preset threshold, then the data sandbox code is determined to be a code to be reviewed, and the preset threshold is determined according to the sensitivity of the data processed by the data sandbox code; If the first detection result indicates that the data sandbox code is a normal code, and the abnormal code rate is less than or equal to the preset threshold, then the data sandbox code is determined to be a compliant code; If the first detection result indicates that the data sandbox code is an abnormal code, and the abnormal code rate is greater than or equal to the preset threshold, then the data sandbox code is determined to be a non-compliant code.

5. The code detection method according to claim 1, It is characterized in that The code detection method also includes: Acquire multiple code samples and code sample running results respectively corresponding to the multiple code samples, wherein the multiple code samples respectively correspond to category labels, and the category labels include normal code labels and abnormal code labels; Performing word segmentation processing on the multiple code samples respectively to obtain word lists corresponding to the multiple code samples respectively, each word list including multiple word lists; For each word segment in the multiple word lists, determining a category score corresponding to the word segment; Determine, according to the code sample running results respectively corresponding to the multiple code samples, the accuracy rates of the category labels respectively corresponding to the multiple code samples, wherein the accuracy rates of the category labels are used to characterize the accuracy rates of the category labels; Generate training data according to the multiple vocabularies, the category scores corresponding to the multiple word segments, the category labels corresponding to the multiple code samples, and the accuracy rates of the category labels corresponding to the multiple code samples; The code detection model to be trained is trained according to the training data to obtain the pre-trained code detection model.

6. The code detection method according to claim 5, It is characterized in that The step of determining, for each word segment in the multiple word lists, a category score corresponding to the word segment, comprises: Determine the word frequency inverse document frequency corresponding to the word segmentation, where the word frequency inverse document frequency is used to characterize the importance of the word segmentation in the corresponding code sample; Determine a first entropy value of the word segmentation between a positive category code sample and a negative category code sample, wherein the category label corresponding to the positive category code sample is a normal code label, and the category label corresponding to the negative category code sample is an abnormal code label; Determine a second entropy value of the word segment in the positive class code sample and a third entropy value of the word segment in the negative class code sample; Determine a positive category score corresponding to the word segment according to the first entropy value and the second entropy value; Determine a negative category score corresponding to the word segment according to the first entropy value and the third entropy value; The category score corresponding to the participle is determined according to the term frequency inverse document frequency corresponding to the participle, the positive category score corresponding to the participle, and the negative category score corresponding to the participle.

7. The code detection method according to claim 5, It is characterized in that The performing word segmentation processing on the multiple code samples to obtain word lists corresponding to the multiple code samples respectively includes: Performing word segmentation processing on the multiple code samples to obtain first word lists corresponding to the multiple code samples respectively; According to the first vocabularies respectively corresponding to the multiple code samples and the contextual relationships respectively corresponding to the multiple code samples, the first vocabularies respectively corresponding to the multiple code samples are expanded to obtain second vocabularies respectively corresponding to the multiple code samples; The vocabularies corresponding to the plurality of code samples are determined according to the first vocabularies corresponding to the plurality of code samples and the second vocabularies corresponding to the plurality of code samples.

8. The code detection method according to claim 5, It is characterized in that Generating training data according to the multiple vocabularies, the category scores corresponding to the multiple word segments, the category labels corresponding to the multiple code samples, and the accuracy rates of the category labels corresponding to the multiple code samples, comprises: Filtering the segmented words in the multiple word lists according to the category scores corresponding to the multiple segmented words, to obtain multiple filtered word lists; Generate multiple original word vectors according to the multiple filtered word lists and the preset word vector length, wherein each original word vector corresponds to a filtered word list, and the preset word vector length is the word vector length that matches the code detection model to be trained; For each original word vector of the plurality of original word vectors, determining the weights corresponding to the participles included in the original word vector according to the category scores corresponding to the participles included in the original word vector; Generate a sample word vector according to the weights corresponding to the word segments included in the original word vector; The training data is generated according to a plurality of sample word vectors, the category labels respectively corresponding to the plurality of code samples, and the accuracy rates of the category labels respectively corresponding to the plurality of code samples.

9. The code detection method according to any one of claims 5 to 8, It is characterized in that The loss function corresponding to the code detection model to be trained is used to perform calculations based on the predicted category probability, the category label, and the category label accuracy.

10. A code detection device for a data sandbox, It is characterized in that include: An acquisition module, used to acquire a data sandbox code, wherein the data sandbox code is used to process the data; A first determination module, configured to determine a first detection result according to a pre-trained code detection model and the data sandbox code, wherein the first detection result is used to characterize whether the data sandbox code is a normal code or an abnormal code; A running module, used to run the data sandbox code in a preset debugging environment to obtain a code running result; The second determination module is used to determine a second detection result according to the first detection result and the code running result, wherein the second detection result is used to characterize whether the data sandbox code is a compliant code, a non-compliant code or a code to be reviewed.