Data detection method and device, electronic equipment and storage medium

Abstract

The invention provides a data detection method and device, electronic equipment and a storage medium, and relates to the technical field of network security. The method comprises the following steps: acquiring each piece of to-be-detected data, and determining current feature information of a preset type sequence in each piece of to-be-detected data; detecting each piece of current feature information by using a preset detection model to obtain feature detection information corresponding to each piece of to-be-detected data; wherein the preset detection model is obtained by training based on historical feature information of preset type sequences corresponding to the positive sample data and the negative sample data; the positive sample data and the negative sample data are determined from historically detected data; and determining alarm data based on the feature detection information and the program instruction behavior of each piece of to-be-detected data. The data detection scheme in the application can adapt to different types of malicious files, and the detection accuracy is also improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a data detection method, device, electronic device and storage medium. Background Art

[0002] In the related art, malicious detection for JS (JavaScript) script files is a detection method based on predefined rules and pattern matching. It scans and matches files based on predefined rules and patterns to determine whether the files contain malicious code or behavior. The detection method based on predefined rules and pattern matching has some shortcomings and limitations, such as being unable to deal with unknown malicious files, being vulnerable to mutation and confusion attacks, and having a high false alarm rate. Therefore, malicious detection of JS script files in the related art has the problems of high false alarm rate and poor adaptability. Summary of the invention

[0003] A data detection method, device, electronic device and storage medium provided in the embodiments of the present application can improve the accuracy of malicious detection of JS script files and can adapt to various types of JS script files.

[0004] The technical solution of this application is implemented as follows:

[0005] The present application provides a data detection method, including:

[0006] Acquire each piece of data to be tested, and determine current feature information of a predetermined type sequence in each piece of data to be tested;

[0007] Detect each of the current feature information using a preset detection model to obtain feature detection information corresponding to each of the data to be tested;

[0008] Wherein, the preset detection model is obtained by training based on historical feature information of predetermined type sequences corresponding to positive sample data and negative sample data respectively; the positive sample data and the negative sample data are determined from historical detection data;

[0009] Based on the feature detection information and program instruction behavior of each of the data to be tested, alarm data is determined.

[0010] In the above scheme, the historical feature information includes: positive historical feature information and negative historical feature information; before using the preset detection model to detect each of the current feature information to obtain the feature detection information corresponding to each of the data to be tested, the method further includes:

[0011] Obtain each of the positive sample data and each of the negative sample data determined after testing the data in the historical period;

[0012] Determine the positive historical feature information of a predetermined type sequence in each of the positive sample data, and the negative historical feature information of a predetermined type sequence in the negative sample data;

[0013] The initial detection model is trained based on each of the positive historical feature information and each of the negative historical feature information until the training condition is met, thereby obtaining the preset detection model; wherein the initial detection model is a model obtained by training before the historical period.

[0014] In the above solution, the determining of the positive historical feature information of the predetermined type sequence in each of the positive sample data and the negative historical feature information of the predetermined type sequence in the negative sample data includes:

[0015] Screening out at least one positive sample sequence of a predetermined type from a plurality of first sequences included in each of the positive sample data, and screening out at least one negative sample sequence of a predetermined type from a plurality of second sequences included in each of the negative sample data;

[0016] For each of the positive sample data, construct a positive sample sequence syntax tree based on the subordination relationship between the corresponding at least one positive sample sequence; for each of the negative sample data, construct a negative sample sequence syntax tree based on the subordination relationship between the corresponding at least one negative sample sequence;

[0017] The positive historical feature information of each of the positive sample data is determined based on each of the positive sample sequence syntax trees, and the negative historical feature information of each of the negative sample data is determined based on each of the negative sample sequence syntax trees.

[0018] In the above solution, the step of determining the current feature information of the predetermined type sequence in each of the data to be tested includes:

[0019] Screening out at least one sequence to be tested of a predetermined type from a plurality of third sequences included in each of the data to be tested;

[0020] For each of the data to be tested, constructing a syntax tree of the sequence to be tested based on the subordinate relationship between the corresponding at least one sequence to be tested;

[0021] The current feature information of each of the data to be tested is determined based on each of the syntax trees of the sequence to be tested.

[0022] In the above solution, the step of determining the alarm data based on the feature detection information and program instruction behavior of each of the data to be tested includes:

[0023] Determine the behavior detection information corresponding to each of the data to be tested; wherein the behavior detection information is used to indicate whether the program instruction behavior of the corresponding data to be tested is abnormal;

[0024] The warning data is determined by combining the behavior detection information and the feature detection information corresponding to each of the data to be tested.

[0025] In the above solution, the step of determining the behavior detection information corresponding to each of the data to be tested includes:

[0026] Performing behavior analysis on each third sequence included in each of the data to be tested, and determining a behavior feature corresponding to each of the data to be tested;

[0027] The behavior feature corresponding to each of the data to be tested is matched with the features in a preset feature library to determine the behavior detection information of each of the data to be tested.

[0028] In the above scheme, the matching of the behavior feature corresponding to each of the data to be tested with the features in the preset feature library to determine the behavior detection information of each of the data to be tested includes one of the following:

[0029] If the behavior feature corresponding to each of the data to be tested matches the first feature in the preset feature library, then determining that the behavior detection information of the corresponding data to be tested is a first value; wherein the first feature represents a normal feature;

[0030] If the behavior feature corresponding to each of the data to be tested matches the second feature in the preset feature library, the behavior detection information of the corresponding data to be tested is determined to be a second value; wherein the second feature represents an abnormal feature; and the second value is greater than the first value.

[0031] In the above solution, the step of combining the behavior detection information and the feature detection information corresponding to each of the data to be tested to determine the alarm data includes:

[0032] Weighting the feature detection information based on the value represented by the behavior detection information of each of the data to be tested, to determine final detection information corresponding to each of the data to be tested;

[0033] The alarm data is determined based on a comparison result between the final detection information of each of the data to be tested and a preset threshold.

[0034] The present application also provides a data detection device, including:

[0035] A data acquisition processing unit, used to acquire each data to be tested, and determine the current feature information of a predetermined type sequence in each of the data to be tested;

[0036] A detection unit, used to detect each of the current feature information using a preset detection model to obtain feature detection information corresponding to each of the data to be tested;

[0037] Wherein, the preset detection model is obtained by training based on historical feature information of predetermined type sequences corresponding to positive sample data and negative sample data respectively; the positive sample data and the negative sample data are determined from historical detection data;

[0038] The screening and determining unit is used to determine the alarm data based on the feature detection information and program instruction behavior of each of the data to be tested.

[0039] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above method when executing the computer program.

[0040] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are implemented.

[0041] An embodiment of the present application provides a data detection method, including: obtaining each data to be tested, and determining the current feature information of a predetermined type sequence in each data to be tested; using a preset detection model to detect each current feature information, and obtaining feature detection information corresponding to each data to be tested; wherein the preset detection model is trained based on the historical feature information of the predetermined type sequence corresponding to each of the positive sample data and the negative sample data; the positive sample data and the negative sample data are determined from the historically detected data; based on the feature detection information and program instruction behavior of each data to be tested, the alarm data is determined. In the present application, the features of the predetermined type sequence of the positive sample data and the negative sample data in the historically detected data are automatically learned to form a preset detection model, and there is no need to manually design feature engineering, so it can adapt to different types of malicious texts. The trained preset detection model can learn more robust features for the preset type sequence from a large amount of historical data, and then by detecting the features of the predetermined type sequence of the data to be tested, the accuracy of the detection of the data to be tested can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 An optional flow chart of a data detection method provided in an embodiment of the present application;

[0043] Figure 2 An optional effect diagram of the data detection method provided in the embodiment of the present application;

[0044] Figure 3 An optional flow chart of a data detection method provided in an embodiment of the present application;

[0045] Figure 4 An optional flow chart of a data detection method provided in an embodiment of the present application;

[0046] Figure 5 An optional flow chart of a data detection method provided in an embodiment of the present application;

[0047] Figure 6 An optional flow chart of a data detection method provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of a data detection device provided in an embodiment of the present application;

[0049] Figure 8 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0051] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0052] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0054] In the current terminal protection scenario, JS scripts account for about 60% of non-executable (Portable Executable, PE) malicious files. Currently, only 0.5 rules are used for manpower operation and detection of relevant rules. The existing detection methods have a high false alarm rate and poor adaptability.

[0055] This application embodiment provides a data detection method, see Figure 1 , is an optional flow chart of the data detection method provided in the embodiment of the present application, which will be combined with Figure 1 The steps shown are explained.

[0056] S101, acquiring each piece of data to be tested, and determining current feature information of a predetermined type sequence in each piece of data to be tested.

[0057] In the embodiment of the present application, the data detection device obtains each data to be tested from other terminals and servers. The data detection device can extract a predetermined type sequence from each data to be tested. And perform structured processing on the predetermined type sequence to obtain current feature information corresponding to the predetermined type sequence in each data to be tested.

[0058] The data detection device may be any one of a terminal, a server and a cloud server with corresponding data processing functions. The predetermined type sequence may be a JS sequence in the data to be tested. The current feature information may be a feature vector obtained by structured processing of the predetermined type sequence. The JS sequence refers to a sequence including a JS script. JS script: refers to JavaScript script, a commonly used web programming language used to implement interactive and dynamic effects on web pages.

[0059] In the embodiment of the present application, before extracting the predetermined type of sequence in the test data, the data detection device can also perform noise removal on all sequences in the test data to simplify the processing, delete the sequences without semantic meaning in the test data, and shorten the sequences whose sequence field length exceeds the length threshold.

[0060] S102. Use a preset detection model to detect each of the current feature information to obtain feature detection information corresponding to each of the data to be tested; wherein the preset detection model is trained based on historical feature information of predetermined type sequences corresponding to the positive sample data and the negative sample data respectively; the positive sample data and the negative sample data are determined from historical detection data.

[0061] In an embodiment of the present application, the data detection device can use a preset detection model to detect the current feature information of each data to be tested, and determine the feature detection information corresponding to each data to be tested. The feature detection information is used to characterize the possibility that the corresponding data to be tested is alarm data. The greater the possibility, the greater the score of the feature detection information corresponding to the data to be tested. The preset detection model is trained based on the historical feature information of the predetermined type sequence corresponding to the positive sample data and the negative sample data respectively; the positive sample data and the negative sample data are determined from the historical detection data.

[0062] In an embodiment of the present application, before using a preset detection model to detect the current feature information, it is also necessary to determine positive sample data and negative sample data in the historical detection data. Positive sample data is normal data in the historical detection data, and negative sample data is alarm data in the historical detection data. The historical detection data may be data detected within a certain period of time from the current moment. After detection, positive sample data and negative sample data can be determined from the data detected within this period of time. The data detection device extracts a predetermined type sequence from the positive sample data, and performs structured processing on the predetermined type sequence to obtain corresponding historical feature information. The data detection device can also extract a predetermined type sequence from the negative sample data, and performs structured processing on the predetermined type sequence to obtain corresponding historical feature information. The data detection device trains the detection model obtained from the last training using the historical feature information of the predetermined type sequences corresponding to the positive sample data and the negative sample data, respectively, to obtain a preset detection model.

[0063] S103: Determine warning data based on the feature detection information and program instruction behavior of each of the data to be tested.

[0064] In the embodiment of the present application, the data detection device can also use the preset rule library to perform behavior detection on each data to be tested to obtain behavior detection information corresponding to each data to be tested. The data detection device can combine the behavior detection information and feature detection information of each data to be tested to screen all the data to be tested and determine the alarm data.

[0065] In some other embodiments, the data detection device can also directly compare the score of the feature detection information with a preset threshold value to determine that the data to be tested corresponding to the feature detection information greater than the preset threshold value is the alarm data. After the alarm data is determined, the alarm data and its source can also be displayed to facilitate the user to understand the alarm information.

[0066] An embodiment of the present application provides a data detection method, including: obtaining each data to be tested, and determining the current feature information of a predetermined type sequence in each data to be tested; using a preset detection model to detect each current feature information, and obtaining feature detection information corresponding to each data to be tested; wherein the preset detection model is trained based on the historical feature information of the predetermined type sequence corresponding to each of the positive sample data and the negative sample data; the positive sample data and the negative sample data are determined from the historically detected data; based on the feature detection information and program instruction behavior of each data to be tested, the alarm data is determined. In the present application, the features of the predetermined type sequence of the positive sample data and the negative sample data in the historically detected data are automatically learned to form a preset detection model, and there is no need to manually design feature engineering, so it can adapt to different types of malicious texts. The trained preset detection model can learn more robust features for the preset type sequence from a large amount of historical data, and then by detecting the features of the predetermined type sequence of the data to be tested, the accuracy of the detection of the data to be tested can be improved.

[0067] In some embodiments, see Figure 2 , Figure 2 An optional flow chart of a data detection method provided in an embodiment of the present application is provided. Figure 1 The illustrated S102 may also include S104 to S106 , which will be described in conjunction with each step.

[0068] S104: Acquire each positive sample data and each negative sample data determined after detecting the data in the historical period.

[0069] In the embodiment of the present application, the data detection device obtains each positive sample data and each negative sample data determined after detecting the data in the historical period, wherein the positive sample data is the normal data in the data in the historical period, and the negative sample data is the warning data in the data in the historical period.

[0070] The historical period may be one week or one month before the current time. The specific length of the historical period is not limited in the embodiment of the present application.

[0071] S105: Determine the positive historical feature information of a predetermined type sequence in each of the positive sample data, and the negative historical feature information of a predetermined type sequence in the negative sample data.

[0072] In the embodiment of the present application, the data detection device can extract the corresponding predetermined type sequence (JS sequence) from each positive sample data, and perform structured processing on the predetermined type sequence to obtain the positive historical feature information corresponding to each positive sample data. Similarly, the data detection device can extract the corresponding predetermined type sequence (JS sequence) from each negative sample data, and perform structured processing on the predetermined type sequence to obtain the negative historical feature information corresponding to each negative sample data.

[0073] In the embodiment of the present application, before extracting the predetermined type of sequence from the positive sample data and the negative sample data, the data detection device can also perform noise removal on all sequences in the positive sample data and the negative sample data to simplify the processing. Sequences without semantics in the positive sample data and the negative sample data are deleted, and sequences whose sequence field length exceeds the length threshold are shortened.

[0074] S106: Train the initial detection model based on each of the positive historical feature information and each of the negative historical feature information until the training condition is met, thereby obtaining the preset detection model.

[0075] In an embodiment of the present application, the data detection device can train the initial detection model obtained from the last training based on the positive historical feature information of each positive sample data and the negative historical feature information of each negative sample until the number of training times is reached or the function of the model converges, thereby obtaining a preset detection model.

[0076] The initial detection model is a model trained before the historical period.

[0077] Among them, the preset detection model can be a Transformer model. Transformer model: is a neural network architecture based on the self-attention mechanism, and its core components include an encoder and a decoder. The encoder is responsible for converting the input sequence into a series of high-level feature representations, while the decoder generates a target sequence based on the encoder's output and context information. Each encoder and decoder layer consists of multiple sublayers, including a self-attention layer and a feedforward neural network layer.

[0078] Self-attention layer: maps the input sequence into multiple subspaces and calculates the attention weight in each subspace. This enables the model to pay attention to different semantic information at the same time, thereby improving the richness and generalization of representation. Feedforward neural network layer: two fully connected layers and an activation function are used to perform nonlinear transformation and feature extraction on the representation of each position.

[0079] In an embodiment of the present application, the process of training a preset detection model can be performed by a training module. The training module is used to train a deep learning model. It uses labeled malicious files (negative sample data) and normal files (positive sample data) data for model training, and receives negative historical feature information of malicious files and positive historical feature information of normal files as input for training to obtain a preset detection model. The main techniques used include label smoothing, weighted loss, warm up, etc. to improve the generalization ability of the model. Malicious file: refers to a file containing malicious code or behavior that may pose a threat to system security, such as viruses, Trojans, etc.

[0080] In the embodiment of the present application, each positive sample data and each negative sample data are obtained through the data detected in the historical period, and the initial detection model obtained by the previous training is trained based on each positive sample data and each negative sample data to obtain a preset detection model. The preset detection model is formed by automatically learning features of the positive sample data and negative sample data in the historical detection data, and there is no need to manually design feature engineering, so it can adapt to different types of malicious files, and can automatically adapt to new malicious file types.

[0081] In some embodiments, see Figure 3 , Figure 3 An optional flow chart of a data detection method provided in an embodiment of the present application is provided. Figure 2 The illustrated S105 may also include S107 to S109 for implementation, which will be described in conjunction with each step.

[0082] S107 . Filter out at least one positive sample sequence of a predetermined type from the multiple first sequences included in each of the positive sample data, and filter out at least one negative sample sequence of a predetermined type from the multiple second sequences included in each of the negative sample data.

[0083] In the embodiment of the present application, the data detection device can filter out at least one positive sample sequence containing a JS script from the multiple first sequences included in each positive sample data, and can filter out at least one negative sample sequence containing a JS script from the multiple second sequences included in each negative sample data.

[0084] In an embodiment of the present application, the data detection device can delete the first sequences without semantics for multiple first sequences and shorten the first sequences with longer sequence lengths. Then, at least one positive sample sequence containing a JS script is extracted. Similarly, the data detection device can use the same method to screen at least one negative sample sequence in the negative sample data.

[0085] S108. For each of the positive sample data, construct a positive sample sequence syntax tree based on the subordination relationship between the corresponding at least one positive sample sequence; for each of the negative sample data, construct a negative sample sequence syntax tree based on the subordination relationship between the corresponding at least one negative sample sequence.

[0086] In an embodiment of the present application, the data detection device may construct a positive sample sequence syntax tree for each positive sample data based on the subordination relationship between at least one corresponding positive sample sequence. The data detection device may construct a positive sample sequence syntax tree for each negative sample data based on the subordination relationship between at least one corresponding negative sample sequence.

[0087] In an embodiment of the present application, the data detection device may use the tree-sitter method to process at least one positive sample sequence to obtain a corresponding positive sample sequence syntax tree. The data detection device may use the tree-sitter method to process at least one negative sample sequence to obtain a corresponding negative sample sequence syntax tree.

[0088] Among them, the tree-sitter method is a parser generator tool and incremental parsing library, which can build a specific syntax tree for the source code file. And when editing the original file, it is efficient to update the syntax tree. It supports the parsing of multiple programming languages, such as Python, Java, C, etc., and supports the use of multiple programming languages.

[0089] S109: Determine positive historical feature information of each positive sample data based on each positive sample sequence syntax tree, and determine negative historical feature information of each negative sample data based on each negative sample sequence syntax tree.

[0090] In the embodiment of the present application, the data detection device can perform structural processing on the positive sample sequence syntax tree of each positive sample data to determine the positive historical feature information of each positive sample data. Similarly, the data detection device can perform structural processing on the negative sample sequence syntax tree of each negative sample data to determine the negative historical feature information of each negative sample data.

[0091] In an embodiment of the present application, the data detection device can tokenize each positive sample sequence syntax tree to obtain positive historical feature information that can be processed by a computer. Similarly, the data detection device can use the same method to process and obtain the corresponding negative historical feature information. Among them, the tokenize technology can segment the natural language text into meaningful units, and then identify the words, phrases or sentences in the text, and process the words, phrases or sentences in the text to obtain the corresponding feature information.

[0092] In an embodiment of the present application, a data detection device extracts a positive sample sequence and a negative sample sequence from each positive sample data and each negative sample data, uses the positive sample sequence and the negative sample sequence to form corresponding sequence trees respectively, and then structures the sequence trees to obtain historical feature information corresponding to each positive sample data and each negative sample data. This process only analyzes the preset type sequence in each positive sample data and each negative sample data to obtain the historical feature information corresponding to each positive sample data and each negative sample data. This process can accurately extract the features of the JS sequence in each positive sample data and each negative sample data, and the preset detection model trained using the features is also more accurate in detecting the data.

[0093] In some embodiments, see Figure 4 , Figure 4 An optional flow chart of a data detection method provided in an embodiment of the present application is provided. Figure 1 The illustrated S101 may also be implemented through S110 to S112 , which will be described in conjunction with each step.

[0094] S110: Screen out at least one sequence to be tested of a predetermined type from a plurality of third sequences included in each of the data to be tested.

[0095] In the embodiment of the present application, the data detection device can screen out at least one sequence to be tested in which a JS script exists from a plurality of third sequences included in each data to be tested.

[0096] In the embodiment of the present application, the data detection device can delete the third sequences without semantic meanings from the multiple third sequences and shorten the third sequences with longer sequence lengths, and then extract at least one sequence to be tested that contains a JS script.

[0097] S111 . For each of the data to be tested, construct a syntax tree of a sequence to be tested based on a subordinate relationship between the corresponding at least one sequence to be tested.

[0098] In the embodiment of the present application, the data detection device may construct a syntax tree of a sequence to be tested based on the subordinate relationship between the corresponding at least one sequence to be tested for each data to be tested.

[0099] In the embodiment of the present application, the data detection device may use a tree-sitter method to process at least one sequence to be tested to obtain a corresponding syntax tree of the sequence to be tested.

[0100] S112: Determine the current feature information of each of the data to be tested based on each of the syntax trees of the sequence to be tested.

[0101] In the embodiment of the present application, the data detection device can perform structured processing on the syntax tree of the sequence to be tested of each data to be tested, and determine the current feature information of each data to be tested.

[0102] In an embodiment of the present application, the data detection device can process the test data through the preprocessing module to obtain the corresponding current feature information. Preprocessing module: mainly responsible for preprocessing the input test data. Some non-semantic sequences are deleted through regular expressions to shorten the length of the sequence, such as extracting the code in the script tag in the Hypertext Markup Language (HTML) file, deleting comments, deleting non-JS code, etc. The main function of this module is to let the model focus on the black features of the JS code syntax. Secondly, the tree-sitter method is used to extract the processed sequence into a syntax tree, and then the syntax tree is tokenized and converted into a format that can be input by the model.

[0103] In the embodiment of the present application, at least one sequence to be tested of a preset type in each data to be tested is extracted, thereby reducing the sequence noise in the data to be tested. Moreover, a corresponding sequence tree is formed through at least one sequence to be tested to obtain current feature information. This process is a fine-grained analysis of the data to be tested, which can fully obtain the features of the data to be tested, thereby improving the accuracy of the detection results of the data to be tested.

[0104] In some embodiments, see Figure 5 , Figure 5 An optional flow chart of a data detection method provided in an embodiment of the present application is provided. Figure 1 The illustrated S103 may also be implemented through S113 to S114, which will be described in conjunction with each step.

[0105] S113, determining behavior detection information corresponding to each of the data to be tested; wherein the behavior detection information is used to indicate whether the program instruction behavior of the corresponding data to be tested is abnormal.

[0106] In the embodiment of the present application, the data detection device can use a preset program to analyze each data to be tested to determine the corresponding behavior detection information, wherein the behavior detection information is used to indicate whether the program instruction behavior of the corresponding data to be tested is abnormal.

[0107] In the embodiment of the present application, the data detection device can use a preset program to perform behavioral analysis on each data to be tested, and determine the behavioral characteristics corresponding to each data to be tested. The behavioral characteristics are matched with the characteristics in the preset rule library, blacklist and whitelist to determine the behavior detection information corresponding to the data to be tested. Among them, the behavior detection information can be a numerical value, and the larger the numerical value, the more abnormal the behavior characteristics of the data to be tested are.

[0108] S114: Determine alarm data by combining the behavior detection information and the feature detection information corresponding to each of the data to be tested.

[0109] In the embodiment of the present application, the data detection device can weight the values ​​corresponding to the behavior detection information and feature detection information corresponding to each data to be detected, and determine the final detection information. Based on the comparison result between the final detection information and the preset threshold, the alarm data is determined.

[0110] In the embodiment of the present application, the data detection device also includes an inference module and a rule module. The inference module is used to perform malicious detection on the data to be tested. It receives the current feature information extracted by the preprocessing module as input, and uses the model trained by the training module for inference. In the terminal protection scenario, there are a large number of samples to be detected, so inference acceleration is required, among which the main techniques used include model quantization, distributed inference, etc. to improve the overall performance of the model.

[0111] Rule module: The rule module is mainly responsible for further processing and judging the data to be tested. The rule module usually includes a rule base, a blacklist, a whitelist, etc. The rule module can filter, modify, and weight the results output by the inference module to improve the accuracy and robustness of the detection.

[0112] In some embodiments, see Figure 6 , Figure 6 An optional flow chart of a data detection method provided in an embodiment of the present application is provided. Figure 5 The illustrated S113 to S114 may also be implemented through S115 to S118, which will be described in conjunction with each step.

[0113] S115 , performing behavior analysis on each third sequence included in each of the data to be tested, and determining a behavior feature corresponding to each of the data to be tested.

[0114] In the embodiment of the present application, each data to be tested includes a plurality of third sequences, and a preset program can be used to perform a behavior analysis on each third sequence to determine the behavior characteristics of the data to be tested.

[0115] In some other embodiments, behavior analysis may be performed on the JS sequence included in each data to be tested to determine the behavior characteristics of each data to be tested.

[0116] S116: Match the behavior feature corresponding to each of the data to be tested with the features in a preset feature library to determine the behavior detection information of each of the data to be tested.

[0117] In an embodiment of the present application, if the behavior feature corresponding to each of the data to be tested matches the first feature in the preset feature library, it is determined that the behavior detection information of the corresponding data to be tested is a first value; wherein the first feature represents a normal feature;

[0118] If the behavior feature corresponding to each of the data to be tested matches the second feature in the preset feature library, the behavior detection information of the corresponding data to be tested is determined to be a second value; wherein the second feature represents an abnormal feature; and the second value is greater than the first value.

[0119] In an embodiment of the present application, the preset feature library includes multiple first features and multiple second features. If the behavior feature of the data to be tested matches any of the first features, the behavior detection information of the data to be tested is determined to be the first value. An exemplary value may be 1. If the behavior feature of the data to be tested matches any of the second features, the behavior detection information of the data to be tested is determined to be the second value, an exemplary value may be 2.

[0120] S117 . Weight the feature detection information based on the value represented by the behavior detection information of each of the data to be tested, and determine final detection information corresponding to each of the data to be tested.

[0121] In an embodiment of the present application, the data detection device may multiply the numerical value represented by the behavior detection information with the numerical value represented by the feature detection information to determine the numerical value represented by the final detection information.

[0122] In an embodiment of the present application, when the data to be tested includes multiple behavior detection information, the values ​​of the multiple behavior detection information can be added and then multiplied by the value of the feature detection information to determine the value represented by the final detection information.

[0123] S118: Determine the alarm data based on a comparison result between the final detection information of each of the data to be tested and a preset threshold.

[0124] In the embodiment of the present application, the data detection device can compare the value of the final detection information with the preset threshold value, and if the value of the final detection information is greater than the preset threshold value, the corresponding data to be tested is determined to be warning data. If the value of the final detection information is not greater than the preset threshold value, the corresponding data to be tested is determined to be normal data.

[0125] In the embodiment of the present application, the output layer of the data detection device is used to extract the JS file whose value of the final detection information is greater than the threshold and push it to the corresponding host.

[0126] In the embodiment of the present application, the feature detection information is weighted based on the value represented by the behavior detection information of each of the data to be tested, and the final detection information corresponding to each of the data to be tested is determined. The alarm data is determined based on the comparison result between the final detection information and the preset threshold value. This process fully considers the sequence characteristics and behavior characteristics of the data to be tested, and can improve the detection accuracy of the alarm data.

[0127] The embodiment of the present application provides a classification model based on a large model: pre-processing unknown files through regular expressions, extracting the syntax tree of the file through tree-sitter, and then calculating the sample score through the large model, using the blacklist and whitelist rules to weight the score. Finally, a method for obtaining malicious JS files is obtained.

[0128] Among them, the big model: The core of the big model technology is the deep learning algorithm, which is an algorithm that simulates the human brain neural network. It realizes high-level feature extraction and classification of data through the combination and calculation of multiple layers of neurons. The deep learning algorithm can process various types of data, including images, voice, text, etc., and has strong generalization ability and adaptability. Llama: It is a huge model built based on deep learning technology, with more than 1.5 trillion parameters, which can process large-scale natural language data. The core of Llama is the Transformer architecture, which is a neural network structure based on the self-attention mechanism. It can process long text sequences and can capture contextual information and semantic relationships in the text.

[0129] See also Figure 7 , Figure 7 A schematic diagram of the structure of a data detection device provided in an embodiment of the present application.

[0130] The embodiment of the present application further provides a data detection device 600 , including: a data acquisition processing unit 601 , a detection unit 602 , and a screening and determination unit 603 .

[0131] The data acquisition processing unit 601 is used to acquire each data to be tested and determine the current feature information of the predetermined type sequence in each of the data to be tested;

[0132] The detection unit 602 is used to detect each of the current feature information using a preset detection model to obtain feature detection information corresponding to each of the data to be tested;

[0133] Wherein, the preset detection model is obtained by training based on historical feature information of predetermined type sequences corresponding to positive sample data and negative sample data respectively; the positive sample data and the negative sample data are determined from historical detection data;

[0134] The screening and determining unit 603 is used to determine the warning data based on the feature detection information and program instruction behavior of each of the data to be tested.

[0135] In an embodiment of the present application, the data acquisition processing unit 601 in the data detection device 600 is used to obtain each of the positive sample data and each of the negative sample data determined after detecting the data within the historical period; determine the positive historical feature information of the predetermined type sequence in each of the positive sample data, and the negative historical feature information of the predetermined type sequence in the negative sample data; the detection unit 602 is used to train the initial detection model based on each of the positive historical feature information and each of the negative historical feature information, and stop when the training condition is met, to obtain the preset detection model; wherein the initial detection model is the model trained before the historical period.

[0136] In the embodiment of the present application, the data acquisition processing unit 601 in the data detection device 600 is used to filter out at least one positive sample sequence of a predetermined type from the multiple first sequences included in each of the positive sample data, and filter out at least one negative sample sequence of a predetermined type from the multiple second sequences included in each of the negative sample data;

[0137] For each of the positive sample data, construct a positive sample sequence syntax tree based on the subordination relationship between the corresponding at least one positive sample sequence; for each of the negative sample data, construct a negative sample sequence syntax tree based on the subordination relationship between the corresponding at least one negative sample sequence;

[0138] The positive historical feature information of each of the positive sample data is determined based on each of the positive sample sequence syntax trees, and the negative historical feature information of each of the negative sample data is determined based on each of the negative sample sequence syntax trees.

[0139] In the embodiment of the present application, the data acquisition processing unit 601 in the data detection device 600 is used to select at least one sequence to be tested of a predetermined type from a plurality of third sequences included in each of the data to be tested;

[0140] For each of the data to be tested, constructing a syntax tree of the sequence to be tested based on the subordinate relationship between the corresponding at least one sequence to be tested;

[0141] The current feature information of each of the data to be tested is determined based on each of the syntax trees of the sequence to be tested.

[0142] In the embodiment of the present application, the screening and determining unit 603 in the data detection device 600 is used to determine the behavior detection information corresponding to each of the data to be tested; wherein the behavior detection information is used to characterize the program instruction behavior of the corresponding data to be tested;

[0143] The warning data is determined by combining the behavior detection information and the feature detection information corresponding to each of the data to be tested.

[0144] In the embodiment of the present application, the screening and determining unit 603 in the data detection device 600 is used to perform a behavior analysis on each third sequence included in each of the data to be tested, and determine a behavior feature corresponding to each of the data to be tested;

[0145] The behavior feature corresponding to each of the data to be tested is matched with the features in a preset feature library to determine the behavior detection information of each of the data to be tested.

[0146] In the embodiment of the present application, the screening and determining unit 603 in the data detection device 600 is used to determine that the behavior detection information of the corresponding data to be tested is a first value if the behavior feature corresponding to each of the data to be tested matches the first feature in the preset feature library; wherein the first feature represents a normal feature;

[0147] If the behavior feature corresponding to each of the data to be tested matches the second feature in the preset feature library, the behavior detection information of the corresponding data to be tested is determined to be a second value; wherein the second feature represents an abnormal feature; and the second value is greater than the first value.

[0148] In the embodiment of the present application, the screening and determining unit 603 in the data detection device 600 is used to weight the feature detection information based on the value represented by the behavior detection information of each of the data to be tested, and determine the final detection information corresponding to each of the data to be tested;

[0149] The alarm data is determined based on a comparison result between the final detection information of each of the data to be tested and a preset threshold.

[0150] It should be noted that in the embodiment of the present application, if the above-mentioned data detection method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a data detection device (which can be a personal computer, etc.) to execute all or part of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0151] Correspondingly, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method on the data detection device side are implemented.

[0152] Correspondingly, an embodiment of the present application provides an electronic device 700, including a memory 702 and a processor 701, wherein the memory 702 stores a computer program that can be executed on the processor 701, and the processor 701 implements the steps in the above method when executing the program.

[0153] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0154] It should be noted that Figure 8 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Figure 8 As shown, the hardware entity of the electronic device 700 includes: a processor 701 and a memory 702, wherein;

[0155] The processor 701 generally controls the overall operation of the electronic device 700 .

[0156] The memory 702 is configured to store instructions and applications executable by the processor 701, and can also cache data to be processed or processed by the processor 701 and various modules in the electronic device 700 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0157] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0158] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0159] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0160] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0161] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0162] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a magnetic disk or an optical disk, and other media that can store program codes.

[0163] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a mobile storage device, a ROM, a magnetic disk, or an optical disk.

[0164] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data detection method, characterized in that: include: Acquire each piece of data to be tested, and determine current feature information of a predetermined type sequence in each piece of data to be tested; Detect each of the current feature information using a preset detection model to obtain feature detection information corresponding to each of the data to be tested; Wherein, the preset detection model is obtained by training based on historical feature information of predetermined type sequences corresponding to positive sample data and negative sample data respectively; the positive sample data and the negative sample data are determined from historical detection data; Based on the feature detection information and program instruction behavior of each of the data to be tested, alarm data is determined.

2. The data detection method according to claim 1, characterized in that: The historical feature information includes: positive historical feature information and negative historical feature information; before using the preset detection model to detect each of the current feature information to obtain the feature detection information corresponding to each of the data to be tested, the method also includes: Obtain each of the positive sample data and each of the negative sample data determined after testing the data in the historical period; Determine the positive historical feature information of a predetermined type sequence in each of the positive sample data, and the negative historical feature information of a predetermined type sequence in the negative sample data; The initial detection model is trained based on each of the positive historical feature information and each of the negative historical feature information until the training condition is met, thereby obtaining the preset detection model; wherein the initial detection model is a model trained before the historical period.

3. The data detection method according to claim 2, characterized in that: The determining of the positive historical feature information of a predetermined type sequence in each of the positive sample data and the negative historical feature information of a predetermined type sequence in the negative sample data includes: Screening out at least one positive sample sequence of a predetermined type from a plurality of first sequences included in each of the positive sample data, and screening out at least one negative sample sequence of a predetermined type from a plurality of second sequences included in each of the negative sample data; For each of the positive sample data, construct a positive sample sequence syntax tree based on the subordination relationship between the corresponding at least one positive sample sequence; for each of the negative sample data, construct a negative sample sequence syntax tree based on the subordination relationship between the corresponding at least one negative sample sequence; The positive historical feature information of each of the positive sample data is determined based on each of the positive sample sequence syntax trees, and the negative historical feature information of each of the negative sample data is determined based on each of the negative sample sequence syntax trees.

4. The data detection method according to any one of claims 1 to 3, characterized in that: The determining of the current feature information of the predetermined type sequence in each of the data to be tested includes: Screening out at least one sequence to be tested of a predetermined type from a plurality of third sequences included in each of the data to be tested; For each of the data to be tested, constructing a syntax tree of a sequence to be tested based on a subordinate relationship between the corresponding at least one sequence to be tested; The current feature information of each of the data to be tested is determined based on each of the syntax trees of the sequence to be tested.

5. The data detection method according to any one of claims 1 to 3, characterized in that: The step of determining the alarm data based on the feature detection information and program instruction behavior of each of the data to be tested includes: Determine the behavior detection information corresponding to each of the data to be tested; wherein the behavior detection information is used to indicate whether the program instruction behavior of the corresponding data to be tested is abnormal; The warning data is determined by combining the behavior detection information and the feature detection information corresponding to each of the data to be tested.

6. The data detection method according to claim 5, characterized in that: The determining of the behavior detection information corresponding to each of the data to be tested includes: Performing behavior analysis on each third sequence included in each of the data to be tested, and determining a behavior feature corresponding to each of the data to be tested; The behavior feature corresponding to each of the data to be tested is matched with the features in a preset feature library to determine the behavior detection information of each of the data to be tested.

7. The data detection method according to claim 6, characterized in that: The step of matching the behavior feature corresponding to each of the data to be tested with features in a preset feature library to determine the behavior detection information of each of the data to be tested includes one of the following: If the behavior feature corresponding to each of the data to be tested matches the first feature in the preset feature library, then determining that the behavior detection information of the corresponding data to be tested is a first value; wherein the first feature represents a normal feature; If the behavior feature corresponding to each of the data to be tested matches the second feature in the preset feature library, the behavior detection information of the corresponding data to be tested is determined to be a second value; wherein the second feature represents an abnormal feature; and the second value is greater than the first value.

8. The data detection method according to claim 5, characterized in that: The combining the behavior detection information and the feature detection information corresponding to each of the data to be tested to determine the alarm data includes: Weighting the feature detection information based on the value represented by the behavior detection information of each of the data to be tested, to determine final detection information corresponding to each of the data to be tested; The alarm data is determined based on a comparison result between the final detection information of each of the data to be tested and a preset threshold.

9. A data detection device, characterized in that: include: A data acquisition processing unit, used to acquire each data to be tested, and determine the current feature information of a predetermined type sequence in each of the data to be tested; A detection unit, used to detect each of the current feature information using a preset detection model to obtain feature detection information corresponding to each of the data to be tested; Wherein, the preset detection model is obtained by training based on historical feature information of predetermined type sequences corresponding to positive sample data and negative sample data respectively; the positive sample data and the negative sample data are determined from historical detection data; The screening and determining unit is used to determine the alarm data based on the feature detection information and program instruction behavior of each of the data to be tested.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.

Images