Feature extraction method and device, and storage medium

Abstract

The application provides a data detection method and device and a storage medium. The method comprises the following steps: obtaining to-be-detected data, and dividing a plurality of sub-data in the to-be-detected data into a plurality of groups of sub-data; detecting, by means of a jump detection mode, whether predetermined sub-data in each group of sub-data is valid data; if the predetermined sub-data in the Mth group of sub-data is valid data, detecting whether other sub-data in the Mth group of sub-data is valid data; the other sub-data is sub-data in the Mth group of sub-data except the predetermined sub-data; when the other sub-data is all valid data, determining that the Mth group of sub-data is a group of characteristic data; and until the detection of the plurality of sub-data is completed, a plurality of groups of characteristic data are obtained. In the process of determining a group of characteristic data, the predetermined sub-data in each group of sub-data is detected in a jump detection mode, and the traversal of all sub-data is not performed, so that the detection time is saved, and the extraction efficiency of data characteristics is effectively improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, and particularly relate to a feature extraction method and device and storage medium. BACKGROUND

[0002] In related technologies, machine learning is generally used for malicious feature extraction, which generally includes file scanning, feature extraction, Artificial Intelligence (AI) prediction and identification, etc. In related technologies, feature extraction often needs to perform full file Input / Output (IO) feature screening and extraction on the entire file, which results in a long time consumption for scanning files and a low efficiency for feature extraction. SUMMARY

[0003] The feature extraction method, device and storage medium provided by the embodiments of the present application can improve the extraction efficiency of data features.

[0004] The technical solution of the present application is implemented as follows:

[0005] The feature extraction method provided by the embodiments of the present application comprises the following steps:

[0006] Obtaining to-be-tested data, and dividing a plurality of sub-data in the to-be-tested data into a plurality of groups of sub-data;

[0007] Detecting whether predetermined sub-data in each group of sub-data is valid data by a skip detection manner;

[0008] If the predetermined sub-data in the Mth group of sub-data is the valid data, detecting whether other sub-data in the Mth group of sub-data is the valid data; the other sub-data is sub-data in the Mth group of sub-data except the predetermined sub-data; M is an integer greater than or equal to 1;

[0009] When the other sub-data are all valid data, determining that the Mth group of sub-data is a group of feature data, until the detection of the plurality of sub-data is completed, and a plurality of groups of feature data are obtained.

[0010] In the above solution, each group of sub-data comprises N sub-data; N is an integer greater than 1;

[0011] The detection of whether the predetermined sub-data in each group of sub-data is valid data by the skip detection manner comprises one of the following:

[0012] Detecting whether the Nth sub-data in each group of sub-data is the valid data by the skip detection manner;

[0013] detecting whether the first sub-data and the Nth sub-data in each group of sub-data are the valid data through a skip detection manner;

[0014] detecting whether the Tth sub-data in each group of sub-data is the valid data through a skip detection manner; T is an integer greater than or equal to 1 and less than or equal to N.

[0015] In the above scheme, after detecting whether the predetermined sub-data in each group of sub-data is the valid data through the skip detection manner, the method further comprises:

[0016] If the predetermined sub-data in the Mth group of sub-data is not the valid data, detecting whether the predetermined sub-data in the M+1th group of sub-data is the valid data until the detection of the plurality of sub-data is completed to obtain the plurality of groups of feature data.

[0017] In the above scheme, the detecting whether the other sub-data in the Mth group of sub-data is the valid data comprises:

[0018] sequentially detecting whether each of the other sub-data is the valid data according to the order of the other sub-data in the Mth group of sub-data.

[0019] In the above scheme, after the detecting whether the other sub-data in the Mth group of sub-data is the valid data if the predetermined sub-data in the Mth group of sub-data is the valid data, the method further comprises:

[0020] If there is non-valid data in the other sub-data, detecting whether the predetermined sub-data in the M+1th group of sub-data is the valid data until the detection of the plurality of sub-data is completed to obtain the plurality of groups of feature data.

[0021] In the above scheme, after the determining that the Mth group of sub-data is a group of feature data when the other sub-data are all valid data until the detection of the plurality of sub-data is completed to obtain the plurality of groups of feature data, the method further comprises:

[0022] detecting the plurality of groups of feature data respectively to determine the target detection result of the to-be-detected data.

[0023] In the above scheme, the detecting the plurality of groups of feature data respectively to determine the target detection result of the to-be-detected data comprises:

[0024] respectively performing black and white identification on the plurality of groups of feature data to obtain an identification result of each group of feature data, and further obtain a plurality of identification results;

[0025] calculating a proportion of a first result in the plurality of identification results;

[0026] If the ratio is greater than a predetermined threshold, the target detection result that the to-be-tested data is abnormal data is determined.

[0027] The embodiment of the present application further provides a feature extraction device, which comprises:

[0028] a data acquisition unit, configured to acquire to-be-tested data, and divide a plurality of sub-data in the to-be-tested data into a plurality of groups of sub-data;

[0029] a detection unit, configured to detect whether predetermined sub-data in each group of sub-data is valid data in a skip detection manner;

[0030] the detection unit is further configured to, if the predetermined sub-data in the Mth group of sub-data is valid data, detect whether other sub-data in the Mth group of sub-data is valid data; the other sub-data is sub-data in the Mth group of sub-data except the predetermined sub-data; M is an integer greater than or equal to 1;

[0031] a determination unit, configured to, when the other sub-data are all valid data, determine that the Mth group of sub-data is a group of feature data, and obtain a plurality of groups of feature data until the detection of the plurality of sub-data is completed.

[0032] The embodiment of the present application further provides a feature extraction device, which comprises a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements the steps in the above method when executing the program.

[0033] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in the above method.

[0034] In the embodiment of the present application, the to-be-tested data is acquired, and the plurality of sub-data in the to-be-tested data is divided into a plurality of groups of sub-data; the skip detection manner is used to detect whether predetermined sub-data in each group of sub-data is valid data; if the predetermined sub-data in the Mth group of sub-data is valid data, the detection of whether other sub-data in the Mth group of sub-data is valid data is performed; the other sub-data is sub-data in the Mth group of sub-data except the predetermined sub-data; M is an integer greater than or equal to 1; when the other sub-data are all valid data, the Mth group of sub-data is determined to be a group of feature data, and a plurality of groups of feature data are obtained until the detection of the plurality of sub-data is completed. Since the skip detection manner is used to detect a small amount of predetermined sub-data in each group of sub-data in the process of determining a group of feature data, the traversal of all sub-data is not performed, the detection time is saved, and the extraction efficiency of data features is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1An optional flowchart of the feature extraction method provided by the embodiment of the present application is shown in the figure.

[0036] Figure 2 An optional flowchart of the feature extraction method provided by the embodiment of the present application is shown in the figure.

[0037] Figure 3 A structure diagram of the feature extraction device provided by the embodiment of the present application is shown in the figure.

[0038] Figure 4 A hardware entity diagram of the feature extraction device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0039] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further described in detail below in combination with the drawings and embodiments, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0040] In the following description, “some embodiments” are described, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0041] If the description of “first / second” appears in the invention document, the following description is added, in the following description, the terms “first\second\third” only distinguish similar objects, and do not represent the specific order of the objects, and it can be understood that “first\second\third” can be interchanged in the specific order or sequence as allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0043] Figure 1 An optional flowchart of the feature extraction method provided by the embodiment of the present application is shown in the figure, which will be described in combination with Figure 1 the steps shown in the figure.

[0044] S101, acquiring to-be-tested data, and dividing a plurality of sub-data in the to-be-tested data into a plurality of groups of sub-data.

[0045] In the embodiment of the present application, the feature extraction device acquires the to-be-tested data, and divides a plurality of sub-data in the to-be-tested data into a plurality of groups of sub-data. The to-be-tested data includes a plurality of sub-data.

[0046] In the embodiment of the present application, the feature extraction device can be a terminal or a server with corresponding extraction and detection functions. The to-be-tested data can include text data or other data composed of characters. Correspondingly, the plurality of sub-data can include a plurality of characters in the to-be-tested data.

[0047] In the embodiment of the present application, the feature extraction device can extract the to-be-tested data from a disk thereof. The feature extraction device can also receive the to-be-tested data sent by other terminals or servers.

[0048] In the embodiment of the present application, the feature extraction device can extract the features of the to-be-tested data by a machine learning method, and then identify the features of the to-be-tested data.

[0049] Machine learning is a branch of artificial intelligence. The research history of artificial intelligence has a natural and clear context from focusing on "reasoning" to focusing on "knowledge" and then focusing on "learning". Machine learning is a way to realize artificial intelligence, that is, to solve problems in artificial intelligence by means of machine learning. Machine learning has developed into a multi-disciplinary subject in the past 30 years, involving probability theory, statistics, approximation theory, convex analysis, computational complexity theory, and other disciplines. Machine learning theory mainly designs and analyzes algorithms that allow computers to automatically "learn". Machine learning algorithms are a class of algorithms that automatically analyze rules from data and use the rules to predict unknown data. Because learning algorithms involve a lot of statistical theory, machine learning is particularly closely related to inferential statistics and is also known as statistical learning theory. In terms of algorithm design, machine learning theory focuses on learning algorithms that can be implemented and effective. Many inference problems are difficult to program, so some machine learning research is to develop approximate algorithms that are easy to handle.

[0050] Feature extraction has many applications in machine learning, pattern recognition, and image processing. Feature extraction starts from an initial measured data set, and then constructs derived values that are informative and non-redundant, called feature values. It can help the subsequent learning process and induction steps, and in some cases it can make it easier for people to better interpret the data. Feature extraction is a step of reducing dimensionality, and the initial data set is reduced to a more manageable population (features) for learning, while maintaining the accuracy and integrity of the original data set.

[0051] S102, detecting whether the predetermined sub-data in each group of sub-data is valid data by a way of skip detection.

[0052] In the embodiment of the present application, the feature extraction device detects whether the predetermined sub-data in each group of sub-data is valid data by a way of skip detection.

[0053] In the embodiment of the present application, the feature extraction device can detect whether any one or two discontinuous sub-data in each group of sub-data is valid data.

[0054] In the embodiment of the present application, the predetermined sub-data being valid data can include that the predetermined sub-data is an English letter.

[0055] The predetermined sub-data can be one or two sub-data in a corresponding group of sub-data.

[0056] S103, if the predetermined sub-data in the Mth group of sub-data is valid data, detecting whether other sub-data in the Mth group of sub-data is valid data.

[0057] In the embodiment of the present application, if the predetermined sub-data in the Mth group of sub-data is valid data, the feature extraction device detects whether other sub-data in the Mth group of sub-data is valid data. The other sub-data is sub-data in the Mth group of sub-data except the predetermined sub-data; M is an integer greater than or equal to 1.

[0058] In the embodiment of the present application, the feature extraction device can divide the to-be-detected data into 10 groups of sub-data. The feature extraction device detects that the predetermined sub-data in the 1st group of sub-data is a letter, and then continues to detect whether other sub-data in the 1st group of sub-data is a letter.

[0059] In the embodiment of the present application, each group of sub-data includes N sub-data. N is an integer greater than 1. If the feature extraction device detects that the Nth sub-data in the Mth group of sub-data is valid data, the feature extraction device detects whether the 1st sub-data to the N-1th sub-data in the Mth group of sub-data is valid data.

[0060] For example, each group of sub-data includes 5 characters. The feature extraction device detects the 1st character, determines that the 1st character is valid data, and then skips three characters to detect the 5th character. If the feature extraction device detects that the 5th character is also valid, the feature extraction device detects the 2nd character to the 4th character respectively to detect whether the 2nd character to the 4th character is valid data.

[0061] S104, when the other sub-data is all valid data, determining that the Mth group of sub-data is a group of feature data, until the detection of multiple sub-data is completed, and multiple groups of feature data are obtained.

[0062] In the embodiment of the present application, when the other sub-data are all valid data, the feature extraction device determines the Mth group of sub-data as a group of feature data, until the detection of the plurality of sub-data is completed, and a plurality of groups of feature data are obtained.

[0063] In the embodiment of the present application, if the feature extraction device detects that the other sub-data are all valid data, the feature extraction device forms a group of feature data through the Mth group of sub-data. The feature extraction device continues to perform the jump traversal detection on the remaining each group of sub-data, until the plurality of groups of sub-data are traversed, and a plurality of groups of feature data are determined.

[0064] For example, the first group of sub-data can include five sub-data. In the case that the first sub-data (predetermined sub-data) and the fifth sub-data (predetermined sub-data) are valid data, if the feature extraction device detects that the second sub-data to the fourth sub-data are all valid data, the feature extraction device determines that the first group of sub-data is a group of feature data. If the feature extraction device detects that there is invalid data in the second sub-data to the fourth sub-data, the feature extraction device continues to start the jump traversal detection on the second group of sub-data, until the feature extraction device traverses the plurality of groups of sub-data and determines a plurality of groups of feature data.

[0065] In the embodiment of the present application, the feature extraction device can also detect the plurality of groups of feature data respectively to determine the target detection result of the to-be-detected data.

[0066] In the embodiment of the present application, the feature extraction device detects the plurality of groups of feature data respectively to obtain a plurality of identification results. The feature extraction device calculates the proportion of the first result in the plurality of identification results, and if the proportion is greater than a predetermined threshold, determines that the target detection result of the to-be-detected data is abnormal data, and if the proportion is not greater than the predetermined threshold, determines that the target detection result of the to-be-detected data is normal data.

[0067] In the embodiment of the present application, the to-be-detected data is obtained, and the plurality of sub-data in the to-be-detected data are divided into a plurality of groups of sub-data. The predetermined sub-data in each group of sub-data is detected by a jump detection manner. If the predetermined sub-data in the Mth group of sub-data is valid data, the other sub-data in the Mth group of sub-data are detected. The other sub-data are the sub-data in the Mth group of sub-data except the predetermined sub-data. M is an integer greater than or equal to 1. When the other sub-data are all valid data, the Mth group of sub-data is determined as a group of feature data, until the detection of the plurality of sub-data is completed, and a plurality of groups of feature data are obtained. In the process of determining a group of feature data, the predetermined sub-data in each group of sub-data is detected by a jump detection manner, and the traversal of all sub-data is not performed, so that the detection time is saved, and the extraction efficiency of the data feature is effectively improved.

[0068] In some embodiments, Figure 1 The S102 shown can be implemented by S105, which will be described in combination with the steps.

[0069] S105, detects whether the Nth sub-data in each group of sub-data is valid data in a skip detection manner.

[0070] In the embodiment of the application, the feature extraction device detects whether the Nth sub-data in each group of sub-data is valid data in a skip detection manner. Each group of sub-data includes N sub-data, and N is an integer greater than 1.

[0071] In the embodiment of the application, the feature extraction device can detect the Nth sub-data in each group of sub-data by using a preset program, determine whether the Nth sub-data is a valid character, and obtain a detection result of the Nth sub-data.

[0072] For example, each group of sub-data includes 5 sub-data. The feature extraction device detects whether the 5th sub-data in each group of sub-data is valid data in a skip detection manner.

[0073] In the embodiment of the application, when the Nth sub-data is a valid character, the feature extraction device detects other sub-data. The feature extraction device determines a group of feature data by using the skip detection manner to detect each group of sub-data, reduces the time consumption in actual detection, and improves the feature extraction efficiency.

[0074] In some embodiments, Figure 1 The S102 shown can be implemented by S106, which will be described in combination with the steps.

[0075] S106, detects whether the 1st sub-data and the Nth sub-data in each group of sub-data are valid data in a skip detection manner.

[0076] In the embodiment of the application, the feature extraction device detects whether the 1st sub-data and the Nth sub-data in each group of sub-data are valid data in a skip detection manner. Each group of sub-data can include N sub-data.

[0077] In the embodiment of the application, the feature extraction device can detect the 1st sub-data and the Nth sub-data in each group of sub-data by using a preset program, determine whether the Nth sub-data is a valid character, and obtain a detection result of the 1st sub-data and the Nth sub-data.

[0078] For example, each group of sub-data includes 5 sub-data. The feature extraction device detects whether the 1st sub-data and the 5th sub-data in each group of sub-data are valid data in a skip detection manner.

[0079] In the embodiment of the present application, the feature extraction device can also detect whether the Kth sub-data and the Tth sub-data in each group of sub-data are valid data by means of the skip detection. K is an integer greater than or equal to 1 and less than T. T is an integer greater than or equal to 1 and less than or equal to N.

[0080] For example, each group of sub-data includes 5 sub-data. The feature extraction device detects whether the 2nd sub-data and the 3rd sub-data in each group of sub-data are valid data by means of the skip detection.

[0081] In the embodiment of the present application, the feature extraction device detects other sub-data only when the 1st sub-data and the Nth sub-data are valid characters. The feature extraction device determines a group of feature data by means of the skip detection on each group of sub-data, reduces the time consumption in actual detection, and further improves the feature extraction efficiency.

[0082] In some embodiments, Figure 1 The S102 shown can be implemented by S107, which will be described in combination with each step.

[0083] S107, detecting whether the Tth sub-data in each group of sub-data is valid data by means of the skip detection.

[0084] In the embodiment of the present application, the feature extraction device detects whether the Tth sub-data in each group of sub-data is valid data by means of the skip detection.

[0085] For example, each group of sub-data includes 5 sub-data. The feature extraction device detects whether the 4th sub-data in each group of sub-data is valid data by means of the skip detection.

[0086] In the embodiment of the present application, the feature extraction device detects other sub-data only when the Tth sub-data is a valid character. The feature extraction device determines a group of feature data by means of the skip detection on each group of sub-data, reduces the time consumption in actual detection, and further improves the feature extraction efficiency.

[0087] In some embodiments, Figure 1 The S102 shown can further include S108, which will be described in combination with each step.

[0088] S108, if the predetermined sub-data in the Mth group of sub-data is not valid data, detecting whether the predetermined sub-data in the M+1th group of sub-data is valid data until the detection of the plurality of sub-data is completed, and obtaining a plurality of groups of feature data.

[0089] If the predetermined sub-data in the Mth group of sub-data is valid data, the feature extraction device detects whether the other sub-data in the Mth group of sub-data is valid data in sequence.

[0090] In the embodiment of the present application, the Mth group of sub-data includes N sub-data, and the Nth sub-data is the predetermined sub-data. When the feature extraction device detects that the Nth sub-data is not a letter, the feature extraction device detects whether the predetermined sub-data in the M+1th group of sub-data is a letter. Until the detection of the multiple groups of sub-data is completed, the feature extraction device obtains multiple groups of feature data.

[0091] In some embodiments, Figure 1 The S103 shown can also be implemented by S109, which will be described in combination with the steps.

[0092] In the S109, if the predetermined sub-data in the Mth group of sub-data is valid data, the feature extraction device detects whether the other sub-data in the Mth group of sub-data is valid data in sequence.

[0093] In the embodiment of the present application, if the predetermined sub-data in the Mth group of sub-data is valid data, the feature extraction device detects whether the other sub-data in the Mth group of sub-data is valid data in sequence.

[0094] In the embodiment of the present application, if the feature extraction device detects that all the other sub-data are valid data, the feature extraction device determines that the Mth group of sub-data is a group of feature data.

[0095] In the embodiment of the present application, if the detection result indicates that all the other sub-data in the Mth group of sub-data are English letters, the feature extraction device determines that the Mth group of sub-data is a group of feature data.

[0096] In the embodiment of the present application, if the feature extraction device detects that there is non-valid data in the other sub-data, the feature extraction device starts the detection of the M+1th group of sub-data.

[0097] In the embodiment of the present application, the Mth group of sub-data includes N sub-data. When the predetermined sub-data is the Nth sub-data, the feature extraction device can detect the other sub-data in the Mth group of sub-data in the order of the 1st sub-data to the N-1th sub-data. When the predetermined sub-data is the 1st sub-data and the Nth sub-data, the feature extraction device can detect the other sub-data in the Mth group of sub-data in the order of the 2nd sub-data to the N-1th sub-data. When the predetermined sub-data is the Tth sub-data, the feature extraction device can detect the other sub-data in the Mth group of sub-data in the order of the 1st sub-data to the T-1th sub-data, the T+1th sub-data to the Nth sub-data.

[0098] In the embodiment of the present application, when all the other sub-data of a group of sub-data are valid characters, a group of feature data can be determined, and then the determination and detection of the next group of sub-data are performed. Since the detection of other sub-data is performed after each group of sub-data is determined, the detection time is saved, and the feature extraction efficiency is improved.

[0099] In some embodiments, Figure 1 The S103 shown can be followed by S110, which will be described in combination with the steps.

[0100] S110, if there is non-valid data in the other sub-data, whether the predetermined sub-data in the M+1th group of sub-data is valid data is detected until the detection of the multiple sub-data is completed, and multiple groups of feature data are obtained.

[0101] In the embodiment of the present application, if there is non-valid data in the other sub-data, the feature extraction device detects whether the predetermined sub-data in the M+1th group of sub-data is valid data until the detection of the multiple sub-data is completed, and multiple groups of feature data are obtained.

[0102] In the embodiment of the present application, the predetermined sub-data in the M+1th group of sub-data can be the Nth sub-data, or the 1st sub-data and the Nth sub-data, or the Tth sub-data.

[0103] If there is non-valid data in the other sub-data, it can include that there is non-alphabet character in the other sub-data.

[0104] In some embodiments, Figure 1 The S104 shown can be followed by S111, which will be described in combination with the steps.

[0105] S111, the multiple groups of feature data are respectively detected to determine the target detection result of the to-be-detected data.

[0106] In the embodiment of the present application, the feature extraction device respectively detects the multiple groups of feature data to determine the target detection result of the to-be-detected data.

[0107] In some embodiments, the S111 shown can be implemented through S112 to S114, which will be described in combination with the steps.

[0108] S112, the multiple groups of feature data are respectively identified as black and white to obtain the identification result of each group of feature data, and then multiple identification results are obtained.

[0109] In the embodiment of the present application, the feature extraction device respectively identifies the multiple groups of feature data as black and white to obtain the identification result of each group of feature data, and then multiple identification results are obtained.

[0110] In the embodiment of the present application, the feature extraction device uses the identification program to perform black and white identification on each group of feature data. The identification result corresponding to each group of feature data is obtained, and then a plurality of identification results are obtained.

[0111] The plurality of identification results can include a first result and a second result. The first result represents that the corresponding group of feature data is abnormal, and the second result represents that the corresponding group of feature data is normal.

[0112] S113, calculating a proportion of the first result in the plurality of identification results.

[0113] In the embodiment of the present application, the feature extraction device calculates the proportion of the first result in the plurality of identification results.

[0114] In the embodiment of the present application, the feature extraction device divides the number of the first result in the plurality of identification results by the data of the plurality of identification results to obtain the proportion.

[0115] S114, if the proportion is greater than a predetermined threshold, determining that the target detection result of the to-be-tested data is abnormal data.

[0116] In the embodiment of the present application, if the proportion is greater than a predetermined threshold, it is determined that the target detection result of the to-be-tested data is abnormal data.

[0117] In the embodiment of the present application, if the proportion is not greater than a predetermined threshold, it is determined that the target detection result of the to-be-tested data is normal data.

[0118] In the embodiment of the present application, the feature extraction device determines the target detection result of the to-be-tested data through the proportion of the first result in the plurality of identification results, and improves the accuracy of feature extraction while detecting the sub-data in a leapfrog manner.

[0119] In some embodiments, referring to Figure 2 , Figure 2 An optional flowchart of the feature extraction method provided by the embodiment of the present application will be described in combination with each step.

[0120] S201, file reading.

[0121] The feature extraction device performs file traversal and reading on the disk file.

[0122] S202, judging whether the fifth byte is a valid character.

[0123] The feature extraction device can divide the to-be-tested data extracted from the disk file into a plurality of groups of sub-data, and the feature extraction device detects whether the fifth byte in each group of sub-data is a valid character.

[0124] S203, reading the sixth byte.

[0125] When the 5th character in a group of sub-data is an invalid character, the feature reads the 6th byte and performs the next group of sub-data character loop matching logic.

[0126] S204, read the 1st byte.

[0127] When the 5th byte in a group of sub-data is a valid character, the feature extraction device continues to read the 1st byte in the group of sub-data and performs normal string matching logic, and ends when an invalid string code is encountered.

[0128] S205, normal string matching logic.

[0129] The feature extraction device continues to match according to the above two logics, and directly skips for characters that do not meet the conditions until the file matching ends. This greatly improves the scanning time of malicious file detection and provides the advantage of lightweight malicious file detection.

[0130] See Figure 3 , Figure 3 The structural diagram of the feature extraction device provided by the embodiment of the application.

[0131] The embodiment of the application further provides a feature extraction device 800, which comprises a data acquisition unit 801, a detection unit 802 and a determination unit 803.

[0132] The data acquisition unit 801 is used for acquiring to-be-detected data, and dividing a plurality of sub-data in the to-be-detected data into a plurality of groups of sub-data.

[0133] The detection unit 802 is used for detecting whether predetermined sub-data in each group of sub-data is valid data in a jump detection manner.

[0134] The detection unit 802 is used for detecting whether other sub-data in the Mth group of sub-data is valid data if the predetermined sub-data in the Mth group of sub-data is valid data; the other sub-data is sub-data in the Mth group of sub-data except the predetermined sub-data; M is an integer greater than or equal to 1.

[0135] The determination unit 803 is used for determining that the Mth group of sub-data is a group of feature data when the other sub-data are all valid data, and obtaining a plurality of groups of feature data until the detection of the plurality of sub-data is completed.

[0136] In the embodiment of the application, each group of sub-data comprises N sub-data; N is an integer greater than 1.

[0137] The detection unit 802 in the feature extraction device 800 is configured to detect whether the Nth sub-data in each group of sub-data is valid data by means of skip detection; the detection unit 802 in the feature extraction device 800 is configured to detect whether the 1st sub-data and the Nth sub-data in each group of sub-data are valid data by means of skip detection; the detection unit 802 in the feature extraction device 800 is configured to detect whether the Tth sub-data in each group of sub-data is valid data by means of skip detection; T is an integer greater than or equal to 1 and less than or equal to N.

[0138] In the embodiment of the present application, the detection unit 802 in the feature extraction device 800 is configured to detect whether the predetermined sub-data in the M+1th group of sub-data is valid data if the predetermined sub-data in the Mth group of sub-data is not valid data, until the detection of the plurality of sub-data is completed, and the plurality of groups of feature data are obtained.

[0139] In the embodiment of the present application, the detection unit 802 in the feature extraction device 800 is configured to sequentially detect whether each of the other sub-data in the Mth group of sub-data is valid data according to the order of the other sub-data.

[0140] In the embodiment of the present application, the detection unit 802 in the feature extraction device 800 is configured to detect whether the predetermined sub-data in the M+1th group of sub-data is valid data if there is non-valid data in the other sub-data, until the detection of the plurality of sub-data is completed, and the plurality of groups of feature data are obtained.

[0141] In the embodiment of the present application, the detection unit 802 in the feature extraction device 800 is configured to detect the plurality of groups of feature data respectively to determine the target detection result of the to-be-detected data.

[0142] In the embodiment of the present application, the detection unit 802 in the feature extraction device 800 is configured to respectively identify the plurality of groups of feature data to obtain an identification result of each group of feature data, and further obtain a plurality of identification results.

[0143] The proportion of the first result in the plurality of identification results is calculated.

[0144] If the proportion is greater than a predetermined threshold, the target detection result of the to-be-detected data is determined to be abnormal data.

[0145] In this embodiment of the invention, the data acquisition unit 801 acquires the data to be tested and divides multiple sub-data into multiple groups of sub-data. The detection unit 802 detects whether a predetermined sub-data in each group of sub-data is valid data using a skip detection method. If the detection unit 802 detects that the predetermined sub-data in the Mth group of sub-data is valid data, it then detects whether the other sub-data in the Mth group of sub-data is valid data. The other sub-data are the sub-data in the Mth group of sub-data excluding the predetermined sub-data; M is an integer greater than or equal to 1. When all other sub-data are valid data, the determination unit 803 determines the Mth group of sub-data as a set of feature data, until the detection of multiple sub-data is completed, resulting in multiple sets of feature data. Because this scheme skips and detects non-adjacent sub-data in each group of sub-data during the process of determining a set of feature data, without traversing all sub-data, it saves detection time and effectively improves the efficiency of data feature extraction.

[0146] It should be noted that, in the embodiments of the present invention, if the above-described feature extraction method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a feature extraction device (which may be a personal computer, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk. Thus, the embodiments of the present invention are not limited to any specific hardware and software combination.

[0147] Correspondingly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0148] Correspondingly, this embodiment of the invention provides a feature extraction device, including a memory 902 and a processor 901. The memory 902 stores a computer program that can run on the processor 901. When the processor 901 executes the program, it implements the steps in the above method.

[0149] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the descriptions of the method embodiments of the present invention for understanding.

[0150] It should be noted that,Figure 4 A hardware entity diagram of the feature extraction device provided by the embodiment of the present application is shown in FIG. 9. The hardware entity of the feature extraction device 900 includes a processor 901 and a memory 902, wherein: Figure 4

[0151] The processor 901 generally controls the overall operation of the feature extraction device 900.

[0152] The memory 902 is configured to store instructions and applications executable by the processor 901, and can also cache data (e.g., image data, audio data, voice communication data, and video communication data) to be processed by the processor 901 and modules in the feature extraction device 900, which can be implemented by FLASH or Random Access Memory (RAM).

[0153] It should be understood that the term "one embodiment" or "an embodiment" as used throughout this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Therefore, the appearance of the phrase "in one embodiment" or "in an embodiment" in various places throughout the specification is not necessarily referring to the same embodiment. In addition, these particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that the size of the sequence of the above processes does not mean the execution order in various embodiments of the present application, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above sequence number of the embodiments of the present application is only for description, not representing the advantages and disadvantages of the embodiments.

[0154] It should be noted that in this document, the terms "comprise", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article, or device including the element.

[0155] ​In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. The apparatus embodiments described above are merely illustrative, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed components can be indirect coupling or communication connection through some interfaces, apparatuses or units, which can be electrical, mechanical or other forms.

[0156] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units; they can be located in one place or distributed on a plurality of network units; and part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0157] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or hardware plus software functional unit.

[0158] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps including the above-mentioned method embodiments when executed; and the foregoing storage medium includes mobile storage devices, read only memory (Read Only Memory, ROM), magnetic discs or optical discs and various storage program codes.

[0159] Alternatively, the integrated unit of the present application, if implemented in the form of a software function module and sold or used as an independent product, can also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes mobile storage devices, ROM, magnetic discs or optical discs and various storage program codes.

[0160] The above merely describes the embodiments of the present application, and the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A feature extraction method, characterized in that, include: Acquire the test data and divide the multiple sub-data in the test data into multiple groups of sub-data; The method of skip detection is used to detect whether the predetermined sub-data in each group of sub-data is valid; wherein, the predetermined sub-data is any one or two non-contiguous sub-data in the corresponding group of sub-data; If the predetermined sub-data in the Mth group of sub-data is valid data, then it is checked whether the other sub-data in the Mth group of sub-data is valid data; the other sub-data is the sub-data in the Mth group of sub-data excluding the predetermined sub-data; if the predetermined sub-data in the Mth group of sub-data is not valid data, then it is checked whether the predetermined sub-data in the (M+1)th group of sub-data is valid data, until the detection of the multiple sub-data is completed, and multiple sets of feature data are obtained; if there is invalid data in the other sub-data, then it is checked whether the predetermined sub-data in the (M+1)th group of sub-data is valid data, until the detection of the multiple sub-data is completed, and multiple sets of feature data are obtained; M is an integer greater than or equal to 1; When all other sub-data are valid, the Mth group of sub-data is determined to be a set of feature data, until the detection of multiple sub-data is completed, resulting in multiple sets of feature data.

2. The feature extraction method according to claim 1, characterized in that, Each set of sub-data includes: N sub-data; N is an integer greater than 1; The method of detecting whether a predetermined sub-data in each group of sub-data is valid by skipping detection includes one of the following: The method of skipping detection is used to determine whether the Nth sub-data in each group of sub-data is valid data; The first and Nth sub-data points in each group of sub-data points are detected as valid data using a skip detection method. The method of skipping detection is used to detect whether the Tth sub-data in each group of sub-data is valid data; T is an integer greater than or equal to 1 and less than or equal to N.

3. The feature extraction method according to claim 1, characterized in that, The step of detecting whether other sub-data in the Mth group of sub-data are valid data includes: In accordance with the order of the other sub-data in the Mth group of sub-data, each other sub-data is sequentially checked to see if it is valid data.

4. The feature extraction method according to claim 1, characterized in that, When all other sub-data are valid data, the Mth group of sub-data is determined as a set of feature data. This process continues until the detection of multiple sub-data is completed and multiple sets of feature data are obtained. The method further includes: The multiple sets of feature data are detected separately to determine the target detection result of the data to be tested.

5. The feature extraction method according to claim 4, characterized in that, The step of detecting the multiple sets of feature data respectively to determine the target detection result of the data to be tested includes: The multiple sets of feature data are subjected to black-and-white identification respectively to obtain the identification result of each set of feature data, thereby obtaining multiple identification results; Calculate the percentage of the first result among the multiple identification results; If the percentage is greater than a predetermined threshold, then the target detection result is determined to be abnormal data.

6. A feature extraction device, characterized in that, include: A data acquisition unit is used to acquire the data to be tested and to divide multiple sub-data in the data to be tested into multiple groups of sub-data. The detection unit is used to detect whether a predetermined sub-data in each group of sub-data is valid data by means of skip detection; wherein, the predetermined sub-data is any one or two non-contiguous sub-data in the corresponding group of sub-data; A detection unit is configured to: if the predetermined sub-data in the Mth group of sub-data is valid data, then detect whether other sub-data in the Mth group of sub-data is valid data; the other sub-data refers to sub-data in the Mth group of sub-data excluding the predetermined sub-data; if the predetermined sub-data in the Mth group of sub-data is not valid data, then detect whether the predetermined sub-data in the (M+1)th group of sub-data is valid data, until the detection of the multiple sub-data is completed, resulting in multiple sets of feature data; if there is invalid data among the other sub-data, then detect whether the predetermined sub-data in the (M+1)th group of sub-data is valid data, until the detection of the multiple sub-data is completed, resulting in the multiple sets of feature data; M is an integer greater than or equal to 1. The determining unit is used to determine the Mth group of sub-data as a set of feature data when all other sub-data are valid data, until the detection of multiple sub-data is completed, and multiple sets of feature data are obtained.

7. A feature extraction device, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, the processor executing the program to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 5.

Images