A security assessment method and system for malicious detectors of the Internet of Things

Abstract

The invention discloses a security assessment method and system for an Internet of Things malicious detector, comprising: obtaining malicious samples and normal samples; constructing an alternative detector for the Internet of Things malicious detector; selecting a sample with the largest cosine distance from the malicious sample from the normal sample as a landmark point; searching for candidate adversarial samples from the malicious samples through evolutionary calculation, in the process of evolutionary calculation, adding disturbances to the malicious samples in each generation to generate offspring samples, calculating the cosine distance from the offspring samples to the landmark point, selecting offspring samples with cosine distances greater than a threshold value as candidate adversarial samples, and participating offspring samples with cosine distances not greater than the threshold value in the next round of evolutionary calculation; detecting the candidate adversarial samples through an alternative detector, using the detected samples as adversarial samples to evaluate the security of the Internet of Things malicious detector, and obtaining a security assessment result. The security of the Internet of Things malicious detector can be evaluated in a timely and effective manner.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a security assessment method and system for an Internet of Things malicious detector. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] As the number of malicious attacks on the Internet of Things increases, the malicious detection technology of the Internet of Things is constantly developing. At present, the malicious attacks on the Internet of Things are mainly detected by the Internet of Things malicious detector based on machine learning to prevent malicious attacks on the Internet of Things.

[0004] However, the security of IoT malicious detectors cannot be effectively evaluated, and the IoT malicious detectors cannot be improved based on accurate evaluation results.

[0005] The malicious detector based on streaming text is trained using features extracted from the HTTP header. Currently, malicious samples are mainly generated by modifying any characters in the URL, and then the security assessment of the IoT malicious detector based on streaming text is performed. However, the method of modifying any characters in the URL may render the original malicious functions in the URL invalid. When the security assessment of the IoT malicious detector based on streaming text is performed using this sample, the assessment accuracy is low.

[0006] If the security assessment methods in other fields are modified and then applied to the security assessment of IoT malicious detectors for streaming text, the time to generate adversarial samples will be more than one second, and some will even reach the minute level, and the network delay will be large.

[0007] Therefore, the inventors believe that existing methods cannot timely and effectively evaluate the security of IoT malicious detectors for streaming text. Summary of the invention

[0008] In order to solve the above problems, the present invention proposes a security assessment method and system for an Internet of Things malicious detector, which can perform timely and effective security assessment on the Internet of Things malicious detector.

[0009] To achieve the above object, the present invention adopts the following technical solution:

[0010] First, a security assessment method for IoT malicious detectors is proposed, including:

[0011] Obtain malicious samples and normal samples;

[0012] Building alternative detectors to IoT malicious detectors;

[0013] Select the sample with the largest cosine distance from the malicious sample from the normal sample as the landmark point;

[0014] Through evolutionary computing, candidate adversarial samples are searched from malicious samples. In the process of evolutionary computing, disturbances are added to malicious samples in each generation to generate offspring samples. The cosine distances of offspring samples to landmarks are calculated. Offspring samples with cosine distances greater than a threshold are selected as candidate adversarial samples, and offspring samples with cosine distances less than the threshold are selected to participate in the next round of evolutionary computing.

[0015] Detect candidate adversarial samples through alternative detectors, and use samples that pass the detection as adversarial samples;

[0016] Adversarial samples are used to evaluate the security of IoT malicious detectors and obtain security assessment results.

[0017] Furthermore, the process of obtaining normal samples and malicious samples is as follows:

[0018] Get the original data set;

[0019] Extract feature data from the original data set;

[0020] The IoT malicious detector is used to identify feature data to obtain normal samples and malicious samples.

[0021] Furthermore, the process of constructing an alternative detector is:

[0022] Building alternative models for IoT malicious detectors;

[0023] The constructed substitution model is trained through normal samples and malicious samples, and the trained model is the substitution detector.

[0024] Furthermore, the specific process of selecting landmark points from normal samples is as follows:

[0025] Cluster normal samples into N clusters. For each cluster, calculate the cosine distance from the malicious sample to all samples in the cluster and determine the marking point of the cluster.

[0026] The landmark point with the largest cosine distance from the malicious sample is selected as the landmark point of the normal sample.

[0027] Furthermore, the threshold value is determined as follows:

[0028] Determine the cluster where the landmark point is located;

[0029] Select the first sample e1 and the T*size(C)th sample eT in the cluster where the landmark point is located;

[0030] The threshold value is determined according to the cosine distance between the malicious sample and sample e1 and sample eT.

[0031] Furthermore, disturbances are added to the malicious sample by appending words to the URL, and characters representing URL parameters are set between the words and the URL.

[0032] Furthermore, the process of obtaining the safety assessment results is as follows:

[0033] Detect adversarial samples through IoT malicious detectors and obtain samples that are successfully detected;

[0034] Calculate the percentage of samples detected as cost to the number of adversarial samples;

[0035] The security assessment result of the IoT malicious detector is obtained based on the percentage calculation.

[0036] Secondly, a security assessment system for IoT malicious detectors is proposed, including:

[0037] Sample acquisition module, used to obtain malicious samples and normal samples;

[0038] Alternative detector building blocks for building alternative detectors for IoT malicious detectors;

[0039] A landmark point acquisition module is used to select the sample with the largest cosine distance from the malicious sample from the normal sample as the landmark point;

[0040] The candidate adversarial sample acquisition module is used to search for candidate adversarial samples from malicious samples through evolutionary computing. In the process of evolutionary computing, disturbances are added to malicious samples in each generation to generate offspring samples, and the cosine distances from offspring samples to landmarks are calculated. Offspring samples with cosine distances greater than a threshold are selected as candidate adversarial samples, and offspring samples with cosine distances less than a threshold are selected to participate in the next round of evolutionary computing.

[0041] An adversarial sample acquisition module is used to detect candidate adversarial samples through alternative detectors and use samples that pass the detection as adversarial samples;

[0042] The IoT malicious detector security assessment module is used to evaluate the security of the IoT malicious detector using adversarial samples and obtain security assessment results.

[0043] In a third aspect, an electronic device is proposed, comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps described in a method for security assessment of an Internet of Things malicious detector are completed.

[0044] In a fourth aspect, a computer-readable storage medium is proposed for storing computer instructions. When the computer instructions are executed by a processor, the steps described in a security assessment method for an Internet of Things malicious detector are completed.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. The present invention constructs an alternative detector for the Internet of Things malicious detector, selects candidate adversarial samples from malicious samples by simplifying calculations, and then screens the candidate adversarial samples again through the alternative detector to obtain adversarial samples. The security of the Internet of Things malicious detector is tested through the adversarial samples, thereby ensuring the accuracy of the security assessment of the Internet of Things malicious detector.

[0047] 2. The present invention generates adversarial samples by adding character-level disturbances instead of modifying characters, which avoids destroying the original malicious functions when generating adversarial samples, thereby ensuring the accuracy and effectiveness of the security assessment of the IoT malicious detector.

[0048] 3. The present invention uses evolutionary computation guided by clustering and cosine distance to search for candidate adversarial samples, which speeds up the search time and shortens the delay during detection.

[0049] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0051] Figure 1 The overall flow chart of the method disclosed in Example 1;

[0052] Figure 2 A flow chart for obtaining an alternative detector for Example 1;

[0053] Figure 3 The deformation map of malicious samples towards normal sample landmarks;

[0054] Figure 4 This is a diagram of the process of selecting landmark points and calculating threshold values ​​in Example 1;

[0055] Figure 5 This is a process diagram for obtaining candidate adversarial samples through evolutionary computing in Example 1. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0057] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0058] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0059] Terminology explanation:

[0060] Adversarial attacks refer to attacks against machine learning models. Attackers bypass the detection of machine learning models by adding perturbations to malicious samples.

[0061] Example 1

[0062] In this embodiment, a security assessment method for an IoT malicious detector is disclosed, such as Figure 1 As shown, including:

[0063] S1: Obtain malicious and normal samples and build an alternative detector for IoT malicious detector.

[0064] Specifically: obtain the original data set, which is data obtained with permission and through legal channels;

[0065] Extract feature data from the original data set. The feature extraction here is completely carried out in the same way as the feature extraction of the IoT malicious detector;

[0066] The characteristic data is identified through the IoT malicious detector, and normal or malicious labels are marked on the characteristic data to obtain normal samples and malicious samples.

[0067] The constructed alternative model of the IoT malicious detector is trained using labeled normal samples and malicious samples. The trained model is the alternative detector of the IoT malicious detector, such as Figure 2 shown.

[0068] The alternative model adopts the model used by the IoT malicious detector, which can be any machine learning or deep learning model, such as an SVM model.

[0069] S2: Select the sample with the largest cosine distance from the malicious sample from the normal samples as the landmark point. The process includes: clustering the normal samples into N clusters. For each cluster, calculate the cosine distance from the malicious sample to all samples in the cluster, and determine the landmark point of the cluster.

[0070] The landmark point with the largest cosine distance from the malicious sample is selected as the landmark point of the normal sample.

[0071] In the specific implementation, the purpose of selecting landmark points from normal samples is to make the malicious samples approach the landmark points after adding disturbances to the malicious samples, such as Figure 3 shown.

[0072] The normal samples are clustered into N clusters by the kmeans clustering method. Each cluster is represented by Ci = {e1, e2, e3, ..., en}, where ei represents the samples in the cluster.

[0073] For each cluster, calculate the cosine distance from the malicious sample to be deformed to all samples in the cluster, and sort the normal samples in the cluster from large to small according to the value of the cosine distance, and select the R*size(Ci)th value as the landmark point Fi, where R is a hyperparameter, set according to specific needs, R∈(0,1), and size() represents the number. Keep the landmark point with the largest cosine distance from the malicious sample and its cluster.

[0074] Determine the cluster where the landmark point is located, select the first sample e1 and the T*size(C)th sample eT from the cluster where the landmark point is located, T is a hyperparameter, set according to specific needs, T∈(0,1), C is the cluster where the landmark point is located, and determine the threshold value Threshold by the cosine distance from the malicious sample to the e1 and eT samples, as follows: Figure 4 As shown, specifically:

[0075] Threshold=1-|cosdist(malware,e1)-cosdist(malware,eT))|,

[0076] Among them, cosdist refers to the cosine distance, cosdist(A,B) represents the cosine distance between two samples, and malware refers to malicious samples.

[0077] S3: Search for candidate adversarial samples from malicious samples through evolutionary computation. In the process of evolutionary computation, each generation will add perturbations to the malicious samples to generate offspring samples, calculate the cosine distance from the offspring samples to the landmarks, select the offspring samples with cosine distance greater than the threshold as candidate adversarial samples, and select the offspring samples with cosine distance less than the threshold to participate in the next round of evolutionary computation, such as Figure 5 shown.

[0078] The malicious sample is perturbated by appending words to the URL, and characters representing URL parameters are set between the words and the URL.

[0079] In order to prevent the evolutionary calculation from going on endlessly, only the offspring samples whose cosine distance is not greater than the threshold value but greater than the cosine distance from their parent sample to the landmark point will participate in the next round of evolutionary calculation, and the offspring samples whose cosine distance is not greater than the cosine distance from their parent sample to the landmark point will be eliminated. The termination conditions of the evolutionary calculation are set as follows: all candidate adversarial samples are found, and the evolutionary calculation is terminated; or, in a certain generation, all samples are eliminated, and the evolutionary calculation is terminated; or, a maximum generation limit is set, and the evolutionary calculation is terminated when the maximum generation limit is exceeded.

[0080] S4: Detect candidate adversarial samples through alternative detectors, and use samples that pass the detection as adversarial samples.

[0081] Specifically, the candidate adversarial samples are sent to the alternative detector to attack the alternative detector, and only the samples that pass the detection of the alternative detector are regarded as adversarial samples.

[0082] S5: Use adversarial samples to evaluate the security of IoT malicious detectors and obtain security evaluation results, including:

[0083] The adversarial samples are detected by the IoT malicious detector to obtain samples that are successfully detected, that is, the IoT malicious detector is attacked by the adversarial samples, and the samples detected by the IoT malicious detector are samples that are successfully detected;

[0084] Calculate the percentage of samples detected as successful to the number of adversarial samples sucrate, and obtain the security evaluation result Scrore of the IoT malicious detector based on sucrate, where:

[0085] Scrore=(1-sucrate)*100.

[0086] The security assessment method disclosed in this embodiment constructs an alternative detector for the Internet of Things malicious detector, selects candidate adversarial samples from malicious samples by simplifying calculations, and then screens the candidate adversarial samples again by the alternative detector to obtain adversarial samples. The security of the Internet of Things malicious detector is tested by the adversarial samples, thereby ensuring the accuracy of the security assessment of the Internet of Things malicious detector; the adversarial samples are generated by appending character-level disturbances instead of modifying characters, thereby avoiding destroying the original malicious functions when generating adversarial samples, thereby further ensuring the accuracy and effectiveness of the security assessment of the Internet of Things malicious detector; and the evolutionary calculation guided by clustering and cosine distance is used to search for candidate adversarial samples, thereby speeding up the search time and shortening the delay during detection.

[0087] Example 2

[0088] In this embodiment, a security assessment system for an IoT malicious detector is disclosed, including:

[0089] Sample acquisition module, used to obtain malicious samples and normal samples;

[0090] Alternative detector building blocks for building alternative detectors for IoT malicious detectors;

[0091] A landmark point acquisition module is used to select the sample with the largest cosine distance from the malicious sample from the normal sample as the landmark point;

[0092] The candidate adversarial sample acquisition module is used to search for candidate adversarial samples from malicious samples through evolutionary computing. In the process of evolutionary computing, disturbances are added to malicious samples in each generation to generate offspring samples, and the cosine distances from offspring samples to landmarks are calculated. Offspring samples with cosine distances greater than a threshold are selected as candidate adversarial samples, and offspring samples with cosine distances less than a threshold are selected to participate in the next round of evolutionary computing.

[0093] An adversarial sample acquisition module is used to detect candidate adversarial samples through alternative detectors and use samples that pass the detection as adversarial samples;

[0094] The IoT malicious detector security assessment module is used to evaluate the security of the IoT malicious detector using adversarial samples and obtain security assessment results.

[0095] Example 3

[0096] In this embodiment, an electronic device is disclosed, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, the steps described in a method disclosed in Embodiment 1 are completed.

[0097] Example 4

[0098] In this embodiment, a computer-readable storage medium is disclosed for storing computer instructions. When the computer instructions are executed by a processor, the steps described in a method disclosed in Embodiment 1 are completed.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A security assessment method for an Internet of Things malicious detector, characterized in that: include: Obtain malicious samples and normal samples; Building alternative detectors to IoT malicious detectors; Select the sample with the largest cosine distance from the malicious sample from the normal sample as the landmark point; Through evolutionary computing, candidate adversarial samples are searched from malicious samples. In the process of evolutionary computing, disturbances are added to malicious samples in each generation to generate offspring samples. The cosine distances of offspring samples to landmarks are calculated. Offspring samples with cosine distances greater than a threshold are selected as candidate adversarial samples, and offspring samples with cosine distances less than the threshold are selected to participate in the next round of evolutionary computing. Detect candidate adversarial samples through alternative detectors, and use samples that pass the detection as adversarial samples; Adversarial samples are used to evaluate the security of IoT malicious detectors and obtain security assessment results.

2. The security assessment method of an Internet of Things malicious detector as claimed in claim 1, characterized in that: The process of obtaining normal samples and malicious samples is as follows: Get the original data set; Extract feature data from the original data set; The IoT malicious detector is used to identify feature data to obtain normal samples and malicious samples.

3. According to the security assessment method of the Internet of Things malicious detector as claimed in claim 1, the process of constructing the alternative detector is: Building alternative models for IoT malicious detectors; The constructed substitution model is trained through normal samples and malicious samples, and the trained model is the substitution detector.

4. In the security assessment method of an IoT malicious detector as claimed in claim 1, the specific process of selecting landmark points from normal samples is as follows: Cluster normal samples into N clusters. For each cluster, calculate the cosine distance from the malicious sample to all samples in the cluster and determine the landmark of the cluster. The landmark point with the largest cosine distance from the malicious sample is selected as the landmark point of the normal sample.

5. According to the security assessment method of the Internet of Things malicious detector as claimed in claim 4, the threshold value is determined by: Determine the cluster where the landmark point is located; Select the first sample e1 and the T*size(C)th sample eT in the cluster where the landmark point is located, where: T is a hyperparameter, size() indicates the number, and C is the cluster where the landmark point is located; The threshold value is determined according to the cosine distance between the malicious sample and sample e1 and sample eT.

6. A security assessment method for an IoT malicious detector as described in claim 1, adding disturbances to malicious samples by appending words to the URL, and setting characters representing URL parameters between the words and the URL.

7. The security assessment method of an IoT malicious detector according to claim 1, wherein the process of obtaining the security assessment result is: Detect adversarial samples through IoT malicious detectors and obtain samples that are successfully detected; Calculate the percentage of samples detected as cost to the number of adversarial samples; The security assessment result of the IoT malicious detector is obtained based on the percentage calculation.

8. A security assessment system for an Internet of Things malicious detector, characterized in that: include: Sample acquisition module, used to obtain malicious samples and normal samples; Alternative detector building blocks for building alternative detectors for IoT malicious detectors; A landmark point acquisition module is used to select the sample with the largest cosine distance from the malicious sample from the normal sample as the landmark point; The candidate adversarial sample acquisition module is used to search for candidate adversarial samples from malicious samples through evolutionary computing. In the process of evolutionary computing, disturbances are added to malicious samples in each generation to generate offspring samples, and the cosine distances from offspring samples to landmarks are calculated. Offspring samples with cosine distances greater than a threshold are selected as candidate adversarial samples, and offspring samples with cosine distances less than a threshold are selected to participate in the next round of evolutionary computing. An adversarial sample acquisition module is used to detect candidate adversarial samples through alternative detectors and use samples that pass the detection as adversarial samples; The IoT malicious detector security assessment module is used to evaluate the security of the IoT malicious detector using adversarial samples and obtain security assessment results.

9. An electronic device, characterized in that: The invention comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps of the security assessment method of the Internet of Things malicious detector described in any one of claims 1 to 7 are completed.

10. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of a security assessment method for an Internet of Things malicious detector as described in any one of claims 1-7.

Images