A Real-time Abnormality Detection Method and Device for High-risk User Instructions

Abstract

The present invention provides a real-time anomaly detection method and device for high-risk user instructions. The method includes: collecting instruction data used by a user from a production environment, and obtaining system error logs and automated test results of the instruction data; calculating a first high-risk score of the instruction data according to instruction sub-parameters of the instruction data, system error logs, and automated test results; calculating a second high-risk score of the instruction data based on word vectors of the instruction data; and obtaining an anomaly detection result of the instruction data according to the first high-risk score and the second high-risk score. The present invention reduces the difficulty of data access in a streaming data processing system and improves the throughput of the entire data processing system. By means of various factors such as instruction data, the corresponding number of system error logs, and automated test results, the present invention reduces the operation and maintenance difficulty of traditional abnormal instruction detection and improves the detection speed and accuracy of abnormal instructions.

Description

Technical Field

[0001] The present invention belongs to the field of network security, and specifically proposes a real-time anomaly detection method and device for high-risk user instructions. Background Art

[0002] During operations and maintenance, to facilitate management, operators often gain super privileges on servers, enabling them to perform numerous complex operations. This behavior presents numerous risks, such as stopping running processes or accidentally deleting important server files and data. Such incidents are numerous. A 2014 cybersecurity survey released by the US CERT revealed that 28% of insider attacks resulted in 46% of losses.

[0003] To address the above issues, in 2011 the U.S. Department of Defense proposed a military internal threat detection system called ADAMS (Anomaly Detection at Multiple Scales). It put forward specific requirements from multiple aspects, including system architecture and detection implementation, and conducted operational tests on actual enterprise data, achieving good results.

[0004] As the harm caused by abnormal instructions continues to expand, research on abnormal instruction detection is becoming increasingly systematic, and real-time interception and control of abnormal instructions is becoming a serious problem. However, in the face of security center assurance and operation and maintenance systems, existing abnormal instruction datasets and abnormal instruction detection methods are of limited use, mainly due to the following limitations:

[0005] (1) Most of the existing instruction detection datasets are from before 2000. The data sources are too old and cannot meet the current evolving basic instruction types and number.

[0006] (2) Most existing instruction detection methods, whether based on traditional machine learning or deep learning, cannot meet the growing demand for instruction detection, the accuracy of instruction detection, or the industry's demand for real-time instruction detection.

[0007] (3) Currently, the industry lacks a machine learning-based intelligent abnormal instruction detection tool. Traditional, custom-defined detection systems are the only ones available, which are difficult to detect and cannot accurately detect special instructions or special script instructions. Furthermore, the diversity of business systems increases the practical application difficulty of traditional access control and other system security mechanisms. Existing internal threat detection systems are more experimental and lack practical versions.

[0008] (4) At present, in the academic and industrial circles, the definition of harmful instructions and the distinction of the harm levels of harmful instructions are also outstanding issues. The current datasets are only binary classification datasets, without classifying the harm levels of various instructions, resulting in inaccurate classification of harmful instructions, and the relationship between non-harmful instructions and harmful instructions has not been accurately defined.

[0009] Therefore, it is necessary to systematically analyze the internal threats of the central business system, extract the core features of internal threats, and thus design a new prototype system for detecting abnormal instructions, as Figure 1 shown. While ensuring accuracy, the speed of anomaly detection should also be considered to ensure real-time detection.

[0010] The solutions of the existing technologies are as follows:

[0011] 1. An instruction filtering model mainly based on custom rules. Such models mainly use operation and maintenance personnel or the system itself to add or delete various instruction running rules, such as which instruction names cannot be used, for example, rm, vim, etc. Such an instruction filtering model requires high operation from operation and maintenance personnel. It requires operation and maintenance personnel to use their experience to input the instructions that cannot be used or are dangerous into the system in advance for the system to filter, so as to improve the accuracy of the system.

[0012] 2. An instruction filtering model mainly based on a simulated sandbox. Such models focus on creating an instruction sandbox system that is consistent with the environment of the external system. By inputting each instruction entered by the user into this sandbox system for simulated operation, it is checked whether this instruction causes information damage, information theft, or system crash in the sandbox system to judge the security level of the instruction. This model hands over the instruction scoring to the sandbox system for judgment, and such a system can judge almost all instructions.

[0013] 3. Machine learning-based text analysis models. Such models regard instructions as one or more lines of text and analyze such text to score and determine their risk levels. In 2003, Maxion proposed an abnormal instruction detection method based on Naive Bayes. Based on the SEA dataset, he analyzed the instructions of each user in the SEA data in sets of 100 instructions and obtained preliminary good results (TPR = 66.2%, FPR = 1.5%). Oka et al. used ECM to simulate the instruction usage order of a normal user, then simulated the instruction usage order of a user who used abnormal instructions, and finally compared the two to determine whether the instruction block was an abnormal instruction block. In contrast, Kim and Cha proposed a more lightweight method by introducing the concept of common commands as features and training a support vector machine (SVM) and a voting engine as classifiers for abnormal instructions. Lane and Brodley used two methods to detect abnormal instructions: Hidden Markov Model (HMM) and instance-based learning. Huang and Stamp utilized the location information of user instructions and proposed a Profile HMM (PHMM) method for abnormal instruction detection. Kholidy et al. proposed a data-driven semi-global alignment (DDSGA) method. In the training phase, the sequence alignment parameters of a given user are calculated. In the detection phase, several misaligned sequences are found to indicate the abnormal instruction activities of the user. Schonlau et al. adopted the Bayesian single-step Markov chain method, and when detecting SEA-Full abnormal instructions, the TPR rate was 69.3% and the FPR rate was 6.7%.

[0014] 4. Deep learning-based text analysis models. Such models use deep learning methods to detect abnormal instructions by deeply analyzing and mining a block of instructions (100 lines of instructions). Elmasry et al. conducted a large number of experiments on various DL models, including deep neural networks (DNNs), CNNs, and long short-term memory (LSTM). Using CNN as a text classification model, the camouflage detection problem was regarded as a text classification task. Meng et al. proposed a method based on kernel PCA and LSTM-RNN, integrating data preprocessing, feature extraction, and attack detection into an end-to-end detection system. Abdulhammed et al. used variational autoencoders and principal component analysis for feature dimensionality reduction. Subsequently, using these features, they classified by investigating various models, such as RF, Bayesian networks, linear discriminant analysis, and quadratic discriminant analysis. Kim et al. applied the LSTM architecture and trained the model using the KDD Cup 1999 dataset. They empirically verified that the DL method is effective for IDS. Yin et al. used RNN and developed the RNN-IDS model, which outperformed other ML models in both binary and multi-class classification, including J48, artificial neural networks, random forest (RF), and SVM. For the NSL-KDD dataset, their system significantly outperformed several attack detection strategies using support vector machines, neural networks, and Bayesian methods. Le et al. explored a model containing multiple components, including RNN, LSTM, and gated recurrent units, and discussed which component gave the best results. Summary of the Invention

[0015] To reduce the operation and maintenance difficulty of traditional abnormal instruction detection and improve the detection speed and accuracy of abnormal instructions at the same time, the present invention proposes a real-time abnormal detection method and device for high-risk user instructions, which can be applied to the real-time instruction detection of various operating systems.

[0016] The technical content of the present invention includes:

[0017] A real-time abnormal detection method for high-risk user instructions, applicable to operation and maintenance users, and the steps of the method include:

[0018] Collect the instruction data used by a user from the production environment, and obtain the system error log and the automated test result of the instruction data; calculate the first high-risk score of the instruction data according to the instruction sub-parameters, system error log, and automated test result of the instruction data;

[0019] Calculate the second high-risk score of the instruction data based on the word vector of the instruction data;

[0020] Obtain the abnormal detection result of the instruction data according to the first high-risk score and the second high-risk score.

[0021] Further, collecting instruction data used by a user from the production environment includes:

[0022] Modifying the user instruction history rule based on the needs of the operation and maintenance user;

[0023] Collecting the instruction data used by the user in the production environment according to the modified user instruction history rule.

[0024] Further, after collecting the instruction data used by a user from the production environment, it further includes:

[0025] Cleaning the instruction data to obtain a credibility score for the instruction data;

[0026] Cleaning the instruction data with a credibility score lower than the threshold based on the credibility threshold.

[0027] Further, cleaning the instruction data to obtain a credibility score for the instruction data includes:

[0028] Obtaining a segmentation result of the instruction data, where the segmentation result includes: instruction name and instruction separator, the segmentation result further includes: instruction parameters, and the segmentation result further includes: instruction identifiers;

[0029] Setting the initial credibility score of the instruction data to full marks;

[0030] Based on the segmentation result, obtaining a first credibility score for the instruction data by checking whether the instruction data is an instruction marked in the operating system;

[0031] Based on the segmentation result, obtaining a second credibility score for the instruction data by judging whether the instruction data has a help identifier and the number of the help identifiers;

[0032] Based on the segmentation result, obtaining a third credibility score for the instruction data by judging whether the instruction data exists in the system path;

[0033] Based on the segmentation result, obtaining a fourth credibility score for the instruction data by judging whether the instruction data exists in the context environment where the user is currently located;

[0034] Deducting the first credibility score, the second credibility score, the third credibility score, and the fourth credibility score from the initial credibility score respectively to obtain the credibility score of the instruction data.

[0035] Further, obtaining the segmentation result of the instruction data includes:

[0036] Obtain the instruction data training set of the user;

[0037] For each instruction data sample in the training set, use spaces as delimiters to split the instruction data sample into several instruction sub-parameters;

[0038] Based on the annotation of the instruction sub-parameters, use a semi-supervised method based on sequence annotation to obtain a segmentation model;

[0039] Input the instruction data into the segmentation model to obtain the segmentation result.

[0040] Further, calculating the first high-risk score of the instruction data according to the instruction sub-parameters, system error logs, and automated test results of the instruction data includes:

[0041] Search for the instruction scores in the high-risk instruction library based on the instruction name and instruction sub-parameters of the instruction data to assign a high-risk score to the instruction name;

[0042] Assign a high-risk score to the system error log based on the number of system error logs;

[0043] Assign a high-risk score to the automated test result based on the automated test result;

[0044] Integrate the high-risk score of the instruction name, the high-risk score of the system error log, and the score of the automated test module result, and perform 3-sigma calculation on the normalized data to obtain the first high-risk score of the instruction data.

[0045] Further, calculating the second high-risk score of the instruction data based on the word vector of the instruction data includes:

[0046] Respectively obtain the instruction data training sets of the user and other users;

[0047] Insert the instruction data of other users into the instruction data training set of the user to construct an instruction data set;

[0048] Based on the word embedding method, convert each instruction data in the instruction data set into a sample word vector;

[0049] Input each of the sample word vectors into the TCN network, set the time step according to the instruction input time to analyze the correlation degree between the corresponding instruction data and the previous instruction data, and compare the output of the TCN network with the true score of the sample word vector, and backpropagate the error to optimize the TCN network to obtain an anomaly detection model, where the structure of the TCN network includes: N layers of residual neural network layers and M layers of Pooling layers, and N and M are natural numbers;

[0050] Input the word vector into the anomaly detection model to obtain the second high-risk score of the instruction data.

[0051] Further, obtaining the anomaly detection result of the instruction data according to the first high-risk score and the second high-risk score includes:

[0052] If the anomaly detection result corresponding to the first high-risk score exceeds the 3-sigma threshold, use the anomaly detection result corresponding to the first high-risk score as the anomaly detection result of the instruction data, and enter all instruction sub-parameters in the instruction data into the high-risk anomaly instruction library;

[0053] If the anomaly detection result corresponding to the first high-risk score indicates no danger, use the anomaly detection result corresponding to the second high-risk score as the anomaly detection result of the instruction data.

[0054] A storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when running.

[0055] An electronic device includes a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute any of the above methods.

[0056] Compared with the prior art, the method proposed by the present invention has the following advantages and effects:

[0057] 1. Analysis based on the number of error logs and tests: The traditional custom rule method can use the regular expression method to determine whether there are abnormal keywords in the instruction, but the simple regular expression method cannot analyze complex instructions and does not give specific scores. Attackers can also bypass the keywords and input abnormal instructions. The present invention is based on the number of error logs. By using the number of system error logs, the automated test pass result, and the scoring result of the instruction name in the high-risk anomaly instruction library after a period of time after the instruction is used, the instruction is scored, which can improve the correctness of abnormal instruction judgment compared with expert rules. At the same time, after scoring, the present invention will also store the instruction name with abnormal instructions and its score in the high-risk anomaly instruction library.

[0058] 2. Deep learning: Traditional machine learning methods analyze user instruction blocks to determine whether each user instruction block contains one or more abnormal instructions. The results obtained by such analysis are inaccurate and the speed is slow. The deep learning method of the present invention adopts a supervised learning method. The user abnormal instruction data obtained by the first method is mixed with the instruction data of other users to obtain the abnormal instruction set of the user with other user instructions, and input it into the model for learning. The present invention adopts a model based on the temporal convolutional network TCN to analyze the causal convolution relationship between instructions. The model can perform causal analysis on each instruction and obtain an instruction model for each user. For abnormal instructions and instructions from other users that may be mixed and inserted, the model can analyze and determine whether each instruction is an abnormal instruction, and give the degree of danger of the instruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 Schematic diagram of the user command anomaly detection system's location and function in the entire service system.

[0060] Figure 2 Framework diagram of the user instruction anomaly detection system.

[0061] Figure 3 Flowchart of the user instruction anomaly detection method based on temporal convolutional network. DETAILED DESCRIPTION

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only specific embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0063] The purpose of this invention is to quickly identify abnormal instructions while maximizing accuracy by using error logs, automated testing, and deep learning methods. This improves the speed of instruction identification and enables intelligent instruction analysis. This is mainly divided into the following two aspects:

[0064] 1) Error logs and automated testing: The method of the present invention is based on error logs and automated test results. It analyzes the degree of damage caused by the instruction to the system based on the correlation between the error logs and the failure of the automated test cases that appear in the system after the instruction is run, and evaluates the high risk level of the instruction.

[0065] 2) Deep learning: The deep learning method of the present invention adopts a supervised learning method. The user abnormal instruction data obtained by the first method is mixed with the instruction data of other users to obtain the abnormal instruction set of the present user with the instructions of other users, and it is input into the model for learning. The present invention adopts a model based on the Temporal Convolutional Network (TCN) to analyze the causal convolution relationship between instructions. The model can perform causal analysis on each instruction to obtain an instruction model for each user. For abnormal instructions and instructions of other users that may be mixed in, the model can analyze and determine whether each instruction is an abnormal instruction and give the risk level of the instruction.

[0066] The main implementation steps of the present invention are as Figure 2 shown, and the specific steps are as follows:

[0067] Step 1: Instruction collection.

[0068] Collect some instructions used by ordinary users from the production environment and upload them to the database. Among them, the specific steps of instruction collection are:

[0069] 1) First, by setting a script file, which has the function of modifying the rules of user instruction history records and can set various data to be recorded, such as the user instruction input time (accurate to seconds), user name, user IP address used, user host MAC address, etc.

[0070] 2) Run the script file to start collecting instruction data.

[0071] Step 2: Instruction cleaning.

[0072] Since the instruction data exported from the production environment often has noise, it is necessary to perform data cleaning work according to the actual situation. The present invention performs cleaning operations through different filters to ensure that the present invention will not analyze these invalid instructions to generate errors and noise.

[0073] The specific steps of instruction data cleaning are:

[0074] 1) Split the instruction. Split an instruction text into an instruction name, instruction parameters (if any), instruction identifiers (if any), and instruction connectors (such as pipe symbols |, connectors &&, etc.). The specific steps here are: 1) Use the traditional splitting method with spaces as the delimiter for splitting to divide the instruction into instruction sub-parameters. 2) Use a semi-supervised method based on sequence labeling. First, obtain the splitting data by manually marking the instruction name, parameters, identifiers, etc. of each user instruction, and then input it into the model for training and learning, and finally obtain the model. 3) Apply the model to the instruction annotation of all other instructions.

[0075] 2) Assign a credibility score to each instruction, starting with a full score.

[0076] 3) Check whether the instruction is an instruction flagged in the operating system. The instructions clearly marked in the operating system should be stored in the operating system manual, and it can be checked through the manual whether the instruction exists. If it does not exist in some system manuals, the credibility score of the instruction should be reduced according to the number of non - existent ones.

[0077] 4) Determine whether the instruction has help identifiers (--help, -h, etc.). If the instruction does not have some help identifiers, the credibility score should be reduced according to the number of non - existent ones.

[0078] 5) Determine whether the instruction exists in the system path. If the instruction cannot be found in the system path, the credibility score is reduced.

[0079] 6) Determine whether the instruction exists in the context where the user is currently located. If it does not exist, the credibility score is reduced.

[0080] 7) Summarize to obtain the credibility score. Instructions with a score lower than the credibility threshold will be cleaned and cannot be executed.

[0081] Step 3: Error Log and Automated Testing.

[0082] After the user inputs a high - risk instruction, if it has a long - term impact on the system, the impact will be reflected in the error log and the results of automated testing. The present invention can score the user instruction according to the system error log and the results of automated testing after a certain instruction is input. The specific scoring steps of the custom rules are as follows:

[0083] 1. Query whether each instruction sub - parameter in the user instruction can be found in the high - risk instruction scoring library. If it can be found, add the scores of each instruction sub - parameter in the high - risk instruction scoring library to the user instruction. If not, set the high - risk score of this step to 0;

[0084] 2. Query the error log of the system for a period of time after the user instruction is used. If there is an error log, use the correlation analysis method to calculate the correlation degree between the error log and the user instruction, and convert it into a high - risk score. If not, set the high - risk score of this step to 0;

[0085] 3. Query the most recent automated test failure cases after the user instruction is used, and use the correlation analysis method to calculate the correlation degree between the automated test failure cases and the user instruction, and then convert it into a high - risk score. If there is no automated test module in the system or there are no failure cases in the automated test results, set the high - risk score of this step to 0;

[0086] 4. Finally, sum up all the high-risk scores, normalize them to obtain the final high-risk score of this instruction, and add this score to the high-risk score instruction library.

[0087] Step 4: Deep learning.

[0088] Input the instruction into the deep learning model for judgment, and finally obtain the specific score of each instruction. The specific operation steps are as follows:

[0089] 1. Insert some instructions of other users (including normal instructions and high-risk instructions) into the historical instructions of this user, and assign a certain high-risk score to the instructions of other users to form an instruction dataset. If there is a high-risk score obtained in the previous step in the historical instructions of this user, then the high-risk instruction score of this user's instructions is the high-risk score of the previous step; otherwise, all are set to 0.

[0090] 2. Input the instruction dataset into the model to train the causal instruction model of each user. The specific steps for training the model are as follows:

[0091] a) First, convert the instruction into a word vector through the word embedding method. The word embedding module can be pre-trained to improve the model training speed.

[0092] b) Input the word vector into the TCN network. The TCN network uses 21 layers of residual neural network layers and 5 layers of Pooling layers for model training, as Figure 3 shown. It is necessary to set the time step in advance, and the time step can be set according to the instruction input time to analyze the correlation degree between this instruction and its previous instructions.

[0093] c) The TCN network outputs the high-risk score of this instruction, compares its score with the true score to calculate the error, and performs backpropagation to optimize the network.

[0094] d) Obtain the model for each user.

[0095] 3. Apply the trained user model to the scoring of the input instructions of this user later.

[0096] Step 5: Detection result.

[0097] If the high-risk score of a certain instruction based on the error log and automated test is different from the high-risk score obtained by the deep learning model, and the score obtained by the scoring method based on the error log and automated test results exceeds the threshold (there is a risk), then give priority to evaluating this instruction according to the high-risk score obtained from the error log and automated test and re-optimize the user model based on this high-risk instruction; otherwise, obtain the final instruction high-risk score according to the deep learning model.

[0098] In summary, the present invention evaluates the abnormality and high-risk degree of user instructions through system error logs and automated test results, constructs an abnormal instruction dataset, and uses deep learning methods to quickly analyze the causal convolution relationship of instructions to evaluate the abnormality and high-risk degree of user instructions in real time. Compared with traditional custom rule detection, this method can give the high-risk evaluation score for each instruction; compared with traditional deep learning / machine learning methods, the detection speed of the present invention is increased by more than 200%, greatly accelerating the detection speed, and analyzing based on each instruction rather than instruction blocks, the detection mode is more fine-grained.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Those of ordinary skill in the art can modify or equivalently replace the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to what is described in the claims.

Claims

1. A real-time anomaly detection method for high-risk user instructions, applicable to operation and maintenance users. The steps of the method include: Collect the instruction data used by a user from the production environment, and obtain the system error log and the automated test results of the instruction data; Calculate the first high-risk score of the instruction data according to the instruction sub-parameters, system error log, and automated test results of the instruction data. Among them, calculating the first high-risk score of the instruction data according to the instruction sub-parameters, system error log, and automated test results of the instruction data includes: Search for the instruction scores in the high-risk instruction library based on the instruction name and instruction sub-parameters of the instruction data to assign a high-risk score to the instruction name; Assign a high-risk score to the system error log based on the number of system error logs; Assign a high-risk score to the automated test results based on the automated test results; Integrate the high-risk score of the instruction name, the high-risk score of the system error log, and the score of the automated test module results, and perform 3-sigma calculation on the normalized data to obtain the first high-risk score of the instruction data; Calculate the second high-risk score of the instruction data based on the word vector of the instruction data. Among them, calculating the second high-risk score of the instruction data based on the word vector of the instruction data includes: Respectively obtain the instruction data training sets of the user and other users; Insert the instruction data of other users into the instruction data training set of the user to construct an instruction data set; Based on the word embedding method, convert each instruction data in the instruction data set into a sample word vector; Input each of the sample word vectors into the TCN network, set the time step according to the instruction input time to analyze the correlation degree between the corresponding instruction data and the previous instruction data, and compare the output of the TCN network with the true score of the sample word vector, and backpropagate the error to optimize the TCN network to obtain an anomaly detection model. Among them, the structure of the TCN network includes: N layers of residual neural network layers and M layers of Pooling layers, where N and M are natural numbers; Input the word vector into the anomaly detection model to obtain the second high-risk score of the instruction data; Obtain the anomaly detection result of the instruction data according to the first high-risk score and the second high-risk score. Among them, obtaining the anomaly detection result of the instruction data according to the first high-risk score and the second high-risk score includes: If the anomaly detection result corresponding to the first high-risk score exceeds the 3-sigma threshold, then use the anomaly detection result corresponding to the first high-risk score as the anomaly detection result of the instruction data, and enter all the instruction sub-parameters in the instruction data into the high-risk anomaly instruction library; If the anomaly detection result corresponding to the first high-risk score indicates no danger, then use the anomaly detection result corresponding to the second high-risk score as the anomaly detection result of the instruction data.

2. The method according to claim 1, characterized in that, The collecting the instruction data used by a user from the production environment includes: Modify the user instruction history record rule based on the requirements of the operation and maintenance user; Collect the instruction data used by the user in the production environment according to the modified user instruction history rule.

3. The method according to claim 1, wherein After collecting the instruction data used by a user from the production environment, it further includes: Clean the instruction data to obtain the credibility score of the instruction data; Based on the credibility threshold, clean the instruction data with a credibility score lower than the threshold.

4. The method according to claim 3, wherein The cleaning of the instruction data to obtain the credibility score of the instruction data includes: Obtain the segmentation result of the instruction data, where the segmentation result includes: instruction name and instruction separator, the segmentation result further includes: instruction parameters, and the segmentation result further includes: instruction identifiers; Set the initial credibility score of the instruction data to full marks; Based on the segmentation result, obtain the first credibility score of the instruction data by checking whether the instruction data is an instruction marked in the operating system; Based on the segmentation result, obtain the second credibility score of the instruction data by judging whether the instruction data has a help identifier and the number of help identifiers; Based on the segmentation result, obtain the third credibility score of the instruction data by judging whether the instruction data exists in the system path; Based on the segmentation result, obtain the fourth credibility score of the instruction data by judging whether the instruction data exists in the context environment where the user is currently located; Deduct the first credibility score, the second credibility score, the third credibility score, and the fourth credibility score from the initial credibility score respectively to obtain the credibility score of the instruction data.

5. The method according to claim 4, characterized in that, The obtaining of the segmentation result of the instruction data includes: Obtain the instruction data training set of the user; For each instruction data sample in the training set, use a space as the delimiter for segmentation, and divide the instruction data sample into several instruction sub-parameters; Based on the annotation of the instruction sub-parameters, use a semi-supervised method based on sequence annotation to obtain a segmentation model; Input the instruction data into the segmentation model to obtain the segmentation result.

6. A storage medium, in which a computer program is stored, wherein, The computer program is set to execute the method as described in any one of claims 1-5 when running.

7. An electronic device, characterized in that, It includes a memory and a processor. A computer program is stored in the memory, and the processor is set to run the computer program to execute the method as described in any one of claims 1-5.

Images