System change operation auditing method and device

Abstract

The invention discloses a system change operation auditing method and device, relates to the technical field of information security and can also be used in the financial field, and the method comprises the following steps: classifying change operations in a system change scheme according to a pre-generated change scheme classification model to generate a plurality of classification results; selecting a corresponding word slot in a pre-generated word slot set according to the current classification result, and filling the current classification result into the selected word slot to generate a plurality of word slot results; extracting a plurality of keywords of a code corresponding to the system change operation; and checking whether the system change operation accords with the system change scheme or not according to the vector spaces corresponding to the plurality of word slot results and the vector spaces of the plurality of keywords. According to the invention, the to-be-examined system change scheme is analyzed and compared with the corresponding actual system change operation, so that the accuracy of the system change scheme is efficiently detected.

Description

Technical Field

[0001] This application belongs to the technical field of information security, especially the related technical field of information processing in this field, and specifically relates to a method and device for auditing system change operations. Background Art

[0002] With the rapid development of computer technology and Internet technology, enterprise network environments are becoming increasingly large and complex, and operation and maintenance operators need to frequently perform operation and maintenance operations such as version updates, equipment maintenance, and event emergencies. In new technology frameworks such as cloud and distributed, the operation risks are gradually amplified. For example, due to incorrect or imperfect change plans formulated by operation and maintenance personnel, such as missing change steps, the change implementation fails, thus triggering global system risks.

[0003] In summary, how to effectively audit the accuracy of change plans and prevent operation risks in advance is an urgent problem for operation and maintenance management personnel to solve. Summary of the Invention

[0004] The present invention can be used in the technical field of the application of information security technology in finance, and can also be used in any field other than the financial field.

[0005] An object of the present invention is to provide a method for auditing system change operations, to parse the system change plan to be reviewed and compare it with the corresponding actual system change operation, so as to efficiently detect the accuracy of the system change plan.

[0006] Another object of the present invention is to provide a device for auditing system change operations. Still another object of the present invention is to provide an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method for auditing system change operations are implemented. Still another object of the present invention is to provide a readable medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method for auditing system change operations are implemented.

[0007] To solve the technical problems in the background art of this application, the present invention provides the following technical solutions:

[0008] In the first aspect, the present invention provides a method for auditing system change operations, including:

[0009] Classify the change operations in the system change plan according to a pre-generated change plan classification model to generate multiple classification results;

[0010] Select the corresponding word slots from the pre-generated word slot set according to the current classification results, and fill the current classification results into the selected word slots to generate multiple word slot results;

[0011] Extract multiple keywords corresponding to the system change operation code;

[0012] According to the vector space corresponding to the multiple slot results and the vector space of the multiple keywords, review whether the system change operation complies with the system change plan.

[0013] In some embodiments of the present invention, the steps of generating the change plan classification model include:

[0014] Perform the following iterative operations until the predicted text category of the current training data generated by the change plan classification model is less than the preset threshold compared with the true text category of the current training data:

[0015] Input the current training data into the initial model corresponding to the current training round of the change plan classification model to determine the word encoding, sentence encoding, and position encoding of the training data;

[0016] Determine the association relationship between each word segment in the corresponding current training data according to the word encoding, sentence encoding, position encoding, and the initial model;

[0017] Determine the context features and global association features of the current training data according to the association relationship and the initial model;

[0018] Determine the predicted text category of the current training data according to the context features and the global association features;

[0019] Perform backpropagation on the initial model according to the predicted text category and the true text category to generate the change plan classification model.

[0020] In some embodiments of the present invention, the selecting the corresponding slot in the pre-generated slot set according to the current classification result includes:

[0021] Perform word segmentation on the current classification result to generate a word segmentation result;

[0022] Perform part-of-speech tagging on the word segmentation result to generate a part-of-speech result;

[0023] Identify the entities in the current classification result according to the word segmentation result and the part-of-speech result;

[0024] Determine the semantic structure of the current classification result according to the word segmentation result and the part-of-speech result to determine the roles and relationships of multiple entities in the current classification result;

[0025] Select corresponding word slots from the pre-generated word slot set according to the roles and relationships of multiple entities in the current classification result.

[0026] In some embodiments of the present invention, before auditing whether the system change operation conforms to the system change plan according to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, it further includes:

[0027] Determine the same type of change operations of the system change operation according to the multiple keywords;

[0028] Merge the same type of change operations.

[0029] In some embodiments of the present invention, before auditing whether the system change operation conforms to the system change plan according to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, it further includes:

[0030] Select valid change operations in the system change operation according to the multiple keywords; wherein, the valid change operations include: shutdown operations, user operations, service operations, and system resource operations.

[0031] In some embodiments of the present invention, auditing whether the system change operation conforms to the system change plan according to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords includes:

[0032] Generate a first sub-vector space corresponding to each word slot result;

[0033] Generate the vector space corresponding to the multiple word slot results according to the multiple first sub-vector spaces;

[0034] Generate a second sub-vector space corresponding to each valid change operation;

[0035] Generate the vector space of the multiple keywords according to the multiple second sub-vector spaces;

[0036] Determine the similarity between the vector space corresponding to the multiple word slot results and the vector space of the multiple keywords to audit whether the system change operation conforms to the system change plan.

[0037] In some embodiments of the present invention, a method for auditing a system change operation, the step of generating the word slot set includes:

[0038] Determine the query statements that appear more than a preset value in the system change plan;

[0039] Extract the entities and attributes of the query statements;

[0040] Generate the slot set according to the entity and the attribute.

[0041] In a second aspect, the present invention provides an audit device for system change operations, and the device includes:

[0042] A plurality of classification result generation modules, configured to classify the change operations in the system change plan according to a pre-generated change plan classification model to generate a plurality of classification results;

[0043] A plurality of slot result generation modules, configured to select corresponding slots from a pre-generated slot set according to the current classification result, and fill the current classification result into the selected slots to generate a plurality of slot results;

[0044] A plurality of keyword extraction modules, configured to extract a plurality of keywords of the code corresponding to the system change operation;

[0045] A change operation audit module, configured to audit whether the system change operation conforms to the system change plan according to the vector space corresponding to the plurality of slot results and the vector space of the plurality of keywords.

[0046] In some embodiments of the present invention, an audit device for system change operations further includes:

[0047] A classification model generation module, configured to generate the change plan classification model; the classification model generation module includes:

[0048] An iterative operation unit, configured to perform the following iterative operations until the predicted text category of the current training data generated by the change plan classification model is less than a preset threshold compared with the true text category of the current training data:

[0049] An encoding determination unit, configured to input the current training data into the initial model corresponding to the current training round of the change plan classification model to determine the word encoding, sentence encoding, and position encoding of the training data;

[0050] An association relationship determination unit, configured to determine the association relationship between each word segment in the corresponding current training data according to the word encoding, sentence encoding, position encoding, and the initial model;

[0051] A feature determination unit, configured to determine the context feature and global association feature of the current training data according to the association relationship and the initial model;

[0052] A predicted text category determination unit, configured to determine the predicted text category of the current training data according to the context feature and the global association feature;

[0053] A classification model generation unit, configured to perform backpropagation on the initial model according to the predicted text category and the true text category to generate the change scheme classification model.

[0054] In some embodiments of the present invention, the plurality of word slot result generation modules include:

[0055] A word segmentation result generation unit, configured to perform word segmentation on the current classification result to generate a word segmentation result;

[0056] A part-of-speech result generation unit, configured to perform part-of-speech tagging on the word segmentation result to generate a part-of-speech result;

[0057] An entity recognition unit, configured to recognize entities in the current classification result according to the word segmentation result and the part-of-speech result;

[0058] A role relationship determination unit, configured to determine the semantic structure of the current classification result according to the word segmentation result and the part-of-speech result to determine the roles and relationships of multiple entities in the current classification result;

[0059] A word slot selection unit, configured to select corresponding word slots from the pre-generated word slot set according to the roles and relationships of multiple entities in the current classification result.

[0060] In some embodiments of the present invention, an audit device for system change operations further includes:

[0061] A similar change operation determination module, configured to determine similar change operations of the system change operation according to the plurality of keywords;

[0062] A similar change operation merging module, configured to merge the similar change operations.

[0063] In some embodiments of the present invention, an audit device for system change operations further includes:

[0064] An effective change operation selection module, configured to select effective change operations in the system change operation according to the plurality of keywords; wherein, the effective change operations include: shutdown operations, user operations, service operations, and system resource operations.

[0065] In some embodiments of the present invention, the change operation audit module includes:

[0066] A first subspace generation unit, configured to generate a first sub-vector space corresponding to each word slot result;

[0067] A word slot space generation unit, configured to generate a vector space corresponding to the plurality of word slot results according to the plurality of first sub-vector spaces;

[0068] A second subspace generation unit, configured to generate a second sub-vector space corresponding to each valid change operation;

[0069] A keyword space generation unit, configured to generate a vector space of the plurality of keywords according to the plurality of second sub-vector spaces;

[0070] A change operation auditing unit, configured to determine the similarity between the vector space corresponding to the plurality of word slot results and the vector space of the plurality of keywords, so as to audit whether the system change operation complies with the system change plan.

[0071] In some embodiments of the present invention, an auditing device for system change operations further includes:

[0072] A word slot set generation module, configured to generate the word slot set; the word slot set generation module includes:

[0073] A query statement determination unit, configured to determine a query statement that appears more than a preset value in the system change plan;

[0074] An attribute extraction unit, configured to extract the entity and attributes of the query statement;

[0075] A word slot set generation unit, configured to generate the word slot set according to the entity and the attributes.

[0076] In a third aspect, the present invention provides a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, the steps of an auditing method for system change operations are implemented.

[0077] In a fourth aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the steps of an auditing method for system change operations are implemented.

[0078] In a fifth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of an auditing method for system change operations are implemented.

[0079] As can be seen from the above description, the embodiments of the present invention provide a method and apparatus for auditing system change operations. The corresponding method includes: First, classify the change operations in the system change plan according to a pre-generated change plan classification model to generate multiple classification results; Then, select the corresponding word slots from the pre-generated word slot set according to the current classification results, and fill the current classification results into the selected word slots to generate multiple word slot results; Extract multiple keywords of the code corresponding to the system change operation; Finally, according to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, audit whether the system change operation conforms to the system change plan.

[0080] The method and apparatus for auditing system change operations provided by the present invention perform data processing and similarity analysis on various operation plans such as change plans and emergency scenarios collected by enterprises, and obtain the gaps from the change plan to be reviewed, thereby providing a feasible technical solution for the accuracy review of the change plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0082] Figure 1 Flow schematic of a method for auditing system change operations in an embodiment of the present invention Figure 1 ;

[0083] Figure 2 Flow schematic of a method for auditing system change operations in an embodiment of the present invention Figure 2 ;

[0084] Figure 3 Flow schematic of step 500 of a method for auditing system change operations in an embodiment of the present invention;

[0085] Figure 4 Flow schematic of step 200 of a method for auditing system change operations in an embodiment of the present invention;

[0086] Figure 5 Flow schematic of a method for auditing system change operations in an embodiment of the present invention Figure 3 ;

[0087] Figure 6 Flow schematic of a method for auditing system change operations in an embodiment of the present invention Figure 4 ;

[0088] Figure 7 Schematic diagram of step 400 of a method for auditing system change operations in an embodiment of the present invention;

[0089] Figure 8 Schematic diagram of the process of a method for auditing system change operations in an embodiment of the present invention Figure 5 ;

[0090] Figure 9 Schematic diagram of step 900 of a method for auditing system change operations in an embodiment of the present invention;

[0091] Figure 10 Block diagram of an auditing system for system change operations in a specific embodiment of the present invention;

[0092] Figure 11 Structural diagram of change plan processing module 2 in a specific embodiment of the present invention;

[0093] Figure 12 Structural diagram of operation processing module 3 in a specific embodiment of the present invention;

[0094] Figure 13 Structural diagram of discrimination module 4 in a specific embodiment of the present invention;

[0095] Figure 14 Structural diagram of risk handling module 5 in a specific embodiment of the present invention;

[0096] Figure 15 Mind map of a method for auditing system change operations in a specific embodiment of the present invention;

[0097] Figure 16 Schematic diagram of the process of a method for auditing system change operations in a specific embodiment of the present invention;

[0098] Figure 17 Block diagram of an auditing device for system change operations in an embodiment of the present invention Figure 1 ;

[0099] Figure 18 Block diagram of an auditing device for system change operations in an embodiment of the present invention Figure 2 ;

[0100] Figure 19 Block diagram of classification model generation module 50 in an embodiment of the present invention;

[0101] Figure 20 Block diagram of multiple word slot result generation modules 20 in an embodiment of the present invention;

[0102] Figure 21 The block diagram of an audit device for system change operations in an embodiment of the present invention Figure 3 ;

[0103] Figure 22 The block diagram of an audit device for system change operations in an embodiment of the present invention Figure 4 ;

[0104] Figure 23 The block diagram of the change operation audit module 40 in an embodiment of the present invention;

[0105] Figure 24 The block diagram of an audit device for system change operations in an embodiment of the present invention Figure 5 ;

[0106] Figure 25 The block diagram of the word slot set generation module 90 in an embodiment of the present invention;

[0107] Figure 26 The structural schematic diagram of the electronic device in an embodiment of the present invention. Detailed implementation manners

[0108] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0109] Those skilled in the art should understand that the embodiments of the present invention may be provided as a method, a system, or a computer program product. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] It should be noted that the terms "including" and "having" and any variations thereof in the description, claims and the above-mentioned drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or devices. Without conflict, the embodiments in this application and the features in the embodiments may be combined with each other. The following will detail this application with reference to the drawings and in combination with the embodiments.

[0111] In the technical solution of this application, the acquisition, storage, use, processing, etc. of data all comply with the relevant regulations of laws and regulations. Specifically:

[0112] I. The information collected is information and data authorized by the user or fully authorized by all parties. And for the processing of relevant data such as collection, storage, use, processing, transmission, provision, disclosure and application, etc., all comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0113] II. Provide corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, then enter the expert decision-making process.

[0114] An embodiment of the present invention provides a specific implementation manner of a method for auditing system change operations. Refer to Figure 1 , and the method specifically includes the following contents:

[0115] Step 100: Classify the change operations in the system change plan according to a pre-generated change plan classification model to generate multiple classification results;

[0116] Step 200: Select corresponding word slots from a pre-generated word slot set according to the current classification results, and fill the current classification results into the selected word slots to generate multiple word slot results;

[0117] Step 300: Extract multiple keywords of the code corresponding to the system change operation;

[0118] Step 400: According to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, audit whether the system change operation complies with the system change plan.

[0119] As can be seen from the above description, an embodiment of the present invention provides a method for auditing system change operations, including: First, classify the change operations in the system change plan according to a pre-generated change plan classification model to generate multiple classification results; Then, select corresponding slots from the pre-generated slot set according to the current classification results, and fill the current classification results into the selected slots to generate multiple slot results; Extract multiple keywords of the code corresponding to the system change operation; Finally, according to the vector spaces corresponding to the multiple slot results and the vector spaces of the multiple keywords, audit whether the system change operation conforms to the system change plan.

[0120] The present invention provides an innovative method for using collected operation plans such as changes and emergencies to conduct comparative analysis on the change plan to be reviewed, so as to efficiently detect the accuracy of the change plan.

[0121] Preferably, the multiple classification results in step 100 include: change type, operation object, operation type, operation statement keywords, operating user, IP address.

[0122] For step 200, the slot is used to capture the key information of the current classification result, as well as the variable storing the key information required by the intention, which can be inherited in the conversation. According to the value in the slot, subsequent operations and feedback can be given.

[0123] Filling the current classification result into the selected slot means extracting this key information from the current classification result and filling it into the predefined slot. Limitedly, the way of filling the selected slot is at least one of whole sentence slot filling, intention slot filling, entity slot filling and role slot filling.

[0124] For step 300, as shown in Table 1, the corresponding keywords can be extracted according to the preset rules. For example, multiple keywords of the operation statement are extracted according to the operation object and the corresponding operation type.

[0125] Table 1 Example of parameter configuration white list method configuration

[0126] Operation Object 1 Operation Object 2 Operation Type Operation Statement Keyword Open Platform Server Unix Shutdown Class shutdown Open Platform Server Unix Shutdown Class reboot Open Platform Server Unix User Class useradd

[0127] It can be understood that the vector space in step 400 is a data structure representing the text and words of multiple slot results, as well as the text and words of multiple keywords. Specifically, the text data is converted into a numerical vector so that it can be operated and processed in a mathematical and geometric space.

[0128] In addition, when step 400 is implemented, first, the vector spaces corresponding to the multiple slot results and the vector spaces of the multiple keywords are clustered (grouping similar texts together) and dimensionally reduced (reducing the high-dimensional vector to a low-dimensional space for easy processing) at one time, and then the similarity of these two vector spaces is calculated.

[0129] In some embodiments of the present invention, referring to Figure 2 , a method for auditing system change operations further includes:

[0130] Step 500: Generate the change plan classification model, referring to Figure 3 , and step 500 includes:

[0131] Step 501: Perform the following iterative operations until the predicted text category of the current training data generated by the change plan classification model is less than a preset threshold compared with the true text category of the current training data:

[0132] Step 502: Input the current training data into the initial model corresponding to the current training round of the change plan classification model to determine the word encoding, sentence encoding, and position encoding of the training data;

[0133] Specifically, the current training data is tokenized through the tokenization module of the initial model corresponding to the current training round to determine the word encoding, sentence encoding, and position encoding. Further, tokenization processing can be performed through the open-source tokenization library jieba.

[0134] Step 503: Determine the association relationship between each token in the corresponding current training data according to the word encoding, sentence encoding, position encoding, and the initial model;

[0135] Step 504: Determine the context features and global association features of the current training data according to the association relationship and the initial model;

[0136] Step 505: Determine the predicted text category of the current training data according to the context features and the global association features;

[0137] The context features are used to characterize the context relationship between adjacent tokens in the current training data, and the global association features are used to characterize the mutual influence degree between every two tokens in the current training data.

[0138] Step 506: Perform backpropagation on the initial model according to the predicted text category and the true text category to generate the change plan classification model.

[0139] It can be understood that step 506 calculates the error between the output of the model network and the expected output, and propagates this error signal layer by layer from the output layer back to the input layer, and updates the parameter weights of each layer, so that the output of the network gradually approaches the expected target. The working process of the backpropagation algorithm is as follows:

[0140] Forward propagation: Calculate the output result from the input layer to the output layer through the network.

[0141] Calculation error: Compare the output result with the expected output to calculate the error.

[0142] Backpropagation error: Propagate the error of the output layer backward to each hidden layer to calculate the error gradient of each layer.

[0143] Update parameters: According to the error gradient, use an optimization algorithm (such as gradient descent) to update the parameter weights of the network to minimize the error.

[0144] This process will be repeated for multiple rounds until the performance of the network reaches a satisfactory level.

[0145] The backpropagation algorithm relies on the chain rule to calculate the gradients of each layer, which enables it to effectively optimize the parameters in a deep network.

[0146] In some embodiments of the present invention, referring to Figure 4 , step 200 includes:

[0147] Step 201: Segment the current classification result to generate a word segmentation result;

[0148] Divide the current classification result into independent words.

[0149] Step 202: Perform part-of-speech tagging on the word segmentation result to generate a part-of-speech result;

[0150] Determine the part of speech of each word segmentation result, such as noun, verb, adjective, etc.

[0151] Step 203: Identify the entities in the current classification result according to the word segmentation result and the part-of-speech result;

[0152] Identify the entities in the current classification result, such as marked change type, operation object, operation type, operation statement keyword, operating user, etc.

[0153] Step 204: Determine the semantic structure of the current classification result according to the word segmentation result and the part-of-speech result to determine the roles and relationships of multiple entities in the current classification result;

[0154] Specifically, based on steps 201 to 203, core anaphora resolution (determine which words or phrases in the text refer to the same entity), semantic role labeling, and word sense disambiguation (determine the exact meaning of a polysemous word in a specific context) are performed in sequence. Finally, a dependency graph reflecting the semantic relationships between words (such a graph can represent the semantic structure of a sentence) is established to show who did what and how.

[0155] Step 205: Select corresponding word slots from the pre-generated word slot set according to the roles and relationships of multiple entities in the current classification result.

[0156] In some embodiments of the present invention, referring to Figure 5 , before step 500, a method for auditing system change operations further includes:

[0157] Step 600: Determine the same type of change operations of the system change operation according to the multiple keywords;

[0158] Step 700: Merge the same type of change operations.

[0159] In step 600 and step 700, perform the operation of merging the same type of change operations, eliminate the occasional operations, only retain one operation data for the same type of change operations, and add an operation serial number.

[0160] In some embodiments of the present invention, referring to Figure 6 , before step 500 and after step 700, a method for auditing system change operations further includes:

[0161] Step 800: Select the effective change operations in the system change operation according to the multiple keywords; wherein, the effective change operations include: shutdown operations, user operations, service operations, and system resource operations.

[0162] Specifically, clean the change plan to be analyzed, remove invalid information such as text annotations, and only retain the operation step information to be concerned, including the operation serial number (used to identify the operation sequence of the operation step), operation command, operating user, IP address, etc.

[0163] In some embodiments of the present invention, referring to Figure 7 , step 400 includes:

[0164] Step 401: Generate the first sub-vector space corresponding to each word slot result;

[0165] Step 402: Generate the vector space corresponding to the multiple word slot results according to the multiple first sub-vector spaces;

[0166] Specifically, obtain the vector space corresponding to the multiple word slot results according to formula (1):

[0167] Vr = Σv rj (1)

[0168] Wherein, Vr is the vector space corresponding to the multiple word slot results, and v rj is the first sub-item vector space.

[0169] Step 403: Generate a second sub-vector space corresponding to each valid change operation;

[0170] Step 404: Generate a vector space of the multiple keywords based on the multiple second sub-vector spaces;

[0171] Specifically, obtain the vector space of the multiple keywords according to formula (2):

[0172] Vc = Σv ci (2)

[0173] where Vc is the vector space of the multiple keywords, and v ci is the second sub-vector space.

[0174] Step 405: Determine the similarity between the vector space corresponding to the multiple word slot results and the vector space of the multiple keywords, so as to review whether the system change operation conforms to the system change plan.

[0175] Specifically, calculate the similarity in step 405 according to the cosine similarity, that is, formula (3):

[0176]

[0177] In some embodiments of the present invention, referring to Figure 8 , a method for reviewing a system change operation further includes:

[0178] Step 900: Generate the word slot set. Then, referring to Figure 9 , step 900 includes:

[0179] Step 901: Determine the query statements that appear more than a preset value in the system change plan;

[0180] By step 901, common query statements can be selected. For example: View the definition of database objects: SHOW CREATE TABLE table_name; SHOW CREATE DATABASE database_name; SHOW CREATE VIEW view_name;

[0181] View the status of database objects: SHOW TABLE STATUS; SHOW INDEX FROM table_name; SHOW PROCESSLIST;

[0182] View the database configuration information: SHOW VARIABLES; SHOW GLOBAL VARIABLES; SHOW SESSION VARIABLES;

[0183] Permissions information for viewing the database: SHOW GRANTS FOR user_name; SHOW PRIVILEGES;

[0184] Log information for viewing the database: SHOW BINARY LOGS; SHOW SLAVE STATUS; SHOW MASTER STATUS; and

[0185] Connection information for viewing the database: SHOW FULL PROCESSLIST; SHOW OPEN TABLES; SHOW STATUS;

[0186] Step 902: Extract the entities and attributes of the query statement;

[0187] Specifically, first convert the common query statement into a character sequence. Then, randomly initialize an n-dimensional word vector for each character in the character sequence. Let m be the number of characters in the character sequence after the character conversion operation. At this time, the size of the character embedding matrix obtained is m×d. Finally, extract the entities and attributes in the query statement based on this character embedding matrix.

[0188] Step 903: Generate the word slot set according to the entities and the attributes.

[0189] To further illustrate the solution, the present invention also provides a specific implementation manner of the review method for system change operations, which specifically includes the following content.

[0190] First, refer to Figure 10 , the present invention provides a review system for system change operations, and the system includes: a parameter configuration module 1, a change plan processing module 2, an operation processing module 3, a discrimination module 4, and a risk handling module 5.

[0191] The main function of the parameter configuration module 1 is to configure the operation objects, operation types, and operation statements that need to be concerned. The main function of the change plan processing module 2 is to perform natural language technology processing on the change plan to be reviewed according to the parameter configuration, extract the concerned operation steps, and mark the corresponding tags. The main function of the operation processing module 3 is to perform natural language technology processing on the implementation steps related to operations such as the existing change plans and emergency scenarios to be referred to according to the parameter configuration, extract the concerned operation steps, and mark the corresponding tags, so as to serve as the basis for judging the accuracy of the change implementation plan to be reviewed. The main function of the discrimination module 4 is to perform natural language similarity analysis on the tagged change plan to be reviewed and the reference operation data, and output the matching and analysis results. The risk handling module 5 is to take response measures according to the predetermined process and strategy for the output result of the discrimination module 4. Specifically:

[0192] The parameter configuration module 1 is used to configure the change types, operation objects, operation types, and operation statements that need to be concerned about. Among them, the change types can include types such as version production, emergency drill, emergency handling, and daily operation and maintenance, and are supplemented with more detailed type information such as specific version numbers and emergency scenario numbers as the scenario identifiers for distinguishing specific change operations. The operation objects can include open platform servers, databases, network devices, etc., and a secondary operation object field can be established to further refine the parameter configuration. For example, open platform servers can include Windows servers, Unix servers, etc. The operation types are configured according to the operation objects. For example, open platform servers include shutdown operations, user operations, service operations, system resource operations, software management operations, etc. The operation statements are the operation command keywords that need to be concerned about under specific operation types. The configured operation types and operation statements are shown in Table 1.

[0193] Figure 11 It is a schematic diagram of the internal structure of the change plan processing module 2, including: a change plan classifier 201, a change plan cleaning unit 202, and a change plan classification and tagging unit 203.

[0194] The change plan classifier 201 is used to install, build, and train a change plan classifier, and input labeled data. Each line of the text file of the labeled data contains a list of labels, followed by the corresponding operation information. All labels start with the __label__ prefix and respectively label the change type, operation object, operation type, operation statement keyword, operating user, and IP address.

[0195] The change plan cleaning unit 202 is used to clean the change plan to be analyzed according to the parameter configuration module 1, remove invalid information such as text annotations, and only retain the operation step information that needs to be concerned about, including operation sequence numbers, operation commands, operating users, IP addresses, etc. The operation sequence number identifies the operation sequence of the operation steps.

[0196] The change plan classification and tagging unit 203 is used to classify and tag the cleaned change plan and retain the operation sequence number.

[0197] Figure 12 It is a schematic diagram of the internal structure of the operation processing module 3, including: an operation data classifier 301, an operation data cleaning unit 302, an operation data classification and tagging unit 303, and an operation data processing unit 304. Specifically:

[0198] The operation data classifier 301 is used to input labeled data. Each line of the text file of the labeled data contains a list of labels, followed by the corresponding operation information. All labels start with the __label__ prefix and respectively label the change type, operation object, operation type, operation statement keyword, operating user, and IP address.

[0199] The operation data cleaning unit 302 is used to clean the operation data according to the parameter configuration module 1, remove invalid information such as text annotations, and only retain the operation step information that needs to be concerned about, including operation commands, operating users, IP addresses, etc.

[0200] The operation data classification and tagging unit 303 is used to perform classification and tagging.

[0201] The operation data processing unit 304 is used to use the text clustering algorithm to process the output result of the operation data classification and tagging unit 303 according to the change type, merge similar change operations, eliminate occasional operations, only retain one operation data for the same type of change operation, and add an operation serial number.

[0202] Figure 13 It is a schematic diagram of the internal structure of the discrimination module 4, including: a data matching unit 401 and a result analysis unit 402. Specifically:

[0203] The data matching unit 401 performs a full-volume label comparison on the output data of the change plan classification and tagging unit 203 and the operation data processing unit 304 to obtain the effective change operation steps of the change plan to be reviewed under the relevant change types, and enters the result analysis unit 402.

[0204] The result analysis unit 402 analyzes and processes the output data of the operation data processing unit 304 and the data output by the data matching unit 401. The processing process is as follows: The output data of the operation data processing unit 304 is sent to a relevant model for generating word vectors to obtain the vector space representation of each label of the similar change operations in the system change plan, and the vector representations of each label in the system change plan are added together to obtain the vector representation of the change plan.

[0205] Similarly, after processing the effective change operation data output by the data matching unit 401, the vector space representation of the labels of the effective change operation data is obtained, and the vector space representations of each label are added together to obtain the vector representation. Subsequently, the cosine similarity algorithm is used to calculate the similarity between the operation data and the effective change operation steps.

[0206] Figure 14 It is a schematic diagram of the internal structure of the risk handling module 5. The risk handling module 5 includes: a risk analysis unit 501 and a risk response unit 502.

[0207] The risk analysis unit 501 is used to make a judgment based on the calculation result of the result analysis unit 402. If the similarity between the effective operation steps in the change implementation plan and the operation data is close to 1, it is determined that the change implementation plan is correctly formulated. If the two are much less than 1, it is determined that the change implementation plan is incorrect. The specific similarity threshold can be customized.

[0208] The risk response unit 502 is used to take the determination result of the risk analysis unit 501 and the output result of the data matching unit 401 as the result output, and send it through email or a warning device to prompt the management personnel to perform subsequent processing. It can be linked to the change control platform or the user management platform to prohibit the actual implementation of the change, so as to achieve the role of pre-control.

[0209] Based on the above-mentioned audit system for system change operations, see Figure 15 and Figure 16 , the specific implementation manner of the audit method for system change operations provided by the present invention includes the following steps.

[0210] Step S01: Configure the operation and maintenance operations in the parameter configuration module 1;

[0211] Step S02: Train the change classifier according to the input marked data;

[0212] Step S03: Clean the change plan;

[0213] Step S04: Mark the change plan according to the parameter configuration;

[0214] Step S05: Train the operation data according to the input marked data;

[0215] Step S06: Clean the operation data;

[0216] Step S07: Process the operation data for the same type of change operation;

[0217] Step S08: Mark the operation data according to the parameter configuration;

[0218] Step S09: Match and screen the marked results of the change plan and the operation data to obtain effective change operation steps;

[0219] Step S10: Compare and analyze the effective change operation steps of the change to be reviewed with the reference operation data;

[0220] Step S11: Judge the risk based on the result of the comparative analysis;

[0221] Step S12: The change plan is correct;

[0222] Step S13: The change plan is incorrect, and subsequent risk disposal is carried out.

[0223] As can be seen from the above description, the specific implementation of the present invention provides a method for auditing system change operations, including: First, classify the change operations in the system change plan according to a pre-generated change plan classification model to generate multiple classification results; then, select the corresponding word slots from the pre-generated word slot set according to the current classification results, and fill the current classification results into the selected word slots to generate multiple word slot results; extract multiple keywords of the code corresponding to the system change operation; finally, according to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, audit whether the system change operation conforms to the system change plan.

[0224] The present invention judges the accuracy of the change plan by performing data processing and comprehensive analysis on the change plan to be reviewed, and reference data such as known existing change plans and emergency scenario steps. Specifically, the present invention has the following beneficial effects:

[0225] 1. Wide application range. To ensure the stable operation of each device, each enterprise has formulated a change management process according to standards such as ITIL, incorporated all operations on the production environment into the scope of change management, and required that the change plan be at the instruction level when formulated. Therefore, the present invention is applicable to all change types, including version production, daily operation and maintenance, emergency handling, etc.

[0226] 2. High-efficiency and accurate detection. The natural language processing model is suitable for processing short texts and takes into account word order information. When processing change operations that need to consider time series, it can accurately perform statement recognition, marking, etc. processing. And by cooperating with relevant models that generate word vectors for label embedding, the discrimination result can be made more accurate, giving full play to the advantages of natural language in text processing and providing feasibility for efficient judgment of the accuracy of the change plan.

[0227] Based on the same inventive concept, the embodiment of the present application also provides an auditing device for system change operations, which can be used to implement the method described in the above embodiment, as in the following embodiment. Since the principle of the auditing device for system change operations to solve problems is similar to that of the auditing method for system change operations, the implementation of the auditing device for system change operations can refer to the implementation of the auditing method for system change operations, and the repeated parts will not be elaborated. Hereinafter, the term "unit" or "module" may be a combination of software and / or hardware that can achieve a predetermined function. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0228] The embodiment of the present invention provides a specific implementation of an auditing device for system change operations that can implement the auditing method for system change operations. Refer to Figure 17 , and a specific auditing device for system change operations includes the following:

[0229] A plurality of classification result generation modules 10, configured to classify change operations in a system change plan according to a pre-generated change plan classification model, so as to generate a plurality of classification results;

[0230] A plurality of slot result generation modules 20, configured to select corresponding slots from a pre-generated slot set according to the current classification result, and fill the current classification result into the selected slots, so as to generate a plurality of slot results;

[0231] A plurality of keyword extraction modules 30, configured to extract a plurality of keywords of the code corresponding to the system change operation;

[0232] A change operation review module 40, configured to review whether the system change operation conforms to the system change plan according to the vector space corresponding to the plurality of slot results and the vector space of the plurality of keywords.

[0233] In some embodiments of the present invention, referring to Figure 18 , an auditing device for system change operations further includes:

[0234] A classification model generation module 50, configured to generate the change plan classification model; referring to Figure 19 , the classification model generation module 50 includes:

[0235] An iterative operation unit 50a, configured to perform the following iterative operations until the predicted text category of the current training data generated by the change plan classification model is less than a preset threshold compared with the true text category of the current training data:

[0236] An encoding determination unit 50b, configured to input the current training data into the initial model corresponding to the current training round of the change plan classification model, so as to determine the word encoding, sentence encoding, and position encoding of the training data;

[0237] An association relationship determination unit 50c, configured to determine the association relationship between each word segment in the corresponding current training data according to the word encoding, sentence encoding, position encoding, and the initial model;

[0238] A feature determination unit 50d, configured to determine the context feature and global association feature of the current training data according to the association relationship and the initial model;

[0239] A predicted text category determination unit 50e, configured to determine the predicted text category of the current training data according to the context feature and the global association feature;

[0240] A classification model generation unit 50f, configured to perform backpropagation on the initial model according to the predicted text category and the true text category, so as to generate the change plan classification model.

[0241] In some embodiments of the present invention, referring to Figure 20 , the plurality of word slot result generation modules 20 includes:

[0242] A word segmentation result generation unit 20a, configured to perform word segmentation on the current classification result to generate a word segmentation result;

[0243] A part-of-speech result generation unit 20b, configured to perform part-of-speech tagging on the word segmentation result to generate a part-of-speech result;

[0244] An entity recognition unit 20c, configured to recognize entities in the current classification result according to the word segmentation result and the part-of-speech result;

[0245] A role relationship determination unit 20d, configured to determine the semantic structure of the current classification result according to the word segmentation result and the part-of-speech result, so as to determine the roles and relationships of multiple entities in the current classification result;

[0246] A word slot selection unit 20e, configured to select corresponding word slots from the pre-generated word slot set according to the roles and relationships of multiple entities in the current classification result.

[0247] In some embodiments of the present invention, referring to Figure 21 , an audit device for system change operations further includes:

[0248] A similar change operation determination module 60, configured to determine similar change operations of the system change operation according to the plurality of keywords;

[0249] A similar change operation merging module 70, configured to merge the similar change operations.

[0250] In some embodiments of the present invention, referring to Figure 22 , an audit device for system change operations further includes:

[0251] An effective change operation selection module 80, configured to select effective change operations from the system change operations according to the plurality of keywords; wherein, the effective change operations include: shutdown operations, user operations, service operations, and system resource operations.

[0252] In some embodiments of the present invention, referring to Figure 23 , the change operation audit module 40 includes:

[0253] The first subspace generation unit 40a is configured to generate a first sub-vector space corresponding to each slot result;

[0254] The slot space generation unit 40b is configured to generate a vector space corresponding to the multiple slot results according to the multiple first sub-vector spaces;

[0255] The second subspace generation unit 40c is configured to generate a second sub-vector space corresponding to each valid change operation;

[0256] The keyword space generation unit 40d is configured to generate a vector space of the multiple keywords according to the multiple second sub-vector spaces;

[0257] The change operation review unit 40e is configured to determine the similarity between the vector space corresponding to the multiple slot results and the vector space of the multiple keywords, so as to review whether the system change operation complies with the system change plan.

[0258] In some embodiments of the present invention, referring to Figure 24 , a review device for system change operations further includes:

[0259] The slot set generation module 90 is configured to generate the slot set; then, referring to Figure 25 , the slot set generation module 90 includes:

[0260] The query statement determination unit 90a is configured to determine a query statement that appears more than a preset value in the system change plan;

[0261] The attribute extraction unit 90b is configured to extract the entity and attributes of the query statement;

[0262] The slot set generation unit 90c is configured to generate the slot set according to the entity and the attributes.

[0263] As can be seen from the above description, an embodiment of the present invention provides a review device for system change operations, including: a plurality of classification result generation modules, configured to classify change operations in a system change plan according to a pre-generated change plan classification model to generate a plurality of classification results; a plurality of slot result generation modules, configured to select corresponding slots from a pre-generated slot set according to the current classification result and fill the current classification result into the selected slots to generate a plurality of slot results; a plurality of keyword extraction modules, configured to extract a plurality of keywords of the code corresponding to the system change operation; and a change operation review module, configured to review whether the system change operation complies with the system change plan according to the vector space corresponding to the plurality of slot results and the vector space of the plurality of keywords.

[0264] An audit device for system change operations provided by the present invention performs data processing and similarity analysis on various operation plans such as change plans and emergency scenarios collected by enterprises, and obtains the gaps from the change plans to be reviewed, thereby providing a feasible technical solution for the accuracy review of change plans.

[0265] It should be noted that an audit method and device for system change operations provided by an embodiment of the present invention can be used in the financial field and can also be used in any technical field other than the financial field. The embodiment of the present invention does not limit the application field of an audit method and device for system change operations.

[0266] An embodiment of the present application also provides a specific implementation manner of an electronic device that can implement all steps in the audit method for system change operations in the above embodiment. See Figure 26 , and the electronic device specifically includes the following content:

[0267] A processor (1201), a memory (1202), a communication interface (1203), and a bus (1204);

[0268] Among them, the processor (1201), the memory (1202), and the communication interface (1203) complete mutual communication through the bus (1204); the communication interface (1203) is used to implement information transmission between related devices such as server-side devices and user-side devices;

[0269] The processor (1201) is used to call a computer program in the memory (1202). When the processor executes the computer program, all steps in the audit method for system change operations in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0270] Step 100: Classify the change operations in the system change plan according to a pre-generated change plan classification model to generate multiple classification results;

[0271] Step 200: Select corresponding word slots from a pre-generated word slot set according to the current classification results, and fill the current classification results into the selected word slots to generate multiple word slot results;

[0272] Step 300: Extract multiple keywords of the code corresponding to the system change operation;

[0273] Step 400: According to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, audit whether the system change operation conforms to the system change plan.

[0274] An embodiment of the present application also provides a computer-readable storage medium that can implement all steps in the audit method for system change operations in the above embodiments. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all steps of the audit method for system change operations in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0275] Step 100: Classify the change operations in the system change plan according to a pre-generated change plan classification model to generate multiple classification results;

[0276] Step 200: Select corresponding slots from a pre-generated slot set according to the current classification results, and fill the current classification results into the selected slots to generate multiple slot results;

[0277] Step 300: Extract multiple keywords of the code corresponding to the system change operation;

[0278] Step 400: According to the vector spaces corresponding to the multiple slot results and the vector spaces of the multiple keywords, audit whether the system change operation conforms to the system change plan.

[0279] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program type embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0280] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0281] Although this application provides method operation steps such as in the embodiments or flowcharts, based on routine or non-creative labor, there can be more or fewer operation steps. The step order listed in the embodiments is only one way among many step execution orders and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the method order shown in the embodiments or the drawings or in parallel (such as in an environment of parallel processors or multithreaded processing).

[0282] For the convenience of description, when describing the above device, various modules are described separately according to their functions. Of course, when implementing the embodiments of this specification, the functions of each module can be implemented in the same or multiple software and / or hardware, or the modules implementing the same function can be realized by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0283] Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, the method steps can be logically programmed to enable the controller to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Therefore, such a controller can be regarded as a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.

[0284] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0285] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash RAM). The memory is an example of computer-readable media.

[0286] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the key point of each embodiment is to illustrate the differences from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for relevant content. In the description of this specification, the description of reference terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of this specification. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0287] The above is only the embodiment of the embodiments of this specification and is not used to limit the embodiments of this specification. For those skilled in the art, various changes and modifications can be made to the embodiments of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of this specification shall be included within the scope of the claims of the embodiments of this specification.

Claims

1. A method for auditing system change operations, characterized in that: include: Classifying the change operations in the system change plan according to a pre-generated change plan classification model to generate multiple classification results; Selecting a corresponding word slot from a pre-generated word slot set according to a current classification result, and filling the current classification result into the selected word slot to generate a plurality of word slot results; Extract multiple keywords of the code corresponding to the system change operation; According to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, it is reviewed whether the system change operation complies with the system change plan.

2. The audit method according to claim 1, characterized in that: The steps of generating the change scheme classification model include: The following iterative operations are performed until the predicted text category of the current training data generated by the classification model of the change scheme and the actual text category of the current training data are less than a preset threshold: Inputting the current training data into the initial model corresponding to the current training round of the change scheme classification model to determine the word coding, sentence coding and position coding of the training data; Determine the association relationship between each word segment in the corresponding current training data according to the word code, sentence code, position code and the initial model; Determine the context features and global association features of the current training data according to the association relationship and the initial model; Determine the predicted text category of the current training data according to the context feature and the global association feature; The initial model is back-propagated according to the predicted text category and the true text category to generate the change scheme classification model.

3. The audit method according to claim 1, characterized in that: The selecting a corresponding word slot from the pre-generated word slot set according to the current classification result includes: Performing word segmentation on the current classification result to generate a word segmentation result; Performing part-of-speech tagging on the word segmentation result to generate a part-of-speech result; Identify entities in the current classification result according to the word segmentation result and the part-of-speech result; Determine the semantic structure of the current classification result according to the word segmentation result and the part-of-speech result, so as to determine the roles and relationships of multiple entities in the current classification result; According to the roles and relationships of the multiple entities in the current classification result, a corresponding word slot is selected from the pre-generated word slot set.

4. The audit method according to claim 1, characterized in that: Before reviewing whether the system change operation complies with the system change plan according to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, the method further includes: Determining similar change operations of the system change operation according to the multiple keywords; Merge the same type of change operations.

5. The audit method according to claim 4, characterized in that: Before reviewing whether the system change operation complies with the system change plan according to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, the method further includes: Select valid change operations from the system change operations according to the multiple keywords; wherein the valid change operations include: shutdown operations, user operations, service operations, and system resource operations.

6. The audit method according to claim 5, characterized in that: According to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords, reviewing whether the system change operation complies with the system change plan includes: Generate the first sub-vector space corresponding to each word slot result; Generate vector spaces corresponding to the multiple word slot results according to the multiple first sub-vector spaces; Generate a second subvector space corresponding to each valid change operation; Generate vector spaces of the plurality of keywords according to the plurality of second sub-vector spaces; Determine the similarity between the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords to review whether the system change operation complies with the system change plan.

7. The audit method according to any one of claims 1 to 6, characterized in that: The steps of generating the word slot set include: Determine the query statements in the system change plan that appear more than a preset value in number of times; Extracting entities and attributes of the query statement; The slot set is generated according to the entity and the attribute.

8. A system change operation audit device, characterized in that: include: Multiple classification result generation modules, used for classifying the change operations in the system change plan according to the pre-generated change plan classification model to generate multiple classification results; A multiple word slot result generating module, used for selecting corresponding word slots from a pre-generated word slot set according to a current classification result, and filling the current classification result into the selected word slot to generate multiple word slot results; Multiple keyword extraction modules, used to extract multiple keywords from the code corresponding to the system change operation; The change operation review module is used to review whether the system change operation complies with the system change plan according to the vector spaces corresponding to the multiple word slot results and the vector spaces of the multiple keywords.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the system change operation audit method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the system change operation audit method described in any one of claims 1 to 7 are implemented.

Images