An intelligent evaluation and replay testing method based on deep learning

Abstract

The present invention discloses an intelligent evaluation and replay test method based on deep learning, comprising the following steps: 1) establishing a deep learning model based on user behavior sequences, taking the user behavior sequences as input, and obtaining a general user operation representation; taking the user behavior sequences as input, converting the user operation representation of each behavior into a vector value, and obtaining a vector feature of each behavior node; using a deep learning model based on user behavior sequences to assign different weights to vector feature parameters, and then obtaining a vector weighted cumulative value after XORing with the user features; 2) self-test evaluation and diagnosis and repair based on deep learning, and performing admission analysis and replay based on deep learning. The method of the present invention learns based on the user's behavior sequence, and the operator only needs to make appropriate adjustments and minor modifications according to needs, which greatly reduces the complexity of operation and improves the efficiency of troubleshooting and the system replay capability.

Description

Technical Field

[0001] The present invention relates to ship system information technology, and in particular to an intelligent evaluation and replay testing method based on deep learning. Background Art

[0002] Currently, the testing of key ship systems is mainly based on manual judgment, including consistency comparison of sent and received data and system status judgment. The means of troubleshooting and locating problems found during the test mainly rely on manual screening and message analysis after the detection and admission equipment captures the message. The system does not have intelligent self-inspection and self-repair capabilities; and the screening function of the currently installed detection and admission equipment is single and does not have a custom sorting and optimization function. Manual message analysis and screening and problem positioning are slow, the probability of analysis errors is high, and the efficiency of problem solving is low. Especially for test problems that require parsing a large number of data messages for positioning, such as track continuity problems, batch loss and replacement problems, and inconsistent situation quantity problems, there are often no effective solutions.

[0003] In order to solve the above problems, information assurance collection systems and portable recording equipment were introduced, both of which improved problem-solving efficiency and system replay capabilities through screen operation recording and playback, message recording, playback and replay, but both had limited solution effects. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an intelligent evaluation and replay testing method based on deep learning in response to the defects in the existing technology.

[0005] The technical solution adopted by the present invention to solve the technical problem is: an intelligent evaluation and replay test method based on deep learning, comprising the following steps:

[0006] 1) Establish a deep learning model based on user behavior sequences. Take the user behavior sequence as input and divide each behavior i in the user behavior sequence into object information item_i (IP, message type, message information unit identifier) ​​and behavior attribute property_i (scene, time, operation) to obtain a universal user operation representation;

[0007] The object information includes an IP address, a message type, and a message information unit identifier;

[0008] The behavior attributes include the scene, time and operation of the user behavior;

[0009] Taking the user behavior sequence as input, the user operation representation of each behavior is converted into a vector value, and the vector features of each behavior node are obtained, including the corresponding time, scene, operation status (query, replay, admission, operation and maintenance, testing, etc.), input source IP, destination IP, message type, message information unit identifier, and information field;

[0010] A deep learning model based on user behavior sequences is used to weight the hidden states. Different weights are assigned to the vector feature parameters, which are then XORed with the user features to obtain the vector weighted cumulative value.

[0011] 2) Self-test evaluation and diagnosis and repair based on deep learning

[0012] Self-test evaluation and diagnosis and repair based on deep learning include self-test evaluation and self-diagnosis and repair;

[0013] 2.1) Use the self-test evaluation model to conduct self-test evaluation;

[0014] A set of test evaluation rules is set up based on prior experience as the initial test evaluation rules of the self-test evaluation model;

[0015] Periodically conduct self-testing and expand the rule base;

[0016] If the self-test evaluation model fails to properly identify the status failure, the model will start learning after manual intervention to expand the rule base and trigger mechanism;

[0017] If the self-test evaluation model correctly identifies the system / device status but fails to properly identify data transmission and reception anomalies, the model will learn after manual intervention to expand the rule base and trigger mechanism;

[0018] If the self-test evaluation model correctly identifies abnormal data transmission and reception, but fails to properly identify abnormal data parsing, the model will learn after manual intervention to expand the rule base and trigger mechanism;

[0019] Conduct self-test assessment using the updated self-test assessment model;

[0020] 2.2) Use the self-diagnosis and repair model to perform self-diagnosis and repair;

[0021] Based on prior experience, a set of diagnosis and repair rules is established as the initial diagnosis and repair rules for the self-test evaluation model. The system has the ability to identify typical problems and repair typical, easily repairable software problems, including those that can be accurately located based on abnormal values ​​returned by functions or abnormal analysis code. The system also has the ability to propose solutions to other typical problems.

[0022] Periodically conduct self-testing and expand the rule base;

[0023] If the self-diagnosis and repair model correctly identifies a fault during testing, it enters the self-diagnosis and repair process. For typical software issues identified as easily repairable, automatic error correction is performed. For other typical issues, a rule-based optimal solution is proposed to guide fault location, shortening the fault location cycle.

[0024] If the self-diagnosis and repair model fails to identify a typical problem, the operator will manually intervene to improve the rule base based on the problem solving situation, and the self-diagnosis and repair model will start learning to expand the trigger mechanism and repair mechanism;

[0025] Use the updated self-diagnosis and repair model to perform self-diagnosis and repair;

[0026] 2.3) Conduct deep learning-based admission analysis replay;

[0027] Initially, a set of admission, analysis, and replay rules are set based on prior experience. The default settings include typical feature values ​​such as source IP, destination IP, message sorting method, and message type, and corresponding weights are assigned to them. The system makes real-time corrections to the typical feature values ​​and their weights based on each operator's operation. The specific process is as follows:

[0028] During the data admission process, if the default recommended admission rules meet the operator's needs, the admission process will proceed directly; otherwise, the operator will manually intervene and modify the default recommended admission rules. The model will record and learn the previous operator behavior process, update the weights corresponding to each vector behavior feature parameter, and synchronously update the admission rule plan;

[0029] During the data analysis process, if the default recommended analysis rules meet the operator's needs, the analysis process will proceed directly; otherwise, the operator will manually intervene to modify the data analysis rules. The system will learn based on the previous operator behavior process and update the optimal analysis plan next time based on the weight compensation after learning.

[0030] During the data replay process, if the default replay rules meet the operator's needs, the replay process will be entered directly; otherwise, the operator will manually intervene to modify the data replay rules; the system will learn based on the last operator behavior process, and update the optimal replay plan next time based on the weight compensation after learning.

[0031] The beneficial effects produced by the present invention are:

[0032] The method of the present invention learns based on the user's behavior sequence and generates corresponding fault solutions and fault locations or default setting values. The operator only needs to make appropriate adjustments and minor modifications according to needs, which greatly reduces the complexity of operation and improves the efficiency of fault problem solving and the system's replay ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0034] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0035] Figure 2 Schematic diagram of a network hierarchical structure based on user behavior sequence according to an embodiment of the present invention;

[0036] Figure 3 Schematic diagram of a deep learning architecture based on user behavior sequences according to an embodiment of the present invention;

[0037] Figure 4 This is a self-test evaluation flow chart based on deep learning according to an embodiment of the present invention;

[0038] Figure 5 This is a flowchart of self-diagnosis and repair based on deep learning according to an embodiment of the present invention;

[0039] Figure 6 This is a flowchart of admission analysis and replay based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] like Figure 1 As shown, an intelligent evaluation and replay test method based on deep learning includes the following steps:

[0042] 1) Establish a deep learning model based on user behavior sequences. Take the user behavior sequence as input and divide each behavior i in the user behavior sequence into object information item_i (IP, message type, message information unit identifier) ​​and behavior attribute property_i (scene, time, operation) to obtain a universal user operation representation;

[0043] The object information includes an IP address, a message type, and a message information unit identifier;

[0044] The behavior attributes include the scene, time and operation of the user behavior;

[0045] Taking the user behavior sequence as input, the user operation representation of each behavior is converted into a vector value, and the vector features of each behavior node are obtained, including the corresponding time, scene, operation status (query, replay, admission, operation and maintenance, testing, etc.), input source IP, destination IP, message type, message information unit identifier, and information field;

[0046] The network hierarchy based on user behavior sequence is as follows: Figure 2 As shown in the figure, the structure is divided into an input layer and an embedding layer. It takes the user behavior sequence as input, models it with different characteristic behaviors, quantitatively represents each behavior as a vector value, and completes the serial embedding in the embedding layer.

[0047] Deep learning architecture based on user behavior sequence Figure 3 As shown, i t represents the t-th input threshold, f t represents the forgetting threshold of the t-th behavior, o t represents the output threshold for the tth time, p t represents the user's t-th behavior characteristics, σ represents the model coefficient, b i 、b f 、b c , b0 represent different constants, etc. represent the weighted coefficients of each behavior, and the generated deep learning model based on user behavior sequence is as follows:

[0048]

[0049]

[0050]

[0051]

[0052] Finally, the vector weighted cumulative value h of all object representations for the tth time t It is expressed as follows:

[0053] h t =o t ·tanh(c t )

[0054] A deep learning model based on user behavior sequences is used to weight the hidden state. Different weights are assigned to the vector feature parameters, which are then XORed with the user features to obtain the vector weighted cumulative value.

[0055] This architecture learns a more general user representation through multi-task learning. The model divides each behavior in the user behavior sequence into object information and behavior attributes. Figure 1 The embedding layer gets the hidden state of each behavior node, and then uses Figure 2 The architecture shown weights the hidden state, then XORs it with the user features and outputs a vector of weighted accumulated values.

[0056] 2) Self-test evaluation and diagnosis and repair based on deep learning

[0057] Self-test evaluation and diagnosis and repair based on deep learning include self-test evaluation and self-diagnosis and repair;

[0058] 2.1) Use the self-test evaluation model to conduct self-test evaluation; Figure 4 ;

[0059] A set of test evaluation rules is set up based on prior experience as the initial test evaluation rules of the self-test evaluation model;

[0060] Periodically conduct self-testing and expand the rule base;

[0061] If the self-test evaluation model fails to properly identify the status failure, the model will start learning after manual intervention to expand the rule base and trigger mechanism;

[0062] If the self-test evaluation model correctly identifies the system / device status but fails to properly identify data transmission and reception anomalies, the model will learn after manual intervention to expand the rule base and trigger mechanism;

[0063] If the self-test evaluation model correctly identifies abnormal data transmission and reception, but fails to properly identify abnormal data parsing, the model will learn after manual intervention to expand the rule base and trigger mechanism;

[0064] 2.2) Use the self-diagnosis and repair model to perform self-diagnosis and repair; Figure 5 ;

[0065] Based on prior experience, a set of diagnosis and repair rules is established as the initial diagnosis and repair rules for the self-test evaluation model. The system has the ability to identify typical problems and repair typical, easily repairable software problems, including those that can be accurately located based on abnormal values ​​returned by functions or abnormal analysis code. The system also has the ability to propose solutions to other typical problems.

[0066] Periodically conduct self-testing and expand the rule base;

[0067] If the self-diagnosis and repair model correctly identifies a fault during testing, it enters the self-diagnosis and repair process. For typical software issues identified as easily repairable, automatic error correction is performed. For other typical issues, a rule-based optimal solution is proposed to guide fault location, shortening the fault location cycle.

[0068] If the self-diagnosis and repair model fails to identify a typical problem, the operator will manually intervene to improve the rule base based on the problem solving situation, and the self-diagnosis and repair model will start learning to expand the trigger mechanism and repair mechanism;

[0069] Use updated self-diagnosis and repair models for self-diagnosis and repair;

[0070] 2.3) Conduct admission analysis replay based on deep learning; Figure 6 ;

[0071] Initially, a set of admission, analysis, and replay rules are set based on prior experience. The default settings include typical feature values ​​such as source IP, destination IP, message sorting method, and message type, and corresponding weights are assigned to them. The system makes real-time corrections to the typical feature values ​​and their weights based on each operator's operation. The specific process is as follows:

[0072] During the data admission process, if the default recommended admission rules meet the operator's needs, the admission process will proceed directly; otherwise, the operator will manually intervene and modify the default recommended admission rules. The model will record and learn the previous operator behavior process, update the weights corresponding to each vector behavior feature parameter, and synchronously update the admission rule plan;

[0073] During the data analysis process, if the default recommended analysis rules meet the operator's needs, the analysis process will proceed directly; otherwise, the operator will manually intervene to modify the data analysis rules. The system will learn based on the previous operator behavior process and update the optimal analysis plan next time based on the weight compensation after learning.

[0074] During the data replay process, if the default replay rules meet the operator's needs, the replay process will be entered directly; otherwise, the operator will manually intervene to modify the data replay rules; the system will learn based on the last operator behavior process, and update the optimal replay plan next time based on the weight compensation after learning.

[0075] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. An intelligent assessment and replay testing method based on deep learning, comprising the following steps: 1) Establish a deep learning model based on user behavior sequences, take the user behavior sequence as input, divide each behavior i in the user behavior sequence into object information item_i and behavior attribute property_i, and obtain a general user operation representation; The object information includes an IP address, a message type, and a message information unit identifier; The behavior attributes include the scene, time and operation of the user behavior; Taking the user behavior sequence as input, the user operation representation of each behavior is converted into a vector value, and the vector features of each behavior node are obtained, including the corresponding time, scene, operation status, input source IP, destination IP, message type, message information unit identifier, and information field; A deep learning model based on user behavior sequences is used to assign different weights to vector feature parameters, which are then XORed with user features to obtain the weighted cumulative value of the vector. 2) Self-test evaluation and diagnosis and repair based on deep learning Self-test evaluation and diagnosis and repair based on deep learning include self-test evaluation and self-diagnosis and repair; 2.1) Use the self-test evaluation model to conduct self-test evaluation; A set of test evaluation rules is set up based on prior experience as the initial test evaluation rules of the self-test evaluation model; Periodically conduct self-testing and expand the rule base; If the self-test evaluation model fails to properly identify the status failure, the model will start learning after manual intervention to expand the rule base and trigger mechanism; If the self-test evaluation model correctly identifies the system / device status but fails to properly identify data transmission and reception anomalies, the model will learn after manual intervention to expand the rule base and trigger mechanism; If the self-test evaluation model correctly identifies abnormal data transmission and reception, but fails to properly identify abnormal data parsing, the model will learn after manual intervention to expand the rule base and trigger mechanism; Conduct self-test assessment using the updated self-test assessment model; 2.2) Use the self-diagnosis and repair model to perform self-diagnosis and repair; Based on prior experience, a set of diagnosis and repair rules is established as the initial diagnosis and repair rules for the self-test evaluation model. The system has the ability to identify typical problems and repair typical, easily repairable software problems, including those that can be accurately located based on abnormal values ​​returned by functions or abnormal analysis code. The system also has the ability to propose solutions to other typical problems. Periodically conduct self-testing and expand the rule base; If the self-diagnosis and repair model correctly identifies a fault during testing, it enters the self-diagnosis and repair process. For typical software issues identified as easily repairable, automatic error correction is performed. For other typical issues, a rule-based optimal solution is proposed to guide fault location, shortening the fault location cycle. If the self-diagnosis and repair model fails to identify a typical problem, the operator will manually intervene to improve the rule base based on the problem solving situation, and the self-diagnosis and repair model will start learning to expand the trigger mechanism and repair mechanism; Use the updated self-diagnosis and repair model to perform self-diagnosis and repair; 2.3) Conduct deep learning-based admission analysis replay; Initially, a set of admission, analysis, and replay rules are set based on prior experience. The default settings include typical feature values ​​such as source IP, destination IP, message sorting method, and message type, and corresponding weights are assigned to them. The system makes real-time corrections to the typical feature values ​​and their weights based on each operator's operation. The specific process is as follows: During the data admission process, if the default recommended admission rules meet the operator's needs, the admission process will proceed directly; otherwise, the operator will manually intervene and modify the default recommended admission rules. The model will record and learn the previous operator behavior process, update the weights corresponding to each vector behavior feature parameter, and synchronously update the admission rule plan; During the data analysis process, if the default recommended analysis rules meet the operator's needs, the analysis process will be directly entered; Otherwise, the operator manually intervenes to modify the data analysis rules; the system learns based on the previous operator behavior process and updates the optimal analysis plan next time based on the weight compensation after learning; During the data replay process, if the default replay rules meet the operator's needs, the replay process will be directly entered; Otherwise, the operator manually intervenes to modify the data replay rules; the system learns based on the last operator behavior process, and updates the optimal replay plan next time based on the weight compensation after learning.

2. The intelligent evaluation and replay testing method based on deep learning according to claim 1 is characterized in that: In step 1), the user behavior sequence is used as input, and the user operation representation of each behavior is converted into a vector value through a network hierarchical structure based on the user behavior sequence; The network hierarchical structure is divided into an input layer and an embedding layer. It takes the user behavior sequence as input, models it with different characteristic behaviors, and quantitatively represents each behavior as a vector value.

3. The intelligent assessment and replay test method based on deep learning according to claim 1 is characterized in that: In step 1), a deep learning model based on user behavior sequences is used to assign different weights to the vector feature parameters, which are then XORed with the user features to obtain a vector weighted cumulative value; specifically, as follows: The deep learning model based on user behavior sequence is as follows: Finally, the vector weighted cumulative value h of all object representations for the tth time t It is expressed as follows: h t =o t ·tanh(c t ); Among them, i t represents the t-th input threshold, f t Indicates the forgetting threshold of the t-th behavior, o t represents the output threshold for the tth time, p t represents the user's t-th behavior characteristics, σ represents the model coefficient, b i 、b f 、b c , b0 represent different constants, They represent the corresponding weighting coefficients respectively.

Images