Automatic construction method and device, electronic equipment and storage medium

Abstract

The application provides an automatic mechanism construction method and device, electronic equipment and storage medium. The method comprises the following steps: obtaining an input request sequence; determining a first request sequence according to a current request after deleting a corresponding byte and a previous request; determining a self-similarity score of a first response obtained according to the first request sequence and a similarity score of the first response for a second response in a response pool; if the similarity score is greater than the self-similarity score, adding the first response to an extended response pool; grouping and merging the extended response pool according to a preset threshold to determine a final classification of responses in the extended response pool; selecting a request stored in an input alphabet to form an input sequence, inputting the input sequence into a target device to receive a first sequence response, and inputting the input sequence into the blank automaton to receive a second sequence response; if a first response classification determined according to the first sequence response is the same as a second response classification determined according to the second sequence response, a target automaton is obtained.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an automatic machine construction method, apparatus, electronic device and storage medium. Background Technology

[0002] Research into acquiring prior knowledge for security schemes such as fuzzing through the construction of equivalent automata has become an important trend. In related technologies, automata are typically constructed using methods that identify the state based on the explicit features of the object under test. Specifically, the most representative explicit feature of the object under test is the protocol status code. Status codes are fields in a protocol that indicate the system state, and they are determined by the vendor during the protocol design phase. Taking the HTTP protocol as an example, the header contains HTTP status codes to respond to browser requests. Common status codes include 200 (request successful), 404 (requested resource not found), and 500 (internal server error). Therefore, by determining the system state based on the status codes corresponding to different inputs, an equivalent automaton can be constructed.

[0003] However, in the face of heterogeneous IoT systems, the location of status codes is freely determined by the designer, and not all systems or protocols have status codes. Manually defined explicit features to identify states are limited, and there is a problem of insufficient accuracy caused by the large granularity of state division. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an automatic machine construction method, apparatus, electronic device and storage medium.

[0005] In view of the above objectives, in a first aspect, this application provides an automatic machine construction method, the method comprising:

[0006] Obtain the sequence of input request messages used to trigger different functions on the target software of the target device;

[0007] The first request sequence is determined based on the current request after deleting the corresponding byte from the input request sequence and the previous request of the current request.

[0008] Determine the self-similarity score of the first response obtained based on the first request sequence and the similarity score of the first response to the second response in the response pool;

[0009] If the similarity score is greater than the self-similarity score, then the first response is added to the response pool to obtain the expanded response pool;

[0010] Feature vectors of the extended response pool are extracted to form a feature matrix. A clustering matrix is ​​generated based on the feature matrix. The extended response pool is then grouped and merged according to a preset threshold to determine the final classification of responses in the extended response pool. The final classification of responses is used to indicate the classification result of the request corresponding to each response in the extended response pool.

[0011] An input alphabet is constructed based on the input request sequence, the requests corresponding to each response in the expanded response pool that determine the classification result, and the sequence prefix;

[0012] The input alphabet is input into a pre-built blank automaton, and the requests stored in the input alphabet are selected in sequence to form an input sequence. The input sequence is then input into the target device to receive a first sequence response, and the input sequence is input into the blank automaton to receive a second sequence response.

[0013] If the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response, then the target automaton is obtained.

[0014] In one possible implementation, obtaining the sequence of input request requests for triggering different functions on the target software of the target device includes:

[0015] Iterate through the different functions on the target software corresponding to the target device;

[0016] For each function, a packet capture operation is performed at the network interface card of the target device to obtain the request sequence corresponding to each function to determine the input request sequence.

[0017] In one possible implementation, determining the first request sequence based on the current request after deleting the corresponding byte from the input request sequence and the previous request of the current request includes:

[0018] For each request in the input request sequence

[0019] Delete the first byte, and construct a first request based on the request after deleting the byte and the previous request.

[0020] Delete the second byte, and construct another first request based on the request after deleting the byte and the previous request of the first request;

[0021] The sequence of the first requests is determined based on all of the first requests.

[0022] In one possible implementation, determining the self-similarity score of the first response obtained based on the first request sequence includes:

[0023] Send the first request sequence twice to the target device to obtain a first first response and a second first response;

[0024] The first response is divided into multiple first character segments using the differential method, and the second response is divided into multiple second character segments.

[0025] Multiple first character segments and multiple second character segments are matched pairwise to determine multiple first parts, and the sum of the dissimilar bits of each first part is determined to obtain a first edit distance between the first character segments and the second character segments; wherein, each first part includes a first character segment and a second character segment that matches the first character segment;

[0026] The self-similarity score of the first response is determined based on the first string length of the first response, the second string length of the second response, and the first edit distance.

[0027] In one possible implementation, the response pool includes: a pre-acquired second request sequence;

[0028] Determining the similarity score of the first response to the second response in the response pool includes:

[0029] Send the first request sequence to the target device to obtain the first response, and send the second request sequence from the response pool to the target device to obtain the second response;

[0030] The first response is divided into multiple third character segments using a differential method, and the second response is divided into multiple fourth character segments.

[0031] Multiple third character segments and multiple fourth character segments are matched pairwise to determine multiple second parts, and the sum of the disjoint bits of each second part is determined to obtain a second edit distance between the third character segments and the fourth character segments; wherein each second part includes a third character segment and a fourth character segment that matches the third character segment;

[0032] The similarity score of the first response to the second response in the response pool is determined based on the third string length of the first response, the fourth string length of the second response, and the second edit distance.

[0033] In one possible implementation, the step of extracting feature vectors from the extended response pool to construct a feature matrix based on the feature vectors, generating a clustering matrix based on the feature matrix, and grouping and merging the extended response pool according to a preset threshold to determine the final classification of responses in the extended response pool includes:

[0034] Extract the target attributes of each response in the extended response pool to determine the feature vector; wherein, the target attributes include: the conversion frequency of character bytes to other byte types, the conversion frequency of numeric bytes to other byte types, the conversion frequency of space bytes to other byte types, the conversion frequency of special symbol bytes to other byte types, and the response length;

[0035] The feature matrix is ​​constructed based on all the feature vectors in the extended response pool;

[0036] Each target attribute in the feature vector is used as a location point in five-dimensional space, and the Euclidean distance between any two location points is determined accordingly.

[0037] Starting from any point, perform a merging operation according to the shortest Euclidean distance to obtain clusters, until the number of clusters reaches the preset threshold to generate the clustering matrix;

[0038] The classification information of the extended response pool corresponding to each cluster obtained in each merging operation before the number of clusters reaches the preset threshold is tracked.

[0039] The final response classification is determined based on all classification information to indicate the classification result of the request corresponding to each response in the extended response pool.

[0040] In one possible implementation,

[0041] The step of inputting the input alphabet into a pre-constructed blank automaton, sequentially selecting requests stored in the input alphabet to form an input sequence, inputting the input sequence into the target device to receive a first sequence response, and inputting the input sequence into the blank automaton to receive a second sequence response includes:

[0042] The input alphabet is input into a pre-constructed blank automaton. The length of the input alphabet is n. The requests stored in the input alphabet are selected in turn to combine to obtain an input sequence of length 1 to n.

[0043] The input sequence is input to the target device and the blank automaton respectively to receive the first sequence response and the second sequence response.

[0044] In a second aspect, this application provides an automated machine construction apparatus, the apparatus comprising:

[0045] The acquisition module is configured to acquire a sequence of input request sequences used to trigger different functions on the target software of the target device;

[0046] The first determining module is configured to determine the first request sequence based on the current request after deleting the corresponding byte from the input request sequence and the previous request of the current request.

[0047] The second determining module is configured to determine the self-similarity score of the first response obtained based on the first request sequence and the similarity score of the first response to the second response in the response pool.

[0048] An expansion module is configured to add the first response to the response pool to obtain an expanded response pool if the similarity score is greater than the self-similarity score.

[0049] The third determining module is configured to extract feature vectors from the extended response pool to construct a feature matrix based on the feature vectors, generate a clustering matrix based on the feature matrix, and group and merge the extended response pool according to a preset threshold to determine the final classification of responses in the extended response pool; wherein, the final classification of responses is used to indicate the classification result of the requests corresponding to each response in the extended response pool;

[0050] The fourth determining module is configured to construct an input alphabet based on the input request sequence, the requests corresponding to each response in the extended response pool that determine the classification results, and the sequence prefix;

[0051] The fifth determining module is configured to input the input alphabet into a pre-constructed blank automaton, sequentially select requests stored in the input alphabet to form an input sequence, input the input sequence into the target device to receive a first sequence response, and input the input sequence into the blank automaton to receive a second sequence response;

[0052] The building module is configured to obtain a target automaton if the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response.

[0053] In a third aspect, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the automatic machine construction method as described in the first aspect.

[0054] In a fourth aspect, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the automatic machine construction method as described in the first aspect.

[0055] As can be seen from the above description, this application provides an automatic mechanism construction method, apparatus, electronic device, and storage medium, which acquires an input request sequence for triggering different functions on the target software of a target device; determines a first request sequence based on the current request after deleting corresponding bytes from the input request sequence and the previous request of the current request; determines the self-similarity score of a first response obtained from the first request sequence and the similarity score of the first response to a second response in the response pool; if the similarity score is greater than the self-similarity score, adds the first response to the response pool to obtain an expanded response pool; extracts feature vectors from the expanded response pool to construct a feature matrix based on the feature vectors, generates a clustering matrix based on the feature matrix, and applies a preset threshold to the expanded response pool. The extended response pool is grouped and merged to determine the final classification of responses in the extended response pool; wherein, the final response classification is used to indicate the classification result of the request corresponding to each response in the extended response pool; an input alphabet is constructed based on the input request sequence, the requests corresponding to each response in the extended response pool whose classification results are determined, and the sequence prefix; the input alphabet is input to a pre-constructed blank automaton, and the requests stored in the input alphabet are selected in sequence to form an input sequence, and the input sequence is input to the target device to receive a first sequence response, and the input sequence is input to the blank automaton to receive a second sequence response; if the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response, a target automaton is obtained. This application acquires information through interaction, analyzes the responses generated by the interaction, and constructs equivalent automata to provide the system with universality for target devices with different architectures. In the active automaton learning of effective sequence exploration and improvement, hierarchical clustering technology is introduced to eliminate the randomness of the response and improve the accuracy of the system for responses containing random content. Furthermore, by accurately identifying the state of the target device, the system provides the precision of state differentiation, which can fully mine and identify the device state hidden deep within, thereby improving the coverage of fuzz testing. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 An exemplary flowchart of an automated machine construction method provided in an embodiment of this application is shown.

[0058] Figure 2 An exemplary schematic diagram of an application scenario according to an embodiment of this application is shown.

[0059] Figure 3 An exemplary structural diagram of an interactive system in an application scenario according to an embodiment of this application is shown.

[0060] Figure 4 An exemplary schematic diagram of the processing flow according to an embodiment of this application is shown.

[0061] Figure 5 An exemplary schematic diagram of the algorithmic pseudocode for a coarse classification process according to an embodiment of this application is shown.

[0062] Figure 6 An exemplary schematic diagram of the algorithmic pseudocode for the fine classification process according to an embodiment of this application is shown.

[0063] Figure 7 An exemplary structural diagram of a communication architecture according to an embodiment of this application is shown.

[0064] Figure 8 An exemplary schematic diagram of a dot-line graph of a target automaton generated by software according to an embodiment of this application is shown.

[0065] Figure 9 An exemplary structural diagram of an automated machine construction device provided in an embodiment of this application is shown.

[0066] Figure 10 This illustration shows an exemplary structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0068] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0069] As mentioned in the background section, research on acquiring prior knowledge of security schemes such as fuzzing through the construction of equivalent automata has become an important trend. In related technologies, automata are typically constructed using methods that identify the state based on the explicit features of the object under test. Specifically, the most representative explicit feature of the object under test is the protocol status code. A status code is a field in a protocol that indicates the system state, and it is determined by the vendor during the protocol design phase. Taking the HTTP protocol as an example, the header contains HTTP status codes to respond to browser requests. Common status codes include 200 (request successful), 404 (requested resource not found), and 500 (internal server error). Therefore, by determining the system state based on the status code corresponding to the response field when sending different inputs, an equivalent automaton can be constructed.

[0070] During the research process, the applicant discovered that methods based on explicit features of the object under test to identify its state have low overhead and fast construction speed. However, manually defined explicit features can only identify a limited number of states, resulting in coarse state granularity and insufficient accuracy; the position of the state code is freely determined by the designer, and not all systems or protocols have state codes. Therefore, this method is not applicable when the target device is a heterogeneous IoT system.

[0071] Other implementation methods exist in related technologies. For example, methods for learning features from the interaction information of the test object can be divided into methods for identifying the state based on global features and methods for identifying the state based on module features, depending on the source of the learned features. Methods for identifying the state based on global features often employ active automata learning. Active automata learning constructs test cases based on an alphabet and, through active interaction with the black-box system, obtains the minimum complete automaton of the target system in polynomial time. Active automata learning is well-suited for various stochastic environments, learning optimal values ​​through continuous interaction with unknown environments. Regarding features, since the learning utilizes system responses and the learning process is system-wide, the learned features are global features. In related technologies, in Bluetooth communication scenarios with known protocol state sets, and in schemes that use protocol interaction results as the basis for state judgment, the method utilizes global features learned from the interaction information of the test object to identify the state.

[0072] Methods for identifying state based on module features also rely on self-learning of features, but these learned features often pertain to modules that achieve a specific purpose. Considering the characteristics of module-feature-oriented approaches, these schemes typically construct automata targeting a specific component or function of the system and learn those features through a custom scheme, reducing performance overhead while increasing effectiveness for specific parts. For example, feature learning based on the CAN protocol format and schemes for building prior knowledge based on arbitrary protocol input message formats utilize interactive information from the object under test to learn module features and identify state.

[0073] The applicant's research revealed that methods for learning features and identifying states based on the interaction information of the test object require explicit protocol guidance during the interaction process; otherwise, relying solely on sampling the suffixes of the interaction results to distinguish states makes accurate and comprehensive state identification difficult. In IoT scenarios, protocol specifications vary between IoT manufacturers, and even between different devices from the same manufacturer, and are mostly not publicly available, making it impractical to obtain corresponding protocol information for each device. Furthermore, their active automata learning scheme does not consider random content, which has significant limitations in IoT scenarios.

[0074] Therefore, in the face of heterogeneous IoT systems, the location of status codes is freely determined by the designer, and not all systems or protocols have status codes. Manually defined explicit features to identify states are limited, and there is a problem of insufficient accuracy due to the large granularity of state division.

[0075] Therefore, this application provides an automatic mechanism construction method, apparatus, electronic device, and storage medium, which acquires an input request sequence for triggering different functions on target software of a target device; determines a first request sequence based on the current request after deleting corresponding bytes from the input request sequence and the previous request of the current request; determines the self-similarity score of a first response obtained from the first request sequence and the similarity score of the first response to a second response in a response pool; if the similarity score is greater than the self-similarity score, adds the first response to the response pool to obtain an expanded response pool; extracts feature vectors from the expanded response pool to construct a feature matrix based on the feature vectors, generates a clustering matrix based on the feature matrix, and applies a preset threshold to the expanded response pool. The response pool is grouped and merged to determine the final classification of responses in the extended response pool; wherein, the final response classification is used to indicate the classification result of the request corresponding to each response in the extended response pool; an input alphabet is constructed based on the input request sequence, the requests corresponding to each response in the extended response pool whose classification results are determined, and the sequence prefix; the input alphabet is input to a pre-constructed blank automaton, and the requests stored in the input alphabet are selected in sequence to form an input sequence, and the input sequence is input to the target device to receive a first sequence response, and the input sequence is input to the blank automaton to receive a second sequence response; if the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response, a target automaton is obtained. This application acquires information through interaction, analyzes the responses generated by the interaction, and constructs equivalent automata to provide the system with universality for target devices with different architectures. In the active automaton learning of effective sequence exploration and improvement, hierarchical clustering technology is introduced to eliminate the randomness of the response and improve the accuracy of the system for responses containing random content. Furthermore, by accurately identifying the state of the target device, the system provides the precision of state differentiation, which can fully mine and identify the device state hidden deep within, thereby improving the coverage of fuzz testing.

[0076] The automatic mechanism construction method provided in this application will be specifically described below through specific embodiments.

[0077] Figure 1 An exemplary flowchart of an automated machine construction method provided in an embodiment of this application is shown.

[0078] refer to Figure 1 The automatic machine construction method provided in this application embodiment specifically includes the following steps:

[0079] S102: Obtain the sequence of input request requests used to trigger different functions on the target software of the target device.

[0080] S104: Determine the first request sequence based on the current request after deleting the corresponding byte from the input request sequence and the previous request of the current request.

[0081] S106: Determine the self-similarity score of the first response obtained based on the first request sequence and the similarity score of the first response to the second response in the response pool.

[0082] S108: If the similarity score is greater than the self-similarity score, then the first response is added to the response pool to obtain the expanded response pool.

[0083] S110: Extract feature vectors from the extended response pool to construct a feature matrix based on the feature vectors, generate a clustering matrix based on the feature matrix, and group and merge the extended response pool according to a preset threshold to determine the final classification of responses in the extended response pool; wherein, the final classification of responses is used to indicate the classification result of the requests corresponding to each response in the extended response pool.

[0084] S112: An input alphabet is constructed based on the input request sequence, the requests corresponding to each response in the extended response pool that determine the classification results, and the sequence prefix.

[0085] S114: Input the input alphabet to a pre-built blank automaton, sequentially select the requests stored in the input alphabet to form an input sequence, input the input sequence to the target device to receive a first sequence response, and input the input sequence to the blank automaton to receive a second sequence response.

[0086] S116: If the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response, then the target automaton is obtained.

[0087] Figure 2 An exemplary schematic diagram of an application scenario according to an embodiment of this application is shown.

[0088] In some embodiments, reference Figure 2 The target device can be an IoT device (also known as an Internet of Things device), such as a shared desk lamp, a shared power bank, and... Figure 2 The electronic components shown can be controlled by IoT devices within a local area network through their accompanying target software (i.e., APP). This target software can then be connected to a gateway and interact with the IoT cloud platform.

[0089] Figure 3 An exemplary structural diagram of an interactive system in an application scenario according to an embodiment of this application is shown.

[0090] In some embodiments, reference Figure 3 The method described in this application can obtain corresponding request sequences and responses by interacting with IoT devices. Specifically, it can iterate through different functions on the target software corresponding to the target device. For each function, packet capture can be performed at the network interface card (NIC) of the target device to obtain the request sequence corresponding to each function. The input request sequence is then obtained based on the request sequence corresponding to each function. For example, this application uses certain interactions and packet captures as preliminary work, which can be done with manual intervention. The purpose of this stage is to obtain as many request sequences as possible that trigger different functions of the IoT device. In specific implementation, the interaction module can be used to interact with the IoT device. At the initialization module, functions on the IoT companion APP can be manually selected, and packet capture can be performed at the NIC to obtain and record the corresponding sequences as prior sequences for subsequent work. When manually selecting functions on the IoT companion APP, all function options should be iterated as much as possible. When the acquisition of input request sequences is insufficient, although it will not affect the usability of this application, it will make it difficult to explore certain states and result in insufficient establishment of equivalent automata states.

[0091] Figure 4 An exemplary schematic diagram of the processing flow according to an embodiment of this application is shown.

[0092] Furthermore, a classification module can be used to perform coarse and fine classification, categorizing responses containing random content, thus guiding subsequent exploration of valid inputs and the establishment of equivalent automata. (Reference) Figure 4 The process involves determining whether each request in the input request sequence has undergone coarse classification. If not, for each request in the input request sequence, the first byte is deleted, and a first request is constructed based on the request after deleting the first byte and its preceding request. Further, the second byte is deleted, and another first request is constructed based on the request after deleting the second byte and its preceding request. Finally, the first request sequence is determined based on all first requests. Specifically, this stage uses byte deletion technology, randomly selecting a priori sequence and choosing a message as the processing target. For each processing target, each byte is deleted sequentially, and the modified message and the preceding sequence are sent to attempt to trigger a system response.

[0093] For example, the input request sequence includes request 1 and request 2, where request 1 is the preceding request to request 2. Regarding request 2, if request 2 contains bytes ABC, then first delete byte A from request 2, leaving bytes BC. Then, the request 2' obtained after the deletion operation is combined with the preceding request (i.e., request 1) to construct the first request, resulting in first request X. Further, delete byte B from request 2', leaving byte C. Then, the request 2" obtained after the deletion operation is combined with the preceding request (i.e., request 1) to construct the first request, resulting in first request Y. Further still, delete byte C from request 2" and then the request 2"' obtained after the deletion operation is combined with the preceding request (i.e., request 1) to construct the first request, resulting in first request Z. This process continues until all the first requests determined based on each request in the input request sequence are obtained, thus determining the first request sequence.

[0094] Figure 5 An exemplary schematic diagram of the algorithmic pseudocode for a coarse classification process according to an embodiment of this application is shown.

[0095] In some embodiments, reference Figure 5 The coarse classification algorithm takes each individual request in the initial input sequence as input and outputs the updated response pool. For the requests selected in the initial input sequence, the coarse classification algorithm performs the following operations:

[0096] Step 1: Select each byte of each request in the input sequence in sequence, delete the corresponding byte of the request, and form a new sequence with the preceding request of the input sequence for that request;

[0097] Step 2: Send the new sequence twice and use the difference method to calculate the self-similarity score of the two responses.

[0098] The calculation of self-similarity score examines the minimum number of operations required to transform the strings to be detected. Based on the difference method, the two strings are divided into several character segments using the LCS algorithm. Then, these character segments are matched, and the sum of the dissimilar bits in each part is used as the edit distance between the two strings. The final similarity score is calculated based on the edit distance and the length of the longest string using the following formula.

[0099] Step 3: Calculate the similarity score between the response obtained from the new sequence and the response in the response pool. The similarity score calculation algorithm is the same as the self-similarity score calculation algorithm, only the input of the algorithm needs to be updated.

[0100] Step four: Compare the similarity score with the self-similarity score calculated in step two.

[0101] In some embodiments, for step two, the response information recorded in the response pool can be polled for the triggered system response, and differential operations can be performed sequentially to determine whether the same response already exists. Specifically, either of the first requests in the first request sequence can be sent to the target device to obtain the first response and the second response. Then, the first response is divided into multiple first character segments and the second response is divided into multiple second character segments according to the differential algorithm. Further, the multiple first character segments and multiple second character segments are matched pairwise to determine multiple first parts, and the sum of the dissimilar bits of each first part is determined to obtain the first edit distance between the first character segments and the second character segments. Each first part includes a first character segment and a second character segment that matches the first character segment. For example, the first response can be divided into 3 first character segments, namely A1B1C1, and the second response can be divided into 3 second character segments, namely A2B2C2. For example, the matching result is that the first part includes: A1A2, B1B2, and C1C2.

[0102] Furthermore, the calculation of the self-similarity score examines the minimum number of operations required to transform the strings to be detected. Using the difference method and the LCS algorithm, the two strings are divided into several character segments. These segments are then matched, and the sum of the dissimilarity bits in each segment is used as the edit distance between the two strings. The self-similarity score of the first response is determined based on the length of the first string in the first response, the length of the second string in the second response, and the first edit distance. The specific formula is as follows:

[0103] similarity_score=(1-ED / max(len(str1),len(str2)))*100

[0104] Where ED represents the first edit distance, len(str1) represents the length of the first string, and len(str2) represents the length of the second string.

[0105] Furthermore, in step three, a similarity score can be determined between the first response corresponding to each request in the first request sequence and the second response in the response pool. The response pool may include a pre-acquired second request sequence. Then, any one of the first requests in the first request sequence is sent to the target device to obtain a first response, and any one of the second requests in the second request sequence from the response pool is sent to the target device to obtain a second response. The first response is then divided into multiple third character segments according to a differential algorithm, and the second response is divided into multiple fourth character segments. The third character segments can be the same as the first or second character segments in the above embodiment, or they can be obtained by dividing a new first response after inputting it into the target device. Further, the multiple third character segments and multiple fourth character segments are paired to determine multiple second parts, and the sum of the dissimilarity bits of each second part is determined to obtain a second edit distance between the third and fourth character segments. Each second part includes a third character segment and a fourth character segment that matches the third character segment. For example, the first response can be divided into three first character segments, namely A1B1C1, and the second response can be divided into three second character segments, namely A3B3C3. For example, the matching result includes the second part: A1A3, B1B3, and C1C3.

[0106] Furthermore, the formula used to calculate the self-similarity score in the above embodiments can be used to replace the first edit distance with the second edit distance and the second string length with the third string length of the second response pair, thereby obtaining the similarity score.

[0107] It should be noted that the self-similarity score of each first response and the similarity score of each first response to each second response can be calculated using the above embodiments.

[0108] In some embodiments, if the similarity score is greater than the self-similarity score, the first response is added to the response pool to obtain an expanded response pool. For example, if the similarity between the response generated by the new sequence and all responses in the response pool is greater than the self-similarity, no further examination is conducted, the new response is considered unique, the new sequence is recorded in the input pool, and the new response is expanded into the response pool. The PI array records the corresponding classification number.

[0109] Furthermore, if the self-similarity score is greater than the similarity score, it is considered that the similarity distance between the selected response in the response pool and the new sequence response is less than the similarity distance caused by the random value in the two responses of the new sequence. Since the similarity distance caused by the random number is definitely less than the similarity distance caused by different message content, the new response and the corresponding response in the response pool can be classified into the same category at this time. There is no need to expand the response pool. The response classification number can be recorded at the index corresponding to the deleted byte position in the PI array maintained in the data structure of the request selected in the first step.

[0110] In some embodiments, reference Figure 4 The coarse classification algorithm can be executed in the classification module and can be executed repeatedly. That is, the expanded response pool will be continuously updated until each request and its corresponding response in the input request sequence has been coarsely classified by the coarse classification algorithm. Then, it can be input into the exploration module to perform further classification by the fine classification algorithm.

[0111] Figure 6 An exemplary schematic diagram of the algorithmic pseudocode for the fine classification process according to an embodiment of this application is shown.

[0112] refer to Figure 6 The fine-classification algorithm performs the following operations:

[0113] Step 1: Extract feature vectors from the response pool;

[0114] Step 2: Construct a feature matrix from all feature vectors in the response pool, build a clustering process matrix in a hierarchical clustering manner, and select the clustering termination point by considering a predefined threshold.

[0115] Step 3: Considering the grouping information maintained in the data structure of the input pool after coarse classification, the constructed clustering process matrix is ​​used to merge the groups by tracking the group merging process before reaching the threshold, so that the merged grouping information covers the original groups and updates the input pool data structure.

[0116] Specifically, for step one, the target attributes of each response in the extended response pool can be extracted to determine the feature vector. The target attributes include: the frequency of character bytes converting to other byte types, the frequency of numeric bytes converting to other byte types, the frequency of space bytes converting to other byte types, the frequency of special symbol bytes converting to other byte types, and the response length. The feature vector, aiming to smooth out random content in the response as much as possible, selects relevant attributes from the response to form the following quintuple:

[0117] (CaTr,DiTr,BlTr,OTr,Len)

[0118] Among them, CaTr represents the frequency of character bytes being converted to other byte types, DiTr represents the frequency of numeric bytes being converted to other byte types, BlTr represents the frequency of space bytes being converted to other byte types, OTr represents the frequency of special symbol bytes being converted to other byte types, and Len represents the response length.

[0119] It should be noted that, for character bytes, other byte types are numeric bytes, space bytes, and special symbol bytes; for numeric bytes, other byte types are character bytes, space bytes, and special symbol bytes; for space bytes, other byte types are character bytes, numeric bytes, and special symbol bytes; for special symbol bytes, other byte types are character bytes, numeric bytes, and space bytes.

[0120] For step two, a feature matrix can be constructed based on all feature vectors in the extended response pool. Each target attribute in the feature vector is used as a location point in five-dimensional space, and the Euclidean distance between any two location points is determined accordingly. Starting from any location point, a merging operation is performed according to the shortest Euclidean distance to obtain clusters, until the number of clusters reaches a preset threshold to generate a clustering matrix. Specifically, hierarchical clustering uses the average distance algorithm with Euclidean distance. Specifically, the algorithm uses the quintuples constructed in step one as location points in five-dimensional space and calculates the Euclidean distance between any two points sequentially:

[0121]

[0122] Subsequently, the algorithm merges single points into clusters based on the shortest distance. When performing cluster operations, the distance between clusters is defined as the average distance between all members.

[0123] The hierarchical aggregation algorithm can assume there are 6 sample points {A, B, C, D, E, F}. The first step is to assume each sample point belongs to a cluster ABCDEF, calculate the similarity between each cluster, and obtain the similarity matrix.

[0124] Step 2: If B and C have the highest similarity, merge clusters B and C into one cluster. Now there are five clusters: A, BC, D, E, and F.

[0125] Step 3: Update the similarity matrix between clusters. The similarity matrix has a size of 5 rows and 5 columns. If clusters BC and D have the highest similarity, merge clusters BC and D into one cluster. Now there are four clusters: A, BCD, E, and F.

[0126] Step 4: Update the similarity matrix between clusters. The similarity matrix has a size of 4 rows and 4 columns. If clusters E and F have the highest similarity, merge clusters E and F into one cluster. Now there are 3 clusters: A, BCD, and EF.

[0127] Step 5: Repeat step 4. Clusters BCD and EF have the highest similarity, so merge these two clusters. Now there are 2 clusters left, namely A and BCDEF.

[0128] Step 6: Finally, merge clusters A and BCDEF into one cluster, and the hierarchical clustering algorithm ends.

[0129] It should be noted that the number of clusters remaining can be determined based on a preset threshold.

[0130] Furthermore, the classification information of the expanded response pool corresponding to each cluster obtained from each merging operation before the cluster number reaches a preset threshold can be tracked. The final grouping information is determined based on all grouping information, and the classification number of the response corresponding to each request is determined based on the final classification information. Specifically, the grouping information maintained in the data structure maintained by the input pool after coarse classification can be considered. Using the constructed clustering process matrix, the groups are merged by tracking the group merging process before reaching the threshold, and the merged grouping information overwrites the original groups, updating the input pool data structure.

[0131] It should be noted that identical entries can be identified and divided according to a preset threshold. For identical entries, only one is retained in the response pool, and the corresponding input and preceding sequence are deleted from the input pool to reduce the overhead of subsequent stages.

[0132] In some embodiments, reference Figure 3 The equivalent automaton generation part is supported by the automaton learning module and the effective input feedback module. This part realizes the generation of equivalent automata starting from the effective input table. It includes active automaton learning with improved response analysis supported by the active automaton learning module and effective input and state merging supported by the effective input feedback module. The active automaton learning module implements an improved active automaton learning process. The module takes the effective input table as input. Specifically, the effective input table is first visualized, and protocol messages are constructed according to the contents of the effective input table. When determining the system state, the visualized protocol message is sent and a specific response is obtained. Secondly, for the specific response, due to the randomness of IoT device responses, the module uses hierarchical clustering to determine which category the response belongs to, and abstracts it into a symbol, which is input into the active automaton learning method to construct the equivalent model.

[0133] The active automaton learning phase is based on traditional active automaton learning, but improves upon the problem that the output containing random numbers generated by random systems is difficult to utilize. Specifically, this application adds an output interface to the traditional active automaton learning interaction process, abstracts the interaction response, that is, extracts features from the response and responses in the response pool, and then performs hierarchical clustering, using the abstract response with random numbers removed as the basis for learning. New responses explored in this process will further update the response pool and input pool.

[0134] Specifically, the active automaton learns by taking an input alphabet as input and the constructed equivalent automaton as output, and then learning according to the improved L* algorithm to determine the final equivalent automaton. For example, the improved L* algorithm initially builds an empty automaton to represent the target IoT system. The algorithm continuously interacts with the IoT device based on the input to be examined, obtaining the device's response. In each examination, content that cannot be closed is added to the observation table and examined again; the equivalent parts are used to improve the constructed automaton. This process is repeated until the constructed automaton is equivalent to the IoT device. The input alphabet can be set to a length of n, and requests stored in the input alphabet are selected sequentially, forming an input sequence of length 1 to n, which is examined sequentially in each round.

[0135] Further, for the sequence to be examined in the general approach, requests in the sequence are sent sequentially to the IoT device, first sequence responses are received, and the extended response pool and corresponding final response classification are copied. For each first sequence response, it is added to the copied extended response pool, and a feature matrix of the extended response pool is constructed according to the method in the aforementioned embodiments. A clustering matrix is ​​generated based on the feature matrix, and the extended response pool is grouped and merged according to a preset threshold (e.g., 1). Specifically, the target attributes of each response in the extended response pool are extracted to determine the feature vector; wherein, the target attributes include: the conversion frequency of character bytes to other types of bytes, the conversion frequency of numeric bytes to other types of bytes, the conversion frequency of space bytes to other types of bytes, the conversion frequency of special symbol bytes to other types of bytes, and the response length; a feature matrix is ​​constructed based on all feature vectors in the extended response pool; each target attribute in the feature vector is used as a position point in five-dimensional space, and the Euclidean distance between every two position points is determined accordingly; starting from any position point, a merging operation is performed according to the shortest Euclidean distance to obtain a cluster, and finally the cluster with the closest distance to the first sequence response is found, and the classification number of the cluster is recorded.

[0136] Until the examination of the first sequence response for each request in the input sequence is completed, each response is abstracted into a cluster classification number, which serves as the abstraction of the first sequence response.

[0137] The requests in the input sequence are simultaneously fed into the blank automaton to receive the second sequence response and determine the abstraction of the second sequence response. If the abstraction of the first sequence response is the same as the abstraction of the second sequence response, then a target automaton equivalent to the target device (e.g., an IoT device) is obtained.

[0138] If the requests in the input sequence are simultaneously fed into a blank automaton, assuming the automaton does not yet have a corresponding processing function (i.e., the output is empty), then the corresponding information is added to the constructed automaton.

[0139] Furthermore, the effective input feedback module reduces the exploration space and merges states. Specifically, when the active automaton learning module encounters a nondeterministic anomaly—that is, when the same input in the same state has different abstract symbols for the output—this module handles the situation. On one hand, it merges the outputs represented by the two different symbols and their corresponding effective inputs, selecting one to retain and removing the rest, thus updating the effective input table. On the other hand, it terminates the previous round of equivalent automaton construction and re-learns the equivalent automaton to reduce the state space and improve the effectiveness of the equivalent automaton.

[0140] It should be noted that the non-deterministic anomaly is caused by responses with the same state and input being abstracted and clustered into different response pool contents. Therefore, these two abstractions can be considered the same. At this point, the two are merged in this stage, the response pool and input are updated, and active automaton learning is repeated until the equivalent automaton of the IoT system is generated.

[0141] The automaton construction method proposed in this application can effectively filter random content in IoT systems. The responses of random systems often contain random elements such as timestamps and checksums. Traditional differential analysis identification techniques struggle to filter this random content. On one hand, the random content results in an excessively small classification set, failing to effectively guide the construction of equivalent automata. On the other hand, the states learned by active automata are based on interaction differences; random values ​​lead to different interaction results, thus causing classification errors. Therefore, this application effectively filters random content in IoT systems by introducing string feature extraction and hierarchical clustering techniques. In terms of effectiveness, this application increases the tolerance for random values ​​in responses during classification from 0 in traditional schemes to f-level hierarchical clustering (threshold), effectively guiding the identification of IoT system states.

[0142] Furthermore, accurately and comprehensively defining the state of an IoT system is crucial. In terms of accuracy, effectively filtering out random content within the IoT system and precisely determining the boundaries of these state divisions are often critical issues in constructing equivalent automata for related systems. Traditional equivalent automata learning techniques can divide non-random system states based on system responses to construct equivalent automata. However, for stochastic systems like IoT devices, the randomness of the responses leads to overly fine-grained differential state divisions, resulting in an excessively large and meaningless state space for each state. This application, based on traditional active automata learning schemes, introduces hierarchical clustering to abstract intermediate automata responses and a state merging scheme based on nondeterministic anomalies. Multiple effective classifications of responses are used to narrow the state space, define state boundaries, and accurately define the IoT system state.

[0143] Furthermore, regarding comprehensiveness, traditional research on equivalent automata learning and hierarchical clustering is often based on given inputs or known prior knowledge, which often results in incomplete outcomes and fails to fully represent the target system. Therefore, this application leverages the advantages of automated state recognition and effective filtering of random content in IoT systems through prior interaction to automatically explore effective inputs, ensuring the comprehensiveness of the constructed equivalent automata.

[0144] In terms of effectiveness, the equivalent automaton established in this application 1) uses states as nodes. These states are abstract merged states, meaning adjacent states unrelated to interaction are merged. 2) uses valid inputs as transition edges. Valid inputs represent valid inputs; that is, inputs producing the same output are recorded, while other inputs are simply marked. 3) It possesses good completeness. The valid input table is expanded before the automaton learns, thus the resulting equivalent automaton can better describe the target system.

[0145] Figure 7 An exemplary structural diagram of a communication architecture according to an embodiment of this application is shown.

[0146] refer to Figure 7 According to the IoT device communication architecture described in the attached figures, this embodiment, in addition to the original IoTEac solution, adds a Miio interface module between the interaction interface and the IoT device based on the characteristics of the Miio protocol and provides a corresponding interface. Furthermore, communication is implemented within a local area network and targets port 54321, which is the fixed UDP packet receiving port in the Xiaomi IoT device, to reduce the overhead of the general solution. Therefore, this embodiment can be said to be specifically for the development and access of the corresponding platform interface module. The development platform is a Linux environment with an x86_64 instruction set architecture, and the system kernel is Linux ubuntu 5.15.0-56-generic.

[0147] The Miio interface module refactors the Miio communication portion. Miio is a local area network communication interface provided by Xiaomi, which encapsulates simple control commands or raw command transmission for Xiaomi IoT devices.

[0148] The Miio module is developed based on this interface. Due to its high level of encapsulation, the input is filtered, and the output directly returns a status code or throws an exception, making it difficult to easily obtain the required response. Therefore, it has been refactored.

[0149] Regarding the reconstruction, firstly, the original communication component filters abnormal data values ​​and cannot send arbitrary raw input; furthermore, the component automatically interprets response content, making it difficult to obtain the original response. Therefore, this embodiment implements the software's communication module based on the Devices class and the private _Protocol class, extracting the input filtering part of the original tool and directly returning the raw output, thus conforming to the protocol communication requirements of this software.

[0150] Secondly, the original component stores protocol parameters in JSON, and includes JSON format verification before encryption, making it difficult to send unformatted strings. Therefore, this embodiment reconstructs the RC4 encryption method, removing the JSON format verification and decoding-related parts, and achieving direct encryption of string types by specifying the protocol structure format.

[0151] Furthermore, Xiaomi IoT devices require a private token for local area network authentication. Obtaining this token requires access to the Mi Home platform. Therefore, the software was developed and integrated with automated login, token, and IP acquisition methods to automate software operation.

[0152] Furthermore, this application can be used for admission assessment. The automated information acquisition of IoT devices is expected to accurately obtain the target IoT device's IP, Token, ID, MAC address, kernel model, and other information. The test request sequence used for testing can include the content of the protocol value field obtained after packet capture and decrypted using RC4. The IoT device response includes random output, i.e., two consecutive message transmissions with a self-similarity score below 100. Traditional equivalent automata models in related technologies cannot achieve a self-similarity score below 100 for two consecutive message transmissions, indicating that traditional equivalent automata models cannot be applied to this scenario. Based on coarse and fine classification algorithms, the classification results of the original input and exploration input can be obtained. The order of each array in the object represents the exploration message whose corresponding byte was deleted, and the PI content represents the classification result of that message. One of the important criteria for selecting subsequent valid inputs is that the classification points to a different class of response.

[0153] Figure 8An exemplary schematic diagram of a dot-line graph of a target automaton generated by software according to an embodiment of this application is shown.

[0154] refer to Figure 8 The software is expected to accurately generate the equivalent automaton of the target IoT device and represent it as a dot-line graph. The representation includes state transitions and input conditions.

[0155] For the purposes of this application, an effective input exploration technique based on clustering feedback is proposed. This technique consists of two parts: an initial classification exploration and a subsequent determination of effective input based on classification.

[0156] The initial classification exploration utilizes a two-stage classification algorithm. Given a limited initial message sequence, a certain number of new inputs are constructed using a byte deletion scheme. Subsequently, feature values ​​are extracted from the corresponding random responses to construct a feature vector matrix and perform hierarchical clustering. Finally, the fusion process is terminated by a threshold to obtain intermediate classification results.

[0157] In the later stage, based on classification, valid input is determined for all message sequences. Similarly, a byte deletion scheme is used to construct as many variant inputs as possible. The initial classification is used as feedback to quickly calculate the distance with each classification, thereby quickly and accurately determining whether the input is valid and thus determining the valid input table.

[0158] This application proposes an abstract-based active automaton learning technique for IoT systems. This technique employs an abstract approach, concretely representing the input alphabet as protocol content for interaction; and abstracting the interaction results through hierarchical clustering of response classification results. Thus, active automaton learning is achieved with IoT devices as the target.

[0159] This application proposes an input merging technique based on nondeterministic anomalies. This technique is used to merge valid input tables and equivalent automaton states generated by active automaton learning, thereby improving the accuracy and efficiency of active automaton learning techniques for IoT systems. It merges abstractions in active automaton learning based on nondeterministic anomalies, updates the response pool and inputs / outputs, and performs secondary classification of the input and response pool contents, thus optimizing active automaton learning.

[0160] As can be seen from the above description, this application provides an automatic mechanism construction method, apparatus, electronic device, and storage medium, which acquires an input request sequence for triggering different functions on the target software of a target device; determines a first request sequence based on the current request after deleting corresponding bytes from the input request sequence and the previous request of the current request; determines the self-similarity score of a first response obtained from the first request sequence and the similarity score of the first response to a second response in the response pool; if the similarity score is greater than the self-similarity score, adds the first response to the response pool to obtain an expanded response pool; extracts feature vectors from the expanded response pool to construct a feature matrix based on the feature vectors, generates a clustering matrix based on the feature matrix, and applies a preset threshold to the expanded response pool. The extended response pool is grouped and merged to determine the final classification of responses in the extended response pool; wherein, the final response classification is used to indicate the classification result of the request corresponding to each response in the extended response pool; an input alphabet is constructed based on the input request sequence, the requests corresponding to each response in the extended response pool whose classification results are determined, and the sequence prefix; the input alphabet is input to a pre-constructed blank automaton, and the requests stored in the input alphabet are selected in sequence to form an input sequence, and the input sequence is input to the target device to receive a first sequence response, and the input sequence is input to the blank automaton to receive a second sequence response; if the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response, a target automaton is obtained. This application acquires information through interaction, analyzes the responses generated by the interaction, and constructs equivalent automata to provide the system with universality for target devices with different architectures. In the active automaton learning of effective sequence exploration and improvement, hierarchical clustering technology is introduced to eliminate the randomness of the response and improve the accuracy of the system for responses containing random content. Furthermore, by accurately identifying the state of the target device, the system provides the precision of state differentiation, which can fully mine and identify the device state hidden deep within, thereby improving the coverage of fuzz testing.

[0161] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0162] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] Figure 9 An exemplary structural diagram of an automated machine construction device provided in an embodiment of this application is shown.

[0164] Based on the same inventive concept, corresponding to any of the above-described embodiments, this application also provides an automatic machine construction apparatus.

[0165] refer to Figure 9 The automatic machine construction device includes: an acquisition module, a first determination module, a second determination module, an extension module, a third determination module, a fourth determination module, a fifth determination module, and a construction module; wherein,

[0166] The acquisition module is configured to acquire a sequence of input request sequences used to trigger different functions on the target software of the target device;

[0167] The first determining module is configured to determine the first request sequence based on the current request after deleting the corresponding byte from the input request sequence and the previous request of the current request.

[0168] The second determining module is configured to determine the self-similarity score of the first response obtained based on the first request sequence and the similarity score of the first response to the second response in the response pool.

[0169] An expansion module is configured to add the first response to the response pool to obtain an expanded response pool if the similarity score is greater than the self-similarity score.

[0170] The third determining module is configured to extract feature vectors from the extended response pool to construct a feature matrix based on the feature vectors, generate a clustering matrix based on the feature matrix, and group and merge the extended response pool according to a preset threshold to determine the final classification of responses in the extended response pool; wherein, the final classification of responses is used to indicate the classification result of the requests corresponding to each response in the extended response pool;

[0171] The fourth determining module is configured to construct an input alphabet based on the input request sequence, the requests corresponding to each response in the extended response pool that determine the classification results, and the sequence prefix;

[0172] The fifth determining module is configured to input the input alphabet into a pre-constructed blank automaton, sequentially select requests stored in the input alphabet to form an input sequence, input the input sequence into the target device to receive a first sequence response, and input the input sequence into the blank automaton to receive a second sequence response;

[0173] The building module is configured to obtain a target automaton if the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response.

[0174] In one possible implementation, the acquisition module is further configured as follows:

[0175] Iterate through the different functions on the target software corresponding to the target device;

[0176] For each function, a packet capture operation is performed at the network interface card of the target device to obtain the request sequence corresponding to each function to determine the input request sequence.

[0177] In one possible implementation, the first determining module is further configured as follows:

[0178] For each request in the input request sequence

[0179] Delete the first byte, and construct a first request based on the request after deleting the byte and the previous request.

[0180] Delete the second byte, and construct another first request based on the request after deleting the byte and the previous request of the first request;

[0181] The sequence of the first requests is determined based on all of the first requests.

[0182] In one possible implementation, the second determining module is further configured as follows:

[0183] Send the first request sequence twice to the target device to obtain a first first response and a second first response;

[0184] The first response is divided into multiple first character segments using the differential method, and the second response is divided into multiple second character segments.

[0185] Multiple first character segments and multiple second character segments are matched pairwise to determine multiple first parts, and the sum of the dissimilar bits of each first part is determined to obtain a first edit distance between the first character segments and the second character segments; wherein, each first part includes a first character segment and a second character segment that matches the first character segment;

[0186] The self-similarity score of the first response is determined based on the first string length of the first response, the second string length of the second response, and the first edit distance.

[0187] In one possible implementation, the response pool includes: a pre-acquired second request sequence;

[0188] The second determining module is further configured as follows:

[0189] Send the first request sequence to the target device to obtain the first response, and send the second request sequence from the response pool to the target device to obtain the second response;

[0190] The first response is divided into multiple third character segments using a differential method, and the second response is divided into multiple fourth character segments.

[0191] Multiple third character segments and multiple fourth character segments are matched pairwise to determine multiple second parts, and the sum of the disjoint bits of each second part is determined to obtain a second edit distance between the third character segments and the fourth character segments; wherein each second part includes a third character segment and a fourth character segment that matches the third character segment;

[0192] The similarity score of the first response to the second response in the response pool is determined based on the third string length of the first response, the fourth string length of the second response, and the second edit distance.

[0193] In one possible implementation, the third determining module is further configured as follows:

[0194] Extract the target attributes of each response in the extended response pool to determine the feature vector; wherein, the target attributes include: the conversion frequency of character bytes to other byte types, the conversion frequency of numeric bytes to other byte types, the conversion frequency of space bytes to other byte types, the conversion frequency of special symbol bytes to other byte types, and the response length;

[0195] The feature matrix is ​​constructed based on all the feature vectors in the extended response pool;

[0196] Each target attribute in the feature vector is used as a location point in five-dimensional space, and the Euclidean distance between any two location points is determined accordingly.

[0197] Starting from any point, perform a merging operation according to the shortest Euclidean distance to obtain clusters, until the number of clusters reaches the preset threshold to generate the clustering matrix;

[0198] The classification information of the extended response pool corresponding to each cluster obtained in each merging operation before the number of clusters reaches the preset threshold is tracked.

[0199] The final response classification is determined based on all classification information to indicate the classification result of the request corresponding to each response in the extended response pool.

[0200] In one possible implementation, the fourth determining module is further configured as follows:

[0201] The input alphabet is input into a pre-constructed blank automaton. The length of the input alphabet is n. The requests stored in the input alphabet are selected in turn to combine to obtain an input sequence of length 1 to n.

[0202] The input sequence is input to the target device and the blank automaton respectively to receive the first sequence response and the second sequence response.

[0203] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0204] The apparatus of the above embodiments is used to implement the corresponding automatic mechanism construction method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0205] Figure 10 This illustration shows an exemplary structural diagram of an electronic device provided in an embodiment of this application.

[0206] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the automatic machine construction method described in any of the above embodiments. Figure 10 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0207] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0208] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0209] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0210] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0211] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0212] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0213] The electronic devices described above are used to implement the corresponding automatic mechanism construction methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0214] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the automatic machine construction method as described in any of the above embodiments.

[0215] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0216] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the automatic machine construction method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0217] Based on the same inventive concept, corresponding to the automated machine construction method described in any of the above embodiments, this disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to perform the automated machine construction method. Corresponding to the execution entity for each step in each embodiment of the automated machine construction method, the processor executing the corresponding step may belong to the corresponding execution entity.

[0218] The computer program products of the above embodiments are used to cause the computer and / or the processor to execute the automatic machine construction method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0219] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0220] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0221] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0222] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for constructing an automatic machine, characterized in that, The method includes: Obtain the sequence of input request requests used to trigger different functions on the target software of the target device; The first request sequence is determined based on the current request after deleting the corresponding byte from the input request sequence and the previous request of the current request. Determine the self-similarity score of the first response obtained based on the first request sequence and the similarity score of the first response to the second response in the response pool; If the similarity score is greater than the self-similarity score, then the first response is added to the response pool to obtain the expanded response pool; Feature vectors of the extended response pool are extracted to form a feature matrix. A clustering matrix is ​​generated based on the feature matrix. The extended response pool is then grouped and merged according to a preset threshold to determine the final classification of responses in the extended response pool. The final classification of responses is used to indicate the classification result of the request corresponding to each response in the extended response pool. An input alphabet is constructed based on the input request sequence, the requests corresponding to each response in the expanded response pool that determine the classification result, and the sequence prefix; The input alphabet is input into a pre-built blank automaton, and the requests stored in the input alphabet are selected in sequence to form an input sequence. The input sequence is then input into the target device to receive a first sequence response, and the input sequence is input into the blank automaton to receive a second sequence response. If the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response, then the target automaton is obtained.

2. The method according to claim 1, characterized in that, The step of acquiring the input request sequence for triggering different functions on the target software of the target device includes: Iterate through the different functions on the target software corresponding to the target device; For each function, a packet capture operation is performed at the network interface card of the target device to obtain the request sequence corresponding to each function to determine the input request sequence.

3. The method according to claim 1, characterized in that, The step of determining the first request sequence based on the current request after deleting the corresponding byte from the input request sequence and the previous request of the current request includes: For each request in the input request sequence Delete the first byte, and construct a first request based on the request after deleting the first byte and the previous request. Delete the second byte, and construct another first request based on the request after deleting the second byte and the previous request of the request; The sequence of the first requests is determined based on all of the first requests.

4. The method according to claim 1, characterized in that, Determining the self-similarity score of the first response obtained based on the first request sequence includes: Send the first request sequence twice to the target device to obtain a first first response and a second first response; The first response is divided into multiple first character segments using the differential method, and the second response is divided into multiple second character segments. Multiple first character segments and multiple second character segments are matched pairwise to determine multiple first parts, and the sum of the dissimilar bits of each first part is determined to obtain a first edit distance between the first character segments and the second character segments; wherein, each first part includes a first character segment and a second character segment that matches the first character segment; The self-similarity score of the first response is determined based on the first string length of the first response, the second string length of the second response, and the first edit distance.

5. The method according to claim 1, characterized in that, The response pool includes: a pre-acquired second request sequence; Determining the similarity score of the first response to the second response in the response pool includes: Send the first request sequence to the target device to obtain the first response, and send the second request sequence from the response pool to the target device to obtain the second response; The first response is divided into multiple third character segments using a differential method, and the second response is divided into multiple fourth character segments. Multiple third character segments and multiple fourth character segments are matched pairwise to determine multiple second parts, and the sum of the disjoint bits of each second part is determined to obtain a second edit distance between the third character segments and the fourth character segments; wherein each second part includes a third character segment and a fourth character segment that matches the third character segment; The similarity score of the first response to the second response in the response pool is determined based on the third string length of the first response, the fourth string length of the second response, and the second edit distance.

6. The method according to claim 1, characterized in that, The step of extracting feature vectors from the extended response pool to construct a feature matrix based on the feature vectors, generating a clustering matrix based on the feature matrix, and grouping and merging the extended response pool according to a preset threshold to determine the final classification of the responses in the extended response pool includes: Extract the target attributes of each response in the extended response pool to determine the feature vector; wherein, the target attributes include: the conversion frequency of character bytes to other byte types, the conversion frequency of numeric bytes to other byte types, the conversion frequency of space bytes to other byte types, the conversion frequency of special symbol bytes to other byte types, and the response length; The feature matrix is ​​constructed based on all the feature vectors in the extended response pool; Each target attribute in the feature vector is used as a location point in five-dimensional space, and the Euclidean distance between any two location points is determined accordingly. Starting from any point, perform a merging operation according to the shortest Euclidean distance to obtain clusters, until the number of clusters reaches the preset threshold to generate the clustering matrix; The classification information of the extended response pool corresponding to each cluster obtained in each merging operation before the number of clusters reaches the preset threshold is tracked. The final response classification is determined based on all classification information to indicate the classification result of the request corresponding to each response in the extended response pool.

7. The method according to claim 1, characterized in that, The step of inputting the input alphabet into a pre-constructed blank automaton, sequentially selecting requests stored in the input alphabet to form an input sequence, inputting the input sequence into the target device to receive a first sequence response, and inputting the input sequence into the blank automaton to receive a second sequence response includes: The input alphabet is input into a pre-constructed blank automaton. The length of the input alphabet is n. The requests stored in the input alphabet are selected in turn to combine to obtain an input sequence of length 1 to n. The input sequence is input to the target device and the blank automaton respectively to receive the first sequence response and the second sequence response.

8. An automatic machine-made device, characterized in that, The device includes: The acquisition module is configured to acquire a sequence of input request sequences used to trigger different functions on the target software of the target device; The first determining module is configured to determine the first request sequence based on the current request after deleting the corresponding byte from the input request sequence and the previous request of the current request. The second determining module is configured to determine the self-similarity score of the first response obtained based on the first request sequence and the similarity score of the first response to the second response in the response pool. An expansion module is configured to add the first response to the response pool to obtain an expanded response pool if the similarity score is greater than the self-similarity score. The third determining module is configured to extract feature vectors from the extended response pool to construct a feature matrix based on the feature vectors, generate a clustering matrix based on the feature matrix, and group and merge the extended response pool according to a preset threshold to determine the final classification of responses in the extended response pool; wherein, the final classification of responses is used to indicate the classification result of the requests corresponding to each response in the extended response pool; The fourth determining module is configured to construct an input alphabet based on the input request sequence, the requests corresponding to each response in the extended response pool that determine the classification results, and the sequence prefix; The fifth determining module is configured to input the input alphabet into a pre-constructed blank automaton, sequentially select requests stored in the input alphabet to form an input sequence, input the input sequence into the target device to receive a first sequence response, and input the input sequence into the blank automaton to receive a second sequence response; The building module is configured to obtain a target automaton if the first response classification determined based on the first sequence response is the same as the second response classification determined based on the second sequence response.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to implement the method according to any one of claims 1 to 7.

Images