A method for fuzz testing of data transmission software

Abstract

This invention discloses a fuzz testing method for data transmission software, comprising: obtaining a correct data set S t S t Each element in the data transmission software includes the correct command value C and the telemetry value T corresponding to the command value C; according to S t Generate an abnormal data set S f S f Each element in the table includes the instruction value C obtained by anomalously processing the instruction value C. y and the telemetry value T corresponding to the command value C; according to S f Generate mutation dataset S n S n Each element in the list includes a pair of instruction values ​​C. y The instruction value C obtained by mutation processing x and the telemetry value T corresponding to the command value C; for S n Prioritize the data and generate a dataset S for fuzz testing. m ; Send S sequentially to the data transmission software m The instruction value C contained in each element x Obtain the instruction value C x Corresponding telemetry value T x Compare the corresponding T x The fuzzy test results of the data transmission software are obtained by combining T with the given information. This invention achieves efficient and reliable testing of data transmission software and is easy to implement in engineering.

Description

Technical Field

[0001] This invention belongs to the field of satellite data transmission technology, and designs a fuzzy testing method for data transmission software. Background Technology

[0002] Many spacecraft malfunctions are related to software errors, and software testing is a primary technique for preventing such failures. Interface testing and functional testing are typically conducted during the testing process to ensure software quality. In embedded systems, the diverse application scenarios lead to varied interfaces and protocols, similarly increasing testing complexity. Satellite data transmission software is generally used to receive remote commands, perform necessary calculations, forward them to corresponding functional systems and standalone machines, and simultaneously forward received telemetry data to the command sender. In satellite data transmission software testing, due to the high complexity of the data transmission system and the need for continuous maintenance or upgrades, advanced technologies must be employed to improve reliability and availability. To address this, highly automated and efficient testing methods are required to meet the ever-increasing demands for development and testing efficiency. Summary of the Invention

[0003] The purpose of this invention is to overcome the above-mentioned defects and provide a fuzz testing method for data transmission software. This method solves the technical problem that the high complexity of existing data transmission software makes it difficult to accurately test its reliability and usability. This invention achieves efficient and reliable testing of data transmission software and is easy to implement in engineering.

[0004] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0005] A fuzzing method for data transmission software, comprising:

[0006] Obtain the correct data set S t S t Each element in the data transmission software includes the correct instruction value C and the telemetry value T corresponding to the instruction value C;

[0007] Based on the correct data set S t Generate an abnormal data set S f S f Each element in the table includes the instruction value C obtained by anomalously processing the instruction value C. y and the telemetry value T corresponding to the command value C;

[0008] Based on the abnormal data set S f Generate mutation dataset S n S n Each element in the list includes a pair of instruction values ​​C. y The instruction value C obtained by mutation processing xand the telemetry value T corresponding to the command value C; and the command value C x Mark the fields that have mutated;

[0009] Based on the aforementioned marker, for S n Prioritize the data and generate a dataset S for fuzz testing. m ;

[0010] Send S sequentially to the data transmission software m The instruction value C contained in each element x Obtain the instruction value C x Corresponding telemetry value T x ;

[0011] Compare the corresponding T x And T, to obtain the fuzzy test results of the data transmission software.

[0012] Furthermore, record S t ={(CT)1,……(CT)} n}, where n is S t The total number of elements; instruction value C y Includes instruction value C + and C - S f ={(C + -T)1,……(C + -T) n ,(C - -T)1,……(C - -T) n};

[0013] The basic frame format of a command value includes the following fields: frame header, reserved, source address, destination address, command type, command, identifier, payload length, data content, checksum, and frame trailer; where C + and C - These are the instruction values ​​obtained by performing arithmetic increment and decrement operations on the command field in instruction value C, respectively.

[0014] Furthermore, regarding the instruction value C y Methods for performing mutation treatment include:

[0015] For instruction value C + and C - The frame header, reserved, destination address, and frame trailer fields in the instruction value C are not mutated. + and C - The other fields in the code are mutated to obtain the instruction value C. x .

[0016] Furthermore, mutation methods include bit flipping, insertion / deletion, concatenation, boundary value or arithmetic operations.

[0017] Furthermore, regarding the instruction value C x Fields with mutations are marked as not covered.

[0018] Furthermore, regarding S n Prioritize the data and generate a dataset S for fuzz testing. m The methods include:

[0019] S1 determines S n Is it an empty set? If it is an empty set, end the loop; otherwise, proceed to step S2.

[0020] S2 determines S n Check if there are any elements containing uncovered markers. If so, proceed to step S3; otherwise, proceed to step S6.

[0021] S3 is based on S n , forming set S n，1 S n，2 S n，3 S n，4 S n，5 S n，6 S n，7 set S n，1 S n，2 S n，3 S n，4 S n，5 S n，6 S n，7 Each element contains the following fields: source address, command type, command, identifier, payload length, data content, and checksum, which are marked as uncovered.

[0022] Select set S n，1 S n，2 S n，3 S n，4 S n，5 S n，6 S n，7 The set with the largest number of elements is denoted as S. n，max Proceed to step S4;

[0023] S4 in S n，max Randomly select an element (C) x -T) i This sets the marker for the corresponding field in the element to be overridden, and sets S... n，max After the markers of the corresponding fields in all elements other than the specified element are changed to be overwritten, the other elements are placed in S. n Zhong and replace S nFind the corresponding element; proceed to step S5;

[0024] S5 will (C x -T) i Put S m and (C) x -T) i From S n Delete; return to step S2;

[0025] S6 will S n The field marker changes from pending coverage to uncovered, and the process returns to step S1.

[0026] Furthermore, when the transceiver port first establishes a connection with the data transmission software, or after the data transmission software crashes and restarts, the data transmission software is first initialized, and then S is sent to the data transmission software sequentially. m The instruction value C contained in each element x ;

[0027] The initialization method is to send S sequentially to the data transmission software. t Each element contains an instruction value C, which is compared with the returned telemetry value T. Initialization is complete when the two are consistent.

[0028] Furthermore, compare the returned telemetry value with the telemetry value T. If they are inconsistent, check whether the connection status between the transceiver port and the data transmission software is correct. If the connection status is incorrect, restore it to the correct status. If the inconsistency persists after restoring the correct status, reacquire S. t .

[0029] Furthermore, the test results also included the data transmission software's reception rate and execution efficiency;

[0030] The reception ratio is the number of times the data transmission software successfully receives T. x The number of S sent to the data transmission software m Instruction value C x The ratio of the number of;

[0031] Execution efficiency is the number of instructions processed per unit time by the data transmission software, C. x The number of.

[0032] Furthermore, compare the corresponding T x The specific methods for obtaining the fuzzy test results of the data transmission software, including T, include:

[0033] When T x If it is equal to T, the test fails;

[0034] When T x If the result is not equal to T, the test passes.

[0035] Compared with the prior art, the present invention has at least one of the following advantages:

[0036] (1) This invention creatively proposes a fuzz testing method for data transmission software, which generates a test dataset based on correct data, thereby achieving efficient and reliable testing of data transmission software;

[0037] (2) The present invention can make mutations in different fields of the protocol based on the characteristics of data transmission software, forming an effective set of abnormal test cases, and can effectively identify various abnormal situations that occur in data transmission software.

[0038] (3) The present invention introduces an algorithm to sort the test case set, and the test cases that meet the test requirements are executed first, which improves the efficiency of test execution.

[0039] (4) The principle of this invention is simple and easy to implement in engineering. Attached Figure Description

[0040] Figure 1 This is a flowchart of a fuzz testing method for data transmission software according to the present invention;

[0041] Figure 2 This is a schematic diagram of the basic frame format and mutation operation of the instructions of this invention;

[0042] Figure 3 This is a flowchart of the priority sorting algorithm of the present invention. Detailed Implementation

[0043] The features and advantages of the present invention will become clearer and more apparent from the following detailed description.

[0044] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments. Although various aspects of embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.

[0045] This invention utilizes fuzz testing to implement a novel software testing method that improves the effectiveness of interface testing, functional testing, security testing, and performance testing for software configuration items. By using fuzz testing, instructions are sent, their behavior is observed and intervened upon, unexpected inputs are provided to the software under test, and abnormal results are analyzed to discover software vulnerabilities.

[0046] The fuzz testing method for data transmission software of the present invention includes the following steps:

[0047] S1, based on the data transmission software communication protocol, performs testing on the correct test case set S. tThe preprocessing of the instructions generates an exception test case set S. f .

[0048] S2 for S f Each data entry has 11 fields in its basic frame structure, categorized into C based on field attributes. F C D C T C A There are four types, each using different data mutations and tags, forming a test case set S that is syntactically valid but has incorrect content and format. n .

[0049] S3 tests the test case set S n After screening and evaluation, test cases are placed into a test queue using priority sorting to form a test case set S for fuzz testing. m .

[0050] S4 uses test case set S m Conduct actual tests by communicating with the device under test through the transceiver port. If the device under test fails, control the device to restart.

[0051] The S5 uses a status analyzer to record the status transitions and responses of the device under test, and then analyzes the results to generate a report.

[0052] In one specific implementation, the method for preprocessing the original dataset in step S1 includes:

[0053] (1) According to the data transmission software communication protocol, send and record a single command value C, and return a telemetry value T. Record the correct send-receive relationship of n CTs to generate a test case set S. t ={(CT)1,……(CT)} n}

[0054] (2) Expand the abnormal data set and process each S data point individually. t In the basic frame format of C, the command field arithmetic increment record is C. + Arithmetic subtraction of 1 is recorded as C - Generates an abnormal test case set S f ={(C + -T)1,……(C + -T) n ,(C - -T)1,……(C - -T) n}

[0055] In one specific implementation, in step S2, for the test case set S fThe instruction values ​​C+ and C- in the code use specific data mutation methods depending on the type of the basic format frame of the instruction:

[0056] (1)C F = {Frame Header, Reserved, Destination Address, Frame Trailer}, this field is strictly fixed and has a specific physical meaning in the protocol, and will not be changed.

[0057] (2)C D ={Payload Length, Checksum}, This field is used when processing protocols. If the protocol is not implemented correctly and the checksum is missing, the length is insufficient and the expected content cannot be sent.

[0058] (3)C T ={source address, identifier}, this field may trigger errors related to poor protocol implementation or network communication errors after modification, and invalid data may trigger incorrect handling of timeouts and telemetry counts.

[0059] (4)C A ={Command Type, Command, Data Content}, This field is used to determine the validity of the command and command parameter values, covering both legal and illegal values, and is used to evaluate the correctness of the internal protocol implementation of the device under test.

[0060] Perform specific mutation operations based on the type. The corresponding methods for type and method are as follows: Figure 2 , such as S f The instruction value C in the code is mutated to C. x , will C x Fields that have been modified are marked as not overwritten and recorded in S. n ={(C x -T)1,……(C x -T) n}. S f The CCP contains 2n elements. For the i-th element, assuming its mutation count is j, the number of i-th elements increases from 1 to j. The telemetry values ​​corresponding to these j elements remain the same as the telemetry values ​​corresponding to the i-th element. S n The number of elements m in S f The sum of all elements formed by the mutation and amplification of each element.

[0061] In one specific implementation, in step S3, for the test set S n ={(C x -T)1,……(C x -T) n}, Test case set S for fuzz testing m ={(C m -T)1,……(C m -T)m The priority sorting method is as follows:

[0062] (1) Determine S n Check if the set is empty; if it is, end the loop; otherwise, continue.

[0063] (2) Determine S n If there is a protocol field marked as not covered, skip to step (3) if it is marked, otherwise skip to step (9).

[0064] (3) Select S n C x The field with the most basic frame formats marked is selected, and all test cases marked for this field are formed into set S. tmp ={(C x -T)1,……(C x -T) tmp},

[0065] (4) In set S tmp Randomly select test cases (C) x -T) i ;

[0066] (5) Select the test case (C) from step (4). x -T) i The corresponding field marker is changed to "overridden"; S tmp The remaining test cases in S n The field marker for the corresponding test case in the middle is changed to "to be covered";

[0067] (6) Test Cases (C) x -T) i Put S m ;

[0068] (7) Test Cases (C) x -T) i From S n delete;

[0069] (8) Return to step (2);

[0070] (9) When the loop reaches a certain moment, S n All test case coverage information in S is updated to be covered. n The coverage information changes from pending coverage to uncovered, and we return to step (1).

[0071] In one specific implementation, the method for the actual testing phase in step S4 includes:

[0072] (1) The transmit and receive ports establish a connection with the device under test (i.e., the data transmission software) for the first time, or after the device under test crashes and restarts, the correct test case set S is sent sequentially. t ={(CT)1,……(CT)} n In step C, compare the actual telemetry return data with the returned telemetry value T. If they match, the test is ready. If they don't match, check if the connection status between the transceiver port and the device under test is correct. If the connection status is incorrect, restore it to the correct state. If the connection status still doesn't match after restoring it to the correct state, then consider S. t An error exists; regenerate the test case set S. t .

[0073] (2) Once the test is ready, the test case set S used for fuzz testing will be... m ={(C x -T)1,……(C m -T) m C x The data is sent sequentially to the device, and the telemetry monitoring returns a value of T. x .

[0074] (3) Upon receiving telemetry return data T x If no message is received within the timeout period, log file C will be logged. x T x T, compare T x Differences between T and its content. For example, T... x If the result is equal to T, the test fails, and further manual interpretation is needed to confirm the specific issue.

[0075] In one specific implementation, when a certain C x The corresponding T x Equal to T, according to C x The reasons why mutations and specific field judgments in C can cause errors in data transmission software include: x C D A field of type C has mutated, indicating a protocol error. x C T A field of type C has mutated, indicating a faulty protocol or network communication error. x C A The field of type has undergone a mutation, indicating that the command or command parameter value is incorrect.

[0076] In one specific implementation, in step S5, the following performance indicators are obtained based on the recorded state transitions and responses of the device under test:

[0077] (1) Receiver ratio. This refers to the data T successfully received by the target device. x Number of test cases 'a' and test case set 'S'm The average ratio of the number of transmissions b. The receiving ratio = a / b.

[0078] (2) Execution efficiency. Used for fuzz testing S m ={(C x -T)1,……(C m -T) m Test cases (C) x The average ratio of the number m (-T) to the total time T of the actual test phase. Execution efficiency = m / T.

[0079] Traditional test case design for complex logical functions treats all test cases equally, leading to wasted time testing irrelevant content and neglecting problematic areas. The technical solution of this invention effectively solves these problems. As satellite functions and interfaces become increasingly complex, the requirements for software testing are constantly rising. The implementation scheme of this invention can be widely applied in the field of data transmission. The method of this invention has the advantages of simple principle and ease of engineering implementation.

[0080] Example:

[0081] Figure 1 This is a flowchart of a fuzz testing method for data transmission software according to the present invention. Scenarios with valid protocols are preprocessed to generate an original dataset. A data mutant applies a mutation scheme to form a set of test cases that are syntactically valid but have content and format errors. After screening and evaluation to define priorities, these test cases are added to the test sequence. Communication with the device under test (i.e., the data transmission software) is established through the transceiver port to induce unexpected state transitions in the device under test. For any malfunctions that cause the device under test to fail, a device restart module is used. The bus transceiver monitor captures the state transitions and responses of the device under test, which are then evaluated by a state analyzer. The state analyzer detects any errors, analyzes the results, and generates a report.

[0082] Figure 2 These are mutation operations for each field. The specific methods for each mutation operation shown in the diagram include:

[0083] (1) Bit flipping: Flip the data in the field by 4 bits in a step width of 4 bits, or flip all the bits of the field.

[0084] (2) Insertion and deletion: Insert part of the data in the field with a step width of 4 digits and use the preset values ​​0000B and 1111B in sequence. Delete 4 digits of data in the field with a step width of 4 digits.

[0085] (3) Concatenation: Select 4-digit data from the same field in 2 test cases and concatenate them to obtain 8-digit new data.

[0086] (4) Boundary values: The field takes values ​​according to the boundary with a 16-bit step width. The commonly used set of values ​​is {0x00, 0x01, 0xFE, 0xFF}.

[0087] (5) Arithmetic operations: use 8 bits as the step size, and perform addition and subtraction operations from the beginning according to the width of 8 bits, 16 bits, and 32 bits respectively.

[0088] Figure 3 This is a priority sorting diagram. By prioritizing test cases, test cases with more easily detectable errors are executed first during the actual testing phase, thereby increasing the efficiency of error detection. The specific method for prioritizing is as follows:

[0089] (1) Determine S n Check if the set is empty; if it is, end the loop; otherwise, continue.

[0090] (2) Determine S n If there is a protocol field marked as not covered, skip to step (3) if it is marked, otherwise skip to step (9).

[0091] (3) Select S n C x The field with the most basic frame formats marked is selected, and all test cases marked for this field are formed into set S. tmp ={(C x -T)1,……(C x -T) tmp That is, assuming that in the source address, command type, command, identifier, payload length, data content, and checksum, a certain field (denoted as field M) is marked as an uncovered C. x At most, all elements whose field is marked as uncovered are placed into set S. tmp middle;

[0092] (4) In set S tmp Randomly select test cases (C) x -T) i ;

[0093] (5) Select the test case (C) from step (4). x -T) i The corresponding field marker is changed to "overridden"; S tmp The remaining test cases in S n The field marker for the corresponding test case is changed to "to be covered"; in this step, the field M marker for the remaining test cases is changed to "to be covered".

[0094] (6) Test Cases (C) x -T) i Put S m ;

[0095] (7) Test Cases (C) x -T) i From S n delete;

[0096] (8) Return to step (2);

[0097] (9) When the loop reaches a certain moment, S n All test case coverage information in S is updated to be covered. n The coverage information changes from pending coverage to uncovered, and we return to step (1).

[0098] The present invention has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present invention. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and embodiments of the present invention without departing from the spirit and scope of the invention, and all such modifications and improvements fall within the scope of the present invention. The scope of protection of the present invention is defined by the appended claims.

[0099] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A fuzz testing method for data transmission software, characterized in that, include: Obtain the correct data set S t S t Each element in the data transmission software includes the correct instruction value C and the telemetry value T corresponding to the instruction value C; Based on the correct data set S t Generate an abnormal data set S f S f Each element in the table includes the instruction value C obtained by anomalously processing the instruction value C. y and the telemetry value T corresponding to the command value C; Based on the abnormal data set S f Generate mutation dataset S n S n Each element in the list includes a pair of instruction values ​​C. y The instruction value C obtained by mutation processing x and the telemetry value T corresponding to the command value C; and the command value C x Mark the fields that have mutated; Based on the aforementioned marker, for S n Prioritize the data and generate a dataset S for fuzz testing. m ; Send S sequentially to the data transmission software m The instruction value C contained in each element x Obtain the instruction value C x Corresponding telemetry value T x ; Compare the corresponding T x And T, to obtain the fuzzy test results of the data transmission software; Record S t ={(CT)1,……(CT) n }, where n is S t The total number of elements; instruction value C y Includes instruction value C + and C - S f ={(C + -T)1,……(C + -T) n ,(C - -T)1,……(C - -T) n }; The basic frame format of a command value includes the following fields: frame header, reserved, source address, destination address, command type, command, identifier, payload length, data content, checksum, and frame trailer; where C + and C - These are the instruction values ​​obtained by performing arithmetic increment and decrement operations on the command field in instruction value C, respectively. For instruction value C y Methods for performing mutation treatment include: For instruction value C + and C - The frame header, reserved, destination address, and frame trailer fields in the instruction value C are not mutated. + and C - The other fields in the code are mutated to obtain the instruction value C. x ; For S n Prioritize the data and generate a dataset S for fuzz testing. m The methods include: S1 determines S n Is it an empty set? If it is an empty set, end the loop; otherwise, proceed to step S2. S2 determines S n Check if there are any elements containing uncovered markers. If so, proceed to step S3; otherwise, proceed to step S6. S3 is based on S n , forming set S n，1 S n，2 S n，3 S n，4 S n，5 S n，6 S n，7 set S n，1 S n，2 S n，3 S n，4 S n，5 S n，6 S n，7 Each element contains the following fields: source address, command type, command, identifier, payload length, data content, and checksum, which are marked as uncovered. Select set S n，1 S n，2 S n，3 S n，4 S n，5 S n，6 S n，7 The set with the largest number of elements is denoted as S. n，max Proceed to step S4; S4 in S n，max Randomly select an element (C) x -T) i This sets the marker for the corresponding field in the element to be overridden, and sets S... n，max After the markers of the corresponding fields in all elements other than the specified element are changed to be overwritten, the other elements are placed in S. n Zhong and replace S n Find the corresponding element; proceed to step S5; S5 will (C x -T) i Put S m and (C) x -T) i From S n Delete; return to step S2; S6 will S n The field marker changes from pending coverage to uncovered, and the process returns to step S1.

2. The fuzz testing method for data transmission software according to claim 1, characterized in that, Mutation methods include bit flipping, insertion / deletion, concatenation, boundary value or arithmetic operations.

3. The fuzz testing method for data transmission software according to claim 1, characterized in that, For instruction value C x Fields with mutations are marked as not covered.

4. The fuzz testing method for data transmission software according to claim 1, characterized in that, When the transceiver port first establishes a connection with the data transmission software, or after the data transmission software crashes and restarts, it first initializes the data transmission software, and then sends S to the data transmission software sequentially. m The instruction value C contained in each element x ; The initialization method is to send S sequentially to the data transmission software. t Each element contains an instruction value C, which is compared with the returned telemetry value T. Initialization is complete when the two are consistent.

5. The fuzz testing method for data transmission software according to claim 4, characterized in that, Compare the returned telemetry value with the telemetry value T. If they are inconsistent, check whether the connection status between the transceiver port and the data transmission software is correct. If the connection status is incorrect, restore the correct status. If the inconsistency persists after restoring the correct state, reacquire S. t .

6. The fuzz testing method for data transmission software according to claim 1, characterized in that, The test results also include the reception rate and execution efficiency of the data transmission software; The reception ratio is the number of times the data transmission software successfully receives T. x The number of S sent to the data transmission software m Instruction value C x The ratio of the number of; Execution efficiency is the number of instructions (C) processed per unit time by the data transmission software. x The number of.

7. The fuzz testing method for data transmission software according to claim 1, characterized in that, Compare the corresponding T x The specific methods for obtaining the fuzzy test results of the data transmission software, including T, include: When T x If it is equal to T, the test fails; When T x If the result is not equal to T, the test passes.

Images