A charging protocol boundary fuzzing test method and system based on reinforcement learning

Abstract

The application relates to a charging protocol boundary fuzzing test method and system based on reinforcement learning. The method comprises the following steps: converting a target charging protocol specification into a structured field tree to construct a protocol parsing tree; extracting seed messages from historical interconnection fault logs and normal communication logs to construct a seed library and extract seed feature vectors; inputting the seed feature vectors into a reinforcement learning model for strategy reasoning to output field-level variation strategies; executing field-level variation on the seed messages according to the field-level variation strategies and the protocol parsing tree to generate variation messages; sending the variation messages to a tested charging service platform, generating reward values according to response results and updating strategy network parameters; screening variation messages triggering abnormal responses to generate a defect report. The method can realize field-level directional variation and closed-loop adaptive optimization, improve charging protocol boundary vulnerability mining efficiency and test accuracy, reduce invalid test cases and reduce manual troubleshooting costs.

Description

Technical Field

[0001] This invention belongs to the field of new energy vehicle charging technology, and in particular relates to a fuzzy testing method and system for charging protocol boundaries based on reinforcement learning. Background Technology

[0002] With the development of technology in the field of new energy charging interconnection, charging service platforms have gradually become the core intermediate hub connecting upstream traffic channels and downstream charging SaaS platforms, forming a technical architecture of parallel connection of multiple protocols. This architecture has the characteristics of multi-protocol compatibility and multi-channel business interoperability. The platform needs to adapt to the national standard GB / T27930-2015+, OCPP1.6 / 2.0.1 and various channel private protocols such as Xiaoju, Kuaidian, and Xindiantu. The differences in field precision, time zone processing, null value semantics, enumeration definition and other dimensions of different protocols make it easy for implicit adaptation defects to occur in the platform protocol adaptation process.

[0003] Traditional technologies primarily rely on three conventional solutions to complete charging protocol testing: First, testers manually write forward and reverse test cases, leveraging their industry experience to cover routine business processes and known abnormal scenarios; second, deterministic fault injection technology is used, employing a charging pile simulator to preset fixed fault codes such as overvoltage, undervoltage, and communication interruption to achieve repeatable testing of known fault scenarios; and third, a professional protocol consistency testing platform is used to simulate the handshake communication between the BMS and the charging pile, automatically generating single-protocol test messages to complete basic protocol consistency verification.

[0004] However, current traditional charging protocol testing methods have many shortcomings: Existing mainstream fault injection technologies are all deterministic testing modes with preset fixed scenarios. For example, patent CN120528837A can only complete routine fault data collection and log testing by selecting a fixed charging mode, and patent CN106291198A can only perform repetitive testing with preset fixed charging and communication faults. Both classic solutions can only reproduce known faults, belonging to static, deterministic testing methods, lacking non-deterministic autonomous exploration capabilities, and unable to identify implicit boundary vulnerabilities caused by protocol critical legal inputs. In addition, traditional solutions still have several shortcomings: First, manual test case writing heavily relies on subjective experience, failing to comprehensively cover various implicit boundary scenarios of the protocol, resulting in significant blind spots in test coverage; second, there is no historical fault learning mechanism, failing to accumulate effective testing experience from historical fault logs, and the testing process cannot be iteratively optimized; third, there is a lack of adaptive mutation capabilities, failing to dynamically adjust testing strategies based on the response feedback of the platform under test, resulting in extremely low testing relevance and efficiency; fourth, it is only suitable for single-pile, single-protocol testing scenarios, failing to cover special issues such as protocol precision conflicts and incompatibility in multi-protocol concurrent interaction scenarios. General-purpose fuzzing tools (such as AFL and Peach) employ byte-level random mutation, lacking understanding of the semantic constraints of charging protocol fields. Their mutated messages are discarded during the format verification phase and cannot enter the business processing flow, thus rendering them unsuitable for charging protocol boundary testing. Furthermore, general-purpose tools can only detect low-level defects such as program crashes and memory overflows, failing to detect implicit defects in the charging business logic layer, such as discrepancies in settlement amounts and misaligned billing cycles. This invention overcomes these limitations by achieving semantically aware, field-level precise mutation through protocol parse trees. Summary of the Invention

[0005] Therefore, it is necessary to provide a reinforcement learning-based charging protocol boundary fuzzy testing method and system that can automatically build a test seed library based on historical fault logs, intelligently generate legal mutation test cases for protocol boundaries, achieve adaptive dynamic mutation exploration based on reinforcement learning, and support multi-protocol concurrent boundary testing to uncover cross-protocol latent defects.

[0006] Firstly, this application provides a reinforcement learning-based method for testing the boundary fuzziness of charging protocols, including:

[0007] The target charging protocol specification is converted into a structured field tree, and a protocol parsing tree is constructed.

[0008] Seed packets are extracted from historical interconnection fault logs and normal communication logs to construct a seed library. Feature extraction is performed on various sub-packets in the seed library to obtain seed feature vectors for various sub-packets.

[0009] The seed feature vector is input into the reinforcement learning model to perform policy inference and output a field-level mutation policy.

[0010] Based on the field-level mutation strategy and protocol parsing tree, the seed message is mutated at the field level to generate mutated messages;

[0011] The mutated message is sent to the charging service platform under test, the response result returned by the charging service platform under test is received, a reward value is generated based on the response result, and the policy network parameters of the reinforcement learning model are updated based on the reward value.

[0012] Based on the response results, filter out the variant messages that triggered abnormal responses from the tested charging service platform and generate a defect report.

[0013] Furthermore, the target charging protocol specification is transformed into a structured field tree, and a protocol parse tree is constructed, including:

[0014] Read the field definitions of the target charging protocol specification, parse the data type and structural relationship of each field definition, construct nodes for each field, assign a unique node identifier and parent node identifier to each node, and label each node with field name, data type, string length limit, minimum value, maximum value, decimal precision, time zone requirement, whether it is required, enumeration value list and child node list, and integrate to obtain the field nodes corresponding to each field;

[0015] Based on the parent node identifier of each field node, each field node is attached to its corresponding parent node to construct the protocol parsing tree.

[0016] Furthermore, seed packets are extracted from historical interconnection fault logs and normal communication logs to construct a seed library. Feature extraction is then performed on various sub-packets in the seed library to obtain seed feature vectors for each sub-packet, including:

[0017] Abnormal messages are extracted from historical interconnection fault logs as initial seeds, and legitimate messages are extracted from normal communication logs as control seeds. The initial seeds and control seeds are deduplicated and classified to construct a seed library.

[0018] Calculate the ratio of the number of numeric fields to the total number of fields, the ratio of the number of string fields to the total number of fields, and the ratio of the number of required fields to the total number of fields in various sub-messages to obtain the percentage of numeric fields, the percentage of string fields, and the percentage of required fields.

[0019] Calculate the average value range span of each numeric type field in various sub-messages to obtain the average value range span; where the value range span span ratio is the difference between the maximum and minimum values ​​of each numeric type field divided by the maximum value.

[0020] Calculate the average ratio of the current length of each string type field in various sub-messages to the upper limit of string length to obtain the average string length percentage;

[0021] Extract historical mutation records of various sub-messages from historical interconnection fault logs, and calculate the average historical mutation reward value and the proportion of historical mutations triggering abnormal responses for various sub-messages based on the historical mutation records.

[0022] The seed feature vector is obtained by integrating the proportion of numerical fields, the proportion of string fields, the proportion of required fields, the average range of numerical values, the proportion of average string length, the average reward value of historical mutations, and the proportion of historical mutations that trigger abnormal responses.

[0023] Furthermore, based on the field-level mutation strategy and protocol parse tree, the seed message is subjected to field-level mutation to generate mutated messages, including:

[0024] Based on the field-level mutation strategy, determine the specified field to be mutated in the seed message, obtain the field node corresponding to the specified field in the protocol parsing tree, and read the attribute value in the field node that corresponds to the field-level mutation strategy.

[0025] Generate a mutation replacement value based on the attribute value, replace the specified field of the seed message with the mutation replacement value, and obtain the mutated message.

[0026] Furthermore, a reward value is generated based on the response result, including:

[0027] The system determines the type of response result. A first positive reward value is generated when the response result indicates that the tested charging service platform has crashed or returned a 500 error; a second positive reward value is generated when the response result indicates an exception in the business logic layer; a third positive reward value is generated when the response result indicates a parameter validation error; a zero reward value is generated when the response result is a normal response; and a negative reward value is generated when the response result indicates a format parsing rejection.

[0028] Among them, the first positive reward value is greater than the second positive reward value, the second positive reward value is greater than the third positive reward value, the third positive reward value is greater than the zero reward value, and the zero reward value is greater than the negative reward value.

[0029] Furthermore, the policy network parameters of the reinforcement learning model are updated based on the reward value, including:

[0030] Feature extraction is performed on the mutated message to obtain the mutated seed feature vector. The seed feature vector, field-level mutation policy, reward value and mutated seed feature vector of the seed message are stored as state transition tuples in the experience replay buffer.

[0031] Randomly sample the experience replay buffer to obtain sampled state transition tuples, calculate the Q target value of the sampled state transition tuples, and calculate the loss function of the policy network based on the Q target value. Update the policy network parameters through backpropagation based on the loss function.

[0032] Furthermore, the formula for calculating the loss function is:

[0033] ;

[0034] In the formula, For loss function, As a reward value, As a discount factor, For policy networks, The mutated seed feature vector, These are candidate actions for the next state. For policy networks In state Next candidate action Q-value estimation, Estimate the Q-value of the largest among all candidate actions. For seed feature vectors, For field-level mutation strategy, For policy networks In seed feature vector The following field-level mutation strategy Q-value estimation.

[0035] Furthermore, reinforcement learning-based methods for testing the fuzzy boundaries of charging protocols also include:

[0036] Seed messages corresponding to the GB / T27930 protocol, OCPP protocol, and proprietary protocols of each channel were selected from the seed bank to obtain GB / T27930 seed messages, OCPP seed messages, and channel seed messages.

[0037] Field-level mutations were performed on the national standard seed message, OCPP seed message, and channel seed message respectively to obtain national standard mutated message, OCPP mutated message, and channel mutated message;

[0038] The system concurrently sends the national standard variation message, OCPP variation message, and channel variation message to the charging service platform under test, and receives the response results from the charging service platform under test, thus obtaining the national standard response result, OCPP response result, and channel response result.

[0039] The national standard response results, OCPP response results, and channel response results belonging to the same business order number are compared, and the deviation value is calculated.

[0040] Determine if the deviation value exceeds the accuracy threshold. If the deviation value exceeds the accuracy threshold, mark the comparison result as a cross-protocol accuracy mismatch conflict. Summarize all comparison results marked as cross-protocol accuracy mismatch conflicts and generate a cross-protocol conflict report.

[0041] Secondly, this application also provides a reinforcement learning-based charging protocol boundary fuzzy testing system, which includes:

[0042] The parse tree construction module is used to convert the target charging protocol specification into a structured field tree and construct the protocol parse tree;

[0043] The seed management module is used to extract seed messages from historical interconnection fault logs and normal communication logs, build a seed library, and extract features from various sub-messages in the seed library to obtain seed feature vectors for various sub-messages.

[0044] The mutation policy decision module is used to input seed feature vectors into the reinforcement learning model, perform policy reasoning, and output field-level mutation policies.

[0045] The field-level mutation module is used to perform field-level mutations on seed packets according to the field-level mutation strategy and protocol parse tree to generate mutated packets;

[0046] The test execution and model update module is used to send the mutation message to the charging service platform under test, receive the response result returned by the charging service platform under test, generate a reward value based on the response result, and update the policy network parameters of the reinforcement learning model based on the reward value.

[0047] The defect report generation module is used to filter the variant messages that trigger abnormal responses from the tested charging service platform based on the response results and generate defect reports.

[0048] The aforementioned reinforcement learning-based fuzzy testing method and system for charging protocol boundaries transforms the target charging protocol specification into a structured field tree and constructs a protocol parse tree. Simultaneously, it extracts seed packets from historical interconnection fault logs and normal communication logs to build a seed library. Feature extraction is performed on various sub-packets to obtain seed feature vectors. This achieves a structured, hierarchical parsing representation of the charging protocol and standardized feature quantification of test seed samples. It addresses the problems of traditional fuzzing, such as reliance on manual protocol field analysis, arbitrary test seed selection, and a lack of unified protocol structure constraints leading to poor compliance of mutated packets and a high proportion of invalid test cases. Furthermore, by inputting the seed feature vectors into a reinforcement learning model for policy inference and outputting field-level mutation policies, and then combining these policies with the protocol parse tree to perform field-level mutations on seed packets to generate mutated packets, it achieves the technical effects of automatic generation of precise field-level mutation policies and targeted boundary mutation operations based on reinforcement learning. This solves the problem of traditional fuzzing using random whole-packet mutation modes and being unable to target high-risk boundary fields. The traditional fuzzing method addresses the issues of low vulnerability discovery efficiency and uneven coverage of boundary scenarios caused by targeted testing. By sending mutated packets to the tested charging service platform, receiving the response, generating reward values ​​based on the response, and updating the policy network parameters of the reinforcement learning model according to the reward values, it achieves closed-loop feedback optimization and dynamic iterative updates of the mutation strategy. This solves the problems of traditional fuzzing lacking a test effect feedback mechanism and having fixed mutation strategies that cannot be adaptively adjusted, preventing the test process from converging towards higher returns. Furthermore, by filtering mutated packets that trigger abnormal responses from the tested charging service platform based on its response and generating defect reports, it achieves automatic identification and filtering of test anomalies and standardized automated output of test results. This solves the problems of traditional fuzzing relying on manual anomaly investigation, low defect location efficiency, and time-consuming and laborious test result processing. It improves the efficiency and accuracy of boundary vulnerability discovery in charging protocols, providing accurate defect evidence for the security remediation of charging protocols. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of a reinforcement learning-based charging protocol boundary fuzzy testing method according to the present invention.

[0051] Figure 2 This is a flowchart illustrating the generation of a mutated message in one optional embodiment of the present invention;

[0052] Figure 3 This is a structural diagram of a charging protocol boundary fuzzy testing system based on reinforcement learning according to the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] In one embodiment, such as Figure 1 As shown, a reinforcement learning-based fuzzy testing method for charging protocol boundaries is provided. This embodiment illustrates the application of this method to a fuzzy testing terminal. It is understood that this method can also be applied to a fuzzy testing server, and further to a fuzzy testing system including both a fuzzy testing terminal and a fuzzy testing server, and is implemented through the interaction between the fuzzy testing terminal and the fuzzy testing server. In this embodiment, the method includes the following steps:

[0055] Step S101: Convert the target charging protocol specification into a structured field tree to construct the protocol parsing tree.

[0056] Optionally, the fuzzy testing terminal can read the original format definition document of the target charging protocol specification, identify the field content and hierarchical nesting relationship of all messages in the original format definition document, generate corresponding tree nodes layer by layer according to the subordinate relationship of the fields, and label all field constraint attributes agreed in the target charging protocol specification for each tree node. Finally, all nodes are integrated to obtain the protocol parsing tree.

[0057] Specifically, the target charging protocol specification may include, but is not limited to, the national standard GB / T27930 protocol specification, the OCPP protocol specification, and the private protocol specifications of mainstream traffic diversion channels; the protocol parse tree may be a tree data structure that represents the message field hierarchy and constraint attributes of the target charging protocol specification.

[0058] Step S102: Extract seed messages from historical interconnection fault logs and normal communication logs to construct a seed library, and extract features from various sub-messages in the seed library to obtain seed feature vectors for various sub-messages.

[0059] Optionally, the fuzzy testing terminal can acquire historical interconnection fault logs and normal communication logs, and select legally formatted messages from the communication payloads of the historical interconnection fault logs and normal communication logs as seed messages; the fuzzy testing terminal can classify and store all seed messages according to three dimensions: protocol type, message type, and source type, to construct a seed library; the fuzzy testing terminal can parse the field composition and attribute parameters of various sub-messages in the seed library one by one to construct seed feature vectors.

[0060] Specifically, historical interconnection fault logs and normal communication logs can be log files that record historical communication interaction data of the charging service platform under test; seed messages can be protocol messages that serve as initial input samples for fuzz testing; seed feature vectors can be vectors composed of multiple quantized feature dimensions, and seed feature vectors can be used to characterize the structural attributes and historical variation effects of seed messages.

[0061] Step S103: Input the seed feature vector into the reinforcement learning model, perform policy inference, and output the field-level mutation policy.

[0062] Optionally, the fuzz testing terminal can input the seed feature vector into the policy network of the reinforcement learning model, perform multi-layer nonlinear transformation calculations on the seed feature vector, and output the selection probability distribution corresponding to each candidate field-level mutation policy; the fuzz testing terminal can sample based on the selection probability distribution of each candidate field-level mutation policy to obtain the field-level mutation policy.

[0063] Specifically, the reinforcement learning model can be a neural network model constructed using a deep Q-network or a near-end policy optimization algorithm. The reinforcement learning model can be used to output mutation strategies based on the characteristics of the seed message. The field-level mutation strategy can be a set of operation rules for performing boundary mutations on message fields. The field-level mutation strategy can include, but is not limited to, seven types of strategies: precision truncation, precision threshold, time zone drift, length threshold, boundary offset, enumeration out of bounds, and nested out of bounds.

[0064] Step S104: Based on the field-level mutation strategy and protocol parse tree, perform field-level mutation on the seed message to generate a mutated message.

[0065] Optionally, the fuzzing terminal can determine the target field in the seed message that is applicable to the mutation strategy based on the type of the field-level mutation strategy; the fuzzing terminal can search for the field node corresponding to the target field in the protocol parsing tree and read the constraint attribute values ​​related to the current mutation strategy in the field node; the fuzzing terminal can calculate the mutated field value based on the constraint attribute value and the mutation rule; the fuzzing terminal can replace the target field in the seed message with the mutated field value to obtain the mutated message.

[0066] Specifically, field-level mutation can be a mutation operation that modifies the value of only a specified field in the message while keeping the overall structure of the message and other fields unchanged; the mutated message can be a test case message generated after field-level mutation, with the overall format conforming to the protocol specification requirements, and only the specified field values ​​being near the protocol boundary.

[0067] Step S105: Send the mutated message to the charging service platform under test, receive the response result returned by the charging service platform under test, generate a reward value based on the response result, and update the policy network parameters of the reinforcement learning model based on the reward value.

[0068] Optionally, the fuzz testing terminal can send the mutated message to the corresponding protocol interface of the charging service platform under test through a virtual charging station or a virtual channel. The fuzz testing terminal can monitor the response results returned by the charging service platform under test in real time, identify and parse the response results, and obtain the response result type. The fuzz testing terminal can calculate the reward value corresponding to this mutation based on the reward grading rules and the response result type. Based on the reward value, the fuzz testing terminal can update the policy network parameters of the reinforcement learning model through the backpropagation gradient descent algorithm.

[0069] Specifically, the response result can be the processing result returned by the charging service platform under test after receiving the mutated message. The response result may include, but is not limited to, the response status code, business processing data, and abnormal prompt information. The policy network parameters can be the weights and bias parameters of the policy network in the reinforcement learning model. The policy network parameters can be used to determine the output result of policy inference.

[0070] Step S106: Based on the response results, filter the variant messages that trigger abnormal responses from the tested charging service platform and generate a defect report.

[0071] Optionally, the fuzzing terminal can iterate through the response results of all test rounds, filter out test records with an abnormal response type, and obtain abnormal records. The fuzzing terminal can extract information such as the original seed message, field-level mutation strategy, mutated message, response result of the tested charging service platform, abnormal type determination, and protocol specification basis from the abnormal records, and generate repair suggestions by combining the protocol specification basis and the abnormal type determination result. The fuzzing terminal can integrate the original seed message, field-level mutation strategy, mutated message, response result of the tested charging service platform, abnormal type determination, protocol specification basis, and repair suggestions to obtain a defect report.

[0072] Specifically, a defect report can be a document that records details of defects at the boundary of the charging protocol. A defect report may include, but is not limited to, the original seed message, field-level mutation strategy, mutated message, response result of the charging service platform under test, anomaly type determination, protocol specification basis, and repair suggestions.

[0073] The aforementioned reinforcement learning-based fuzzy testing method for charging protocol boundaries involves transforming the target charging protocol specification into a structured field tree to construct a protocol parse tree; extracting seed packets from historical interconnection fault logs and normal communication logs to build a seed library and extracting seed feature vectors; inputting the seed feature vectors into a reinforcement learning model for policy reasoning to output a field-level mutation policy; performing field-level mutation on the seed packets according to the field-level mutation policy and the protocol parse tree to generate mutated packets; sending the mutated packets to the tested charging service platform, generating reward values ​​and updating policy network parameters based on the response results; and filtering mutated packets that trigger abnormal responses to generate defect reports. This method enables field-level targeted mutation and closed-loop adaptive optimization, improving the efficiency and accuracy of charging protocol boundary vulnerability discovery, reducing invalid test cases, and lowering manual investigation costs.

[0074] In one embodiment, converting the target charging protocol specification into a structured field tree and constructing a protocol parse tree may include:

[0075] Step S201: Read the field definitions of the target charging protocol specification, parse the data type and structural relationship of each field definition, construct nodes for each field, assign a unique node identifier and parent node identifier to each node, and label each node with field name, data type, string length limit, minimum value, maximum value, decimal precision, time zone requirement, whether it is required, enumeration value list and child node list, and integrate to obtain the field nodes corresponding to each field.

[0076] Optionally, the fuzzy testing terminal can parse the field definition entries in the target charging protocol specification to extract the name, data type, and constraint parameters of each field; the fuzzy testing terminal can generate a globally unique node identifier for each field, and determine the node identifier of the corresponding upper-level field based on the nesting relationship of the fields, which serves as the parent node identifier of the current field; the fuzzy testing terminal can write the upper limit of string length, minimum value, maximum value, decimal precision, time zone requirements, whether the attribute is required, the enumeration value list, and the child node list from the constraint parameters into the attribute storage unit of the corresponding field node to construct the field node.

[0077] Specifically, the node unique identifier can be a unique string generated by a combination of numbers and letters, which can be used to distinguish different nodes in the protocol parse tree; the parent node identifier can be the node unique identifier of the parent field to which the current field belongs, which can be used to represent the hierarchical relationship between fields; the data type can include, but is not limited to, seven types: STRING, INT, LONG, DECIMAL, BOOLEAN, OBJECT, and ARRAY.

[0078] Step S202: Based on the parent node identifier of each field node, attach each field node to its corresponding parent node to construct the protocol parsing tree.

[0079] Optionally, the fuzzing terminal can traverse all field nodes, find the corresponding parent node based on the parent node identifier of the field node, attach the field node to the corresponding parent node, and add the field node to the parent node's child node list; the fuzzing terminal can start from the root node and complete the attachment of all child nodes layer by layer to construct the protocol parsing tree.

[0080] In one embodiment, seed packets are extracted from historical interconnection fault logs and normal communication logs to construct a seed library. Feature extraction is then performed on various sub-packets in the seed library to obtain seed feature vectors for each sub-packet. This may include:

[0081] Step S301: Extract abnormal messages from historical interconnection fault logs as initial seeds, and extract legitimate messages from normal communication logs as control seeds. Deduplicate and classify the initial seeds and control seeds to construct a seed library.

[0082] Optionally, the fuzzy testing terminal can traverse all communication records in the historical interconnection fault log and extract abnormal messages with complete payload formats as initial seeds; the fuzzy testing terminal can traverse all communication records in the normal communication log and extract legitimate messages with normal business processes as control seeds; the fuzzy testing terminal can perform deduplication processing on the initial seeds and control seeds based on the hash value of the message payload to obtain deduplicated seed messages; the fuzzy testing terminal can establish a classification index for deduplicated seed messages according to three dimensions: protocol type, message type, and source type, and store the classification index and deduplicated seed messages accordingly to construct a seed library.

[0083] Specifically, the initial seed can be a message that has historically triggered anomalies in the charging service platform under test. The initial seed can be used to guide fuzz testing to explore in high-risk directions. The control seed can be a legitimate message under normal business scenarios. The control seed can be used to verify the legitimacy of boundary variations. The seed library can be a collection of seed messages with multi-dimensional classification indexes.

[0084] Step S302: Calculate the ratio of the number of numeric type fields to the total number of fields, the ratio of the number of string type fields to the total number of fields, and the ratio of the number of required fields to the total number of fields in various sub-messages, to obtain the percentage of numeric fields, the percentage of string fields, and the percentage of required fields.

[0085] Optionally, the fuzzy testing terminal can traverse all fields of various sub-messages and count the total number of fields, the number of numeric fields, the number of string fields, and the number of required fields. The fuzzy testing terminal can divide the number of numeric fields, the number of string fields, and the number of required fields by the total number of fields to obtain the percentage of numeric fields, the percentage of string fields, and the percentage of required fields.

[0086] Step S303: Calculate the average value range span ratio of each numeric type field in various sub-messages to obtain the average numeric range span; wherein, the value range span ratio is the difference between the maximum and minimum values ​​of each numeric type field divided by the maximum value.

[0087] Optionally, the fuzzy testing terminal can read the minimum and maximum values ​​of each numeric type field in various sub-messages from the protocol parsing tree, and divide the difference between the maximum and minimum values ​​by the maximum value to obtain the value range span ratio of each numeric type field; the fuzzy testing terminal can sum the value range span ratios of all numeric type fields and divide by the total number of numeric type fields to obtain the average numeric range span.

[0088] Step S304: Calculate the average ratio of the current length of each string type field in various sub-messages to the upper limit of string length to obtain the average string length percentage.

[0089] Optionally, the fuzzy testing terminal can read the upper limit of the string length corresponding to each string type field in various sub-messages from the protocol parsing tree, and at the same time count the current length of each string type field; the fuzzy testing terminal can calculate the ratio of the current length of each string type field to the corresponding upper limit of the string length, and sum the ratios of all string type fields and divide by the total number of string type fields to obtain the average string length ratio.

[0090] Step S305: Extract historical variation records of various sub-messages from the historical interconnection fault log, and calculate the average historical variation reward value and the proportion of historical variation triggering abnormal response for various sub-messages based on the historical variation records.

[0091] Optionally, the fuzzy testing terminal can retrieve the corresponding historical mutation record in the historical interconnection fault log based on the unique identifier of the seed message. If a historical mutation record is found, the terminal counts the total number of historical mutations, the reward value obtained for each mutation, and the total number of mutations that trigger an abnormal response. The fuzzy testing terminal can sum the reward values ​​of each mutation and divide them by the total number of mutations to obtain the average reward value of historical mutations. The terminal can divide the number of mutations with a reward value greater than zero by the total number of mutations to obtain the proportion of historical mutations that trigger an abnormal response. If no historical mutation record is found, the average reward value of historical mutations and the proportion of historical mutations that trigger an abnormal response are set to zero.

[0092] Step S306: Integrate the proportion of numerical fields, the proportion of string fields, the proportion of required fields, the average numerical range span, the average string length, the average reward value of historical mutations, and the proportion of historical mutations triggering abnormal responses to obtain the seed feature vector.

[0093] Optionally, the fuzz testing terminal can arrange the proportion of numerical fields, the proportion of string fields, the proportion of required fields, the average numerical range span, the average string length, the average reward value of historical mutations, and the proportion of historical mutations triggering abnormal responses in order, and integrate them to obtain a seed feature vector.

[0094] Specifically, the seed feature vector can be a real number vector with a dimension of 7, with each dimension corresponding to a feature index of the seed message.

[0095] In one embodiment, such as Figure 2 As shown, based on the field-level mutation strategy and protocol parse tree, the seed message is mutated at the field level to generate mutated messages, which may include:

[0096] Step S401: Based on the field-level mutation strategy, determine the specified field to be mutated in the seed message, obtain the field node corresponding to the specified field in the protocol parsing tree, and read the attribute value in the field node that corresponds to the field-level mutation strategy.

[0097] Optionally, the fuzzing terminal can match the field data type applicable to the field-level mutation strategy to obtain the mutated data type; the fuzzing terminal can filter out the field that matches the mutated data type from all fields of the seed message as the specified field; the fuzzing terminal can traverse the protocol parsing tree to find the corresponding field node according to the field name of the specified field, and extract the attribute value corresponding to the field-level mutation strategy from the attributes of the field node.

[0098] Specifically, the specified field can be a field that meets the applicable conditions of the current mutation strategy. For example, the precision truncation strategy is only applicable to numeric fields of type DECIMAL, and the time zone drift strategy is only applicable to LONG fields of type timestamp.

[0099] Step S402: Generate a mutation replacement value based on the attribute value, and replace the specified field of the seed message with the mutation replacement value to obtain the mutated message.

[0100] Optionally, the fuzzing terminal can calculate and generate mutated replacement values ​​based on the operation rules of each field-level mutation strategy and the attribute values. The fuzzing terminal can locate the value position of a specified field in the seed message, replace the original value with the mutated replacement value, keep other fields and message structure unchanged, and generate the mutated message.

[0101] In one embodiment, generating a reward value based on the response result may include:

[0102] Step S501: Determine the type of response result. When the response result indicates that the tested charging service platform has crashed or returned a 500 error, generate a first positive reward value; when the response result indicates an exception in the business logic layer, generate a second positive reward value; when the response result indicates a parameter validation error, generate a third positive reward value; when the response result is a normal response, generate a zero reward value; when the response result is a format parsing rejection, generate a negative reward value.

[0103] Optionally, the fuzzing terminal can parse the status code and return content of the response result to determine the type of the response result. The fuzzing terminal can generate different reward values ​​according to the response type and reward grading rules. When the response result indicates that the tested charging service platform has crashed or returned a 500 error, a first positive reward value is generated; when the response result indicates an exception in the business logic layer, a second positive reward value is generated; when the response result indicates a parameter validation error, a third positive reward value is generated; when the response result is a normal response, a zero reward value is generated; and when the response result indicates a format parsing rejection, a negative reward value is generated.

[0104] Specifically, the ratio of the first positive reward value, the second positive reward value, the third positive reward value, the zero reward value, and the negative reward value can be 10:8:5:0:-5; the first positive reward value can correspond to the scenario of discovering a serious system defect, the second positive reward value can correspond to the scenario of discovering a business logic defect, the third positive reward value can correspond to the scenario of discovering a parameter verification defect, and the negative reward value can correspond to the scenario of no response to a mutated message format.

[0105] In one embodiment, updating the policy network parameters of the reinforcement learning model based on the reward value may include:

[0106] Step S601: Extract features from the mutated message to obtain the mutated seed feature vector, and store the seed feature vector, field-level mutation policy, reward value and mutated seed feature vector of the seed message as a state transition tuple in the experience replay buffer.

[0107] Optionally, the fuzz testing terminal can perform field statistics and feature calculations on the mutated message according to the same extraction rules as the seed feature vector to obtain the mutated seed feature vector; the fuzz testing terminal can write the seed feature vector of the current state, the field-level mutation strategy, the reward value, and the mutated seed feature vector of the next state as a state transition tuple into the experience replay buffer.

[0108] Specifically, the experience replay buffer can be a storage area used to store historical state transition data, and the experience replay buffer can be used to sample historical experience during the training of reinforcement learning models.

[0109] Step S602: Randomly sample the experience replay buffer to obtain the sampled state transition tuple, calculate the Q target value of the sampled state transition tuple, calculate the loss function of the policy network based on the Q target value, and update the policy network parameters through backpropagation based on the loss function.

[0110] Optionally, the fuzz testing terminal can set the sampling quantity and randomly sample state transition tuples from the experience replay buffer as training samples according to the sampling quantity; the fuzz testing terminal can calculate the Q target value corresponding to each training sample based on the Bellman equation, and calculate the loss function by combining it with the current Q value output by the policy network; the fuzz testing terminal can backpropagate the loss function through the gradient descent algorithm to update the network parameters of each layer of the policy network.

[0111] In one embodiment, the formula for calculating the loss function can be:

[0112] ;

[0113] In the formula, For loss function, As a reward value, As a discount factor, For policy networks, The mutated seed feature vector, These are candidate actions for the next state. For policy networks In state Next candidate action Q-value estimation, Estimate the Q-value of the largest among all candidate actions. For seed feature vectors, For field-level mutation strategy, For policy networks In seed feature vector The following field-level mutation strategy Q-value estimation.

[0114] In one embodiment, the reinforcement learning-based charging protocol boundary fuzzy testing method may further include:

[0115] Step S701: Select the seed messages corresponding to the GB / T27930 protocol, OCPP protocol and the private protocols of each channel from the seed library to obtain the GB / T27930 seed message, OCPP seed message and channel seed message.

[0116] Optionally, the fuzzy testing terminal can search the seed database according to the protocol type dimension based on the multi-protocol coverage requirements of the test task, and extract the corresponding seed messages under the national standard GB / T27930 protocol, OCPP protocol and the private protocols of each channel, to obtain national standard seed messages, OCPP seed messages and channel seed messages.

[0117] Specifically, the national standard seed message can be a seed message that conforms to the national standard GB / T27930 protocol specification; the OCPP seed message can be a seed message that conforms to the OCPP protocol specification; and the channel seed message can be a seed message that conforms to the private protocol specification of mainstream traffic diversion channels.

[0118] Step S702: Perform field-level mutations on the national standard seed message, OCPP seed message, and channel seed message respectively to obtain national standard mutated message, OCPP mutated message, and channel mutated message.

[0119] Optionally, the fuzz testing terminal can perform field-level mutations on the national standard seed message, OCPP seed message, and channel seed message respectively to generate national standard mutated messages, OCPP mutated messages, and channel mutated messages.

[0120] Step S703: The national standard variation message, OCPP variation message and channel variation message are sent concurrently to the charging service platform under test, and the response results of the charging service platform under test are received to obtain the national standard response result, OCPP response result and channel response result.

[0121] Optionally, the fuzzing terminal can use a multi-threaded concurrent scheduling mechanism to simultaneously send national standard variant messages, OCPP variant messages, and channel variant messages to the different protocol interfaces corresponding to the charging service platform under test; the fuzzing terminal can listen to the return data of the three protocol interfaces in parallel, and receive and store the national standard response results, OCPP response results, and channel response results respectively.

[0122] Specifically, concurrent sending can be multiple protocol variant messages sending requests to the charging service platform under test at the same time, which can simulate the business state of multiple protocols interacting simultaneously in real-world scenarios.

[0123] Step S704: Compare the national standard response results, OCPP response results, and channel response results belonging to the same business order number, and calculate the deviation value.

[0124] Optionally, the fuzzy testing terminal can group the national standard response results, OCPP response results, and channel response results according to the business order number, and extract the processed values ​​of each field in the national standard response results, OCPP response results, and channel response results under the same business order number; the fuzzy testing terminal can select the corresponding deviation calculation method according to the field type, and calculate the deviation value by combining the processed values.

[0125] Optionally, the formula for calculating the deviation value of a numeric semantic field can be:

[0126] Deviation value = |Value of the first protocol field -Value of the second protocol field|

[0127] Optionally, the formula for calculating the deviation value of a state-type semantic field can be:

[0128] Deviation value = 0 (when the values ​​of the two protocol fields are consistent)

[0129] Deviation value = 1 (when the values ​​of the two protocol fields are inconsistent)

[0130] Step S705: Determine whether the deviation value exceeds the accuracy threshold. If the deviation value exceeds the accuracy threshold, mark the comparison result as a cross-protocol accuracy mismatch conflict. Summarize all comparison results marked as cross-protocol accuracy mismatch conflicts and generate a cross-protocol conflict report.

[0131] Optionally, the fuzz testing terminal can set a precision threshold and compare the deviation value with the precision threshold to obtain the comparison result; if the deviation value is greater than the precision threshold, the comparison result is marked as a cross-protocol precision mismatch conflict; the fuzz testing terminal can summarize all comparison results marked as cross-protocol precision mismatch conflicts, supplement the conflict type, involved protocols, deviation details and risk description, and generate a cross-protocol conflict report.

[0132] Specifically, the precision threshold can be a pre-set upper limit of allowable deviation. The precision threshold of numerical fields can be calculated proportionally based on the highest precision agreed upon in the protocol. The precision threshold of timestamp fields can be set to the upper limit of time difference, and the precision threshold of status fields can be set to zero.

[0133] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0134] Based on the same inventive concept, this application also provides a reinforcement learning-based charging protocol boundary fuzz testing system for implementing the reinforcement learning-based charging protocol boundary fuzz testing method described above. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations in one or more reinforcement learning-based charging protocol boundary fuzz testing system embodiments provided below can be found in the limitations of the reinforcement learning-based charging protocol boundary fuzz testing method described above, and will not be repeated here.

[0135] In one exemplary embodiment, such as Figure 3 As shown, a reinforcement learning-based charging protocol boundary fuzzy testing system 500 is provided, comprising:

[0136] The parse tree construction module 501 can be used to convert the target charging protocol specification into a structured field tree and construct the protocol parse tree;

[0137] The seed management module 502 can be used to extract seed messages from historical interconnection fault logs and normal communication logs, construct a seed library, and extract features from various sub-messages in the seed library to obtain seed feature vectors for various sub-messages.

[0138] The mutation policy decision module 503 can be used to input seed feature vectors into a reinforcement learning model, perform policy reasoning, and output field-level mutation policies.

[0139] The field-level mutation module 504 can be used to perform field-level mutations on seed packets according to the field-level mutation strategy and protocol parse tree to generate mutated packets;

[0140] The test execution and model update module 505 can be used to send the mutation message to the charging service platform under test, receive the response result returned by the charging service platform under test, generate a reward value based on the response result, and update the policy network parameters of the reinforcement learning model based on the reward value.

[0141] The defect report generation module 506 can be used to filter the variant messages that trigger abnormal responses from the tested charging service platform based on the response results and generate a defect report.

[0142] The above embodiments merely illustrate several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the embodiments of this application, and these all fall within the protection scope of the embodiments of this application.

Claims

1. A fuzzy testing method for charging protocol boundaries based on reinforcement learning, characterized in that, The method includes: The target charging protocol specification is converted into a structured field tree, and a protocol parsing tree is constructed. Seed packets are extracted from historical interconnection fault logs and normal communication logs to construct a seed library. Feature extraction is performed on each seed packet in the seed library to obtain a seed feature vector for each seed packet. The seed feature vector is input into the reinforcement learning model for policy reasoning, and a field-level mutation policy is output. Based on the field-level mutation strategy and the protocol parsing tree, the seed message is mutated at the field level to generate a mutated message; The mutated message is sent to the charging service platform under test, the response result returned by the charging service platform under test is received, a reward value is generated based on the response result, and the policy network parameters of the reinforcement learning model are updated based on the reward value. Based on the response results, the mutated messages that triggered the abnormal response of the tested charging service platform are filtered out, and a defect report is generated.

2. The method according to claim 1, characterized in that, The process of converting the target charging protocol specification into a structured field tree and constructing a protocol parsing tree includes: Read the field definitions of the target charging protocol specification, parse the data type and structural relationship of each field definition, construct nodes for each field, assign a unique node identifier and a parent node identifier to each node, and label each node with field name, data type, string length limit, minimum value, maximum value, decimal precision, time zone requirement, whether it is required, enumeration value list and child node list, and integrate to obtain the field nodes corresponding to each field; Based on the parent node identifier of each field node, each field node is attached to its corresponding parent node to construct the protocol parsing tree.

3. The method according to claim 2, characterized in that, The process involves extracting seed packets from historical interconnection fault logs and normal communication logs to construct a seed library, and then extracting features from each seed packet in the seed library to obtain a seed feature vector for each seed packet, including: Abnormal messages are extracted from the historical interconnection fault logs as initial seeds, and legitimate messages are extracted from the normal communication logs as control seeds. The initial seeds and the control seeds are deduplicated and classified to construct a seed library. Calculate the ratio of the number of numeric fields to the total number of fields, the ratio of the number of string fields to the total number of fields, and the ratio of the number of required fields to the total number of fields in each seed message to obtain the percentage of numeric fields, the percentage of string fields, and the percentage of required fields. Calculate the average value range span ratio of each numeric type field in each seed message to obtain the average numeric range span; wherein, the value range span ratio is the difference between the maximum value and the minimum value corresponding to each numeric type field divided by the maximum value. Calculate the average ratio of the current length of each string type field in each seed message to the upper limit of the string length to obtain the average string length percentage; Extract historical mutation records of each seed message from the historical interconnection fault log, and calculate the average historical mutation reward value and the proportion of historical mutation triggering abnormal response for each seed message based on the historical mutation records. The seed feature vector is obtained by integrating the proportion of numerical fields, the proportion of string fields, the proportion of required fields, the average numerical range span, the average string length, the average reward value of historical mutations, and the proportion of historical mutations triggering abnormal responses.

4. The method according to claim 2, characterized in that, The step of performing field-level mutations on the seed packet according to the field-level mutation strategy and the protocol parsing tree to generate mutated packets includes: According to the field-level mutation strategy, the specified field to be mutated in the seed message is determined, and the field node corresponding to the specified field is obtained in the protocol parsing tree. The attribute value corresponding to the field-level mutation strategy in the field node is read. A mutation replacement value is generated based on the attribute value, and the specified field of the seed message is replaced with the mutation replacement value to obtain the mutated message.

5. The method according to claim 1, characterized in that, The step of generating a reward value based on the response result includes: The type of the response result is determined. When the response result indicates that the tested charging service platform has crashed or returned a 500 error, a first positive reward value is generated; when the response result indicates a business logic layer exception, a second positive reward value is generated; when the response result indicates a parameter validation error, a third positive reward value is generated; when the response result is a normal response, a zero reward value is generated; and when the response result indicates a format parsing rejection, a negative reward value is generated. Wherein, the first positive reward value is greater than the second positive reward value, the second positive reward value is greater than the third positive reward value, the third positive reward value is greater than the zero reward value, and the zero reward value is greater than the negative reward value.

6. The method according to claim 5, characterized in that, The step of updating the policy network parameters of the reinforcement learning model based on the reward value includes: The mutated message is subjected to feature extraction to obtain the mutated seed feature vector, and the seed feature vector of the seed message, the field-level mutation policy, the reward value and the mutated seed feature vector are stored as a state transition tuple in the experience replay buffer. The experience replay buffer is randomly sampled to obtain sampled state transition tuples. The Q target value of the sampled state transition tuples is calculated, and the loss function of the policy network is calculated based on the Q target value. The parameters of the policy network are updated through backpropagation based on the loss function.

7. The method according to claim 6, characterized in that, The formula for calculating the loss function is as follows: ； In the formula, Let the loss function be... The reward value is... As a discount factor, For policy networks, The mutated seed feature vector, These are candidate actions for the next state. For the policy network In state Next candidate action Q-value estimation, Estimate the Q value that is the largest among all candidate actions. The seed feature vector, For the aforementioned field-level mutation strategy, For the policy network In the seed feature vector The following is a description of the field-level mutation strategy. Q-value estimation.

8. The method according to claim 1, characterized in that, The method further includes: The seed messages corresponding to the GB / T27930 protocol, OCPP protocol and the private protocols of each channel are selected from the seed library to obtain the GB / T27930 seed message, OCPP seed message and channel seed message; The national standard seed message, the OCPP seed message, and the channel seed message are subjected to field-level mutations to obtain national standard mutated messages, OCPP mutated messages, and channel mutated messages, respectively. The national standard variation message, the OCPP variation message, and the channel variation message are sent concurrently to the charging service platform under test, and the response results of the charging service platform under test are received to obtain the national standard response result, the OCPP response result, and the channel response result. The national standard response results, the OCPP response results, and the channel response results belonging to the same business order number are compared, and the deviation value is calculated. Determine whether the deviation value exceeds the accuracy threshold. If the deviation value exceeds the accuracy threshold, mark the comparison result as a cross-protocol accuracy mismatch conflict. Summarize all the comparison results marked as cross-protocol accuracy mismatch conflicts and generate a cross-protocol conflict report.

9. A fuzzy testing system for charging protocol boundaries based on reinforcement learning, characterized in that, The system includes: The parse tree construction module is used to convert the target charging protocol specification into a structured field tree and construct the protocol parse tree; The seed management module is used to extract seed messages from historical interconnection fault logs and normal communication logs, construct a seed library, and extract features from each seed message in the seed library to obtain a seed feature vector for each seed message. The mutation strategy decision module is used to input the seed feature vector into the reinforcement learning model, perform policy reasoning, and output a field-level mutation strategy. The field-level mutation module is used to perform field-level mutations on the seed packet according to the field-level mutation strategy and the protocol parsing tree to generate mutated packets; The test execution and model update module is used to send the mutation message to the charging service platform under test, receive the response result returned by the charging service platform under test, generate a reward value according to the response result, and update the policy network parameters of the reinforcement learning model according to the reward value. The defect report generation module is used to filter the mutated messages that trigger abnormal responses from the tested charging service platform based on the response results, and generate a defect report.

Images