Automated conformance testing methods, systems, and media for Fibre Channel (FC) devices
By introducing a test control engine and protocol monitoring module, the test sequences of Fibre Channel FC devices are automatically executed and monitored, solving the problem of low efficiency of manual operation in existing technologies and realizing efficient and accurate automated testing.
Patent Information
- Application Number
- CN202611464426.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-20
- Publication Date
- 2026-11-17
AI Technical Summary
Current conformance testing of Fibre Channel (FC) equipment relies on manually setting up the hardware environment, which is inefficient and inaccurate, and cannot meet the needs of automated and efficient testing.
The test control engine loads predefined programmable test cases, automatically executes test sequences through FC endpoint simulation software, and monitors and compares Fibre Channel frames by the protocol monitoring and analysis module to generate test reports.
It automates conformance testing of Fibre Channel (FC) devices, improving testing efficiency and accuracy, supporting unattended continuous operation, and is suitable for agile development and continuous integration scenarios.
Smart Images

Figure CN123122649A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of communication technology, and in particular to automated conformance testing methods, systems and media for Fibre Channel (FC) devices. Background Technology
[0002] In the field of learning, research and development and debugging of the Fibre Channel (FC) protocol, performing protocol conformance testing on FC switches is a key step in verifying whether the equipment complies with the FC protocol standard and ensuring equipment interoperability.
[0003] Currently, protocol conformance testing of FC devices mainly relies on manually setting up the hardware environment for testing. This involves testers manually configuring real FC switches, servers, storage arrays, and dedicated FC sub-protocol analyzers, manually operating on the dedicated FC analyzer or writing scripts to construct test code and record the results.
[0004] However, in the above methods, the script for each test needs to be written manually, the test environment setup is time-consuming and laborious, and test execution, result observation and recording all require manual intervention, which is inefficient and inaccurate. Summary of the Invention
[0005] In view of this, embodiments of this application provide an automated conformance testing method, system, and medium for Fibre Channel (FC) devices to achieve automated conformance testing of Fibre Channel devices, thereby improving testing efficiency and accuracy.
[0006] This application provides an automated conformance testing method for Fibre Channel (FC) devices, applied to a test system including a test control engine, a protocol monitoring and analysis module, and at least one FC endpoint simulation software. The method includes: The test control engine loads predefined programmable test cases, which define the test sequence and expected response; The test control engine controls the FC endpoint simulation software to execute the test sequence during the protocol interaction with the FC device under test; The protocol monitoring and analysis module monitors the fiber channel frames transmitted during the protocol interaction process and determines the monitoring results based on the fiber channel frames. The test control engine obtains the monitoring results from the protocol monitoring and analysis module, compares the actual response in the monitoring results with the expected response in the test case, and automatically generates a test report based on the comparison results.
[0007] This application also provides an automated conformance testing system for Fibre Channel (FC) devices, the system comprising: a test control engine, a protocol monitoring and analysis module, and at least one FC endpoint simulation software; The test control engine is used to load predefined programmable test cases, which define test sequences and expected responses. It controls the FC endpoint simulation software to execute the test sequences during protocol interaction with the FC device under test, obtains the monitoring results of the protocol monitoring and analysis module, compares the actual response in the monitoring results with the expected response in the test cases, and automatically generates a test report based on the comparison results. The protocol monitoring and analysis module is used to monitor the fiber channel frames transmitted during the protocol interaction process and determine the monitoring results based on the fiber channel frames.
[0008] This application also provides an electronic device, including: a processor and a machine-readable storage medium for storing machine-executable instructions, wherein the machine-executable instructions, when run by the machine-readable storage medium, cause the processor to perform the steps of the above method.
[0009] This application also provides a machine-readable storage medium storing machine-executable instructions that, when executed, enable the implementation of the steps described above.
[0010] As can be seen from the above technical solution, in this embodiment, firstly, predefined programmable test cases are loaded through the test control engine. These test cases define the test sequence and expected response, and the FC endpoint simulation software is controlled to automatically execute the test sequence to interact with the FC device under test. Simultaneously, the protocol monitoring and analysis module automatically monitors the fiber channel frames transmitted during the interaction. After obtaining the monitoring results, the test control engine automatically compares the actual response with the expected response and generates a test report, achieving full automation from test execution and protocol monitoring to result judgment. Compared to traditional technologies where each test script needs to be manually written and test execution and result observation require manual intervention, this embodiment eliminates the need for manual operation and judgment, significantly improving testing efficiency. It also avoids misjudgments due to human error, ensuring the objectivity and consistency of the test results.
[0011] Furthermore, by automatically comparing the actual response with the expected response and generating a test report, the entire testing process can obtain clear test conclusions without human intervention, enabling consistency testing to run continuously in an unattended manner, effectively supporting the automated regression testing needs in agile development and continuous integration scenarios. Attached Figure Description
[0012] Figure 1 This is a schematic diagram of the architecture of the FC device automated conformance testing system provided in the embodiments of this application; Figure 2Another schematic diagram of the architecture of the FC device automated conformance testing system provided in the embodiments of this application; Figure 3 Another schematic diagram of the architecture of the FC device automated conformance testing system provided in the embodiments of this application; Figure 4 Another schematic diagram of the architecture of the FC device automated conformance testing system provided in the embodiments of this application; Figure 5 A flowchart illustrating the method provided in the embodiments of this application; Figure 6 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0013] To make the methods provided in this application easier to understand, the methods provided in this application will be described in detail below with reference to the accompanying drawings and embodiments.
[0014] First, let's explain the conformance testing of the FC protocol: In the research and development and certification of FC equipment, protocol conformance testing is a key step in verifying whether the equipment's protocol behavior conforms to FC protocol standards (such as Fibre Channel - Framing and Signaling (FC-FS)).
[0015] The content of consistency testing covers multiple dimensions: 1. Does the frame format conform to the specifications? Whether the frame format conforms to the specification, namely whether the Start Of Frame (SOF) / End Of Frame (EOF) delimiters, frame header fields (Source Identifier (S_ID), Destination Identifier (D_ID), Originator Exchange ID (OXID), etc.), and Cyclic Redundancy Check (CRC) encoding and decoding conform to the standard, and whether the frame length exceeds the limit.
[0016] 2. Is the protocol state machine transition correct? Whether the protocol state machine transitions are correct, i.e. whether the processes such as Fabric Login (FLOGI) / Port Login (PLOGI), Logout (LOGO), and Link Reset are executed correctly according to the state sequence defined in the FC protocol specification.
[0017] 3. Does the protocol interaction sequence conform to the specification? For example, upon receiving an FLOGI request, a Fabric LoginAccept (FLOGI_ACC) response should be returned within a specified time; otherwise, it is considered a timeout exception.
[0018] 4. Does the error handling meet the standard expectations? When a frame with an abnormal format or a CRC error is received, does the device return an error response according to the protocol specifications, such as Link Service Reject (LS_RJT), instead of no response or crash?
[0019] 5. Whether critical services are functioning properly, i.e., whether name server registration / query and zoning are in effect in accordance with the specifications.
[0020] Verify the protocol behavior of each of the above-mentioned functional verification devices to ensure it is consistent with the FC-FS standard definition.
[0021] In this embodiment, since FC protocol conformance testing involves multiple protocol interaction dimensions, each dimension requires verification of whether the FC device under test responds correctly according to the protocol specifications. Traditionally, these tests require testers to write scripts for each scenario, manually execute them, and observe the results. This application introduces programmable test cases, pre-defining the above test scenarios in a structured manner, allowing the system to automatically execute and determine the results.
[0022] The following detailed explanation, with reference to the accompanying drawings, illustrates in detail how the method and system provided in the embodiments of this application achieve automated conformance testing: See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the FC device automated conformance testing system provided in an embodiment of this application. Figure 1 As shown, the FC device automated conformance testing system includes: a test control engine, a protocol monitoring and analysis module, and at least one FC endpoint simulation software.
[0023] In this embodiment, the FC device under test can be a real physical FC switch or a simulated FC switch formed by FC endpoint simulation software running on a general-purpose device, depending on the testing requirements.
[0024] The test control engine, as the core scheduling unit of the system, is used to load predefined programmable test cases. The test cases define the test sequence and expected response, and control the FC endpoint simulation software to execute the test sequence during the protocol interaction with the FC device under test.
[0025] Subsequently, the protocol monitoring and analysis module monitors the Fibre Channel frames transmitted during the protocol interaction and determines the monitoring results based on the Fibre Channel frames. The test control engine obtains the monitoring results from the protocol monitoring and analysis module, compares the actual response in the monitoring results with the expected response in the test cases, and automatically generates a test report based on the comparison results.
[0026] An example of FC endpoint emulation software running on a general-purpose computing device. The FC endpoint emulation software is configured to simulate FC endpoint devices, such as initiators or targets, or switching fabrics, interacting with the FC switch under test. The initiator simulated by the FC endpoint emulation software is called the simulated initiator, the target simulated by the FC endpoint emulation software is called the simulated target, and the fabric simulated by the FC endpoint emulation software is called the simulated FC switch.
[0027] Optionally, depending on whether the FC device under test is a real FC switch or a simulated FC switch, and whether the FC endpoint device is a real FC device or a simulated FC device, different network testing environments can be configured. These include, but are not limited to, purely simulated networks consisting of a simulated initiator, a simulated FC switch, and a simulated target (see...). Figure 2 A semi-simulated network consisting of a simulated initiator, a real switch, and a simulated target (see [link]). Figure 3 ), and hybrid networks consisting of real initiators, real FC switches and simulated targets, or simulated initiators, real FC switches and real targets, etc. (see) Figure 4 ).
[0028] for Figure 2 The illustrated pure simulation network, in which the FC endpoint simulation software simulates both the initiator and target on a general-purpose computing device, and the FC device under test is a simulated FC switch running on the same general-purpose computing device. The simulated FC switch itself runs in an IP network environment, and its UDP simulation link management module directly receives UDP packets from the FC endpoint simulation software through the operating system's network protocol stack, extracts the FC frames, and delivers them to the simulated FC switch's FC protocol stack for processing. This system is entirely software-based, requiring no real FC hardware, making it suitable for early testing during the FC switch software development phase and enabling protocol consistency verification before hardware is ready.
[0029] for Figure 3The illustrated pure simulation network uses the FC switch as a physical switch. The FC endpoint simulation software communicates with the FC switch under test via an Internet Protocol (IP) network. Specifically, when simulating the Initiator, the FC endpoint simulation software internally includes a User Datagram Protocol (UDP) encapsulation and adaptation layer, a virtual hardware abstraction layer, and an FC protocol stack processing layer. When sending an FC frame, the FC protocol stack processing layer generates a complete FC frame conforming to the FC-FS standard based on the test sequence issued by the test control engine. The UDP encapsulation and adaptation layer uses this FC frame as the sole data payload of a UDP packet, encapsulates it into a UDP packet with the IP address of the FC switch under test and a predetermined UDP port number as the destination address, and then sends it out.
[0030] When the FC endpoint emulation software emulates the Target, upon receiving an FC frame, the UDP encapsulation adaptation layer identifies the emulated message based on the destination port number carried in the UDP header, strips the UDP header to extract the complete FC frame, and the virtual hardware abstraction layer simulates the interrupt behavior of the FC physical port to deliver the FC frame to the FC protocol stack processing layer. The FC protocol stack processing layer then parses the FC frame and extracts the protocol interaction content.
[0031] The FC switch under test is also configured to support UDP emulation links. The FC switch under test has an Ethernet port for out-of-band management, which is multiplexed as a data port to create a logical UDP emulation port. This logical UDP emulation port is treated as a standard logical FC port (such as F_Port) in the management view of the FC switch under test, so that FC frames sent by the FC endpoint emulation software over the UDP / IP network are processed at the FC protocol stack level of the FC switch under test as if they were received from a physical FC port.
[0032] In this way, the FC endpoint simulation software realizes FC protocol interaction with a real physical FC switch, and can simulate the protocol behavior of a real FC endpoint without installing a physical FC port on a general computing device.
[0033] for Figure 4The hybrid network shown operates as follows: the FC endpoint emulation software simulates an initiator and / or a target running on a general-purpose computing device. Simultaneously, the real physical initiator (real server HBA) and / or the real physical target (real storage device) are connected to a real physical FC switch via physical FC ports. The simulated endpoint connects to the logical UDP emulation port of the real physical FC switch via a UDP / IP network, while the real endpoint connects to the physical FC port of the real physical FC switch via a fiber optic link. The real physical FC switch forwards FC frames simultaneously between the physical FC port and the logical UDP emulation port based on FC addresses, enabling hybrid data interaction between the simulated and real endpoints within the same fabric. This approach is suitable for scenarios requiring simultaneous verification of the behavior of both simulated and real endpoints within the same network, balancing emulation flexibility with hardware realism.
[0034] All of the above network configurations can execute the consistency test methods provided in this application example.
[0035] With the above deployment method, the test control engine and protocol monitoring and analysis module run centrally on an independent test server (such as an x86 server or PC), while the FC endpoint simulation software instance runs on a general-purpose computing device. The test server uniformly schedules and manages the simulation software instances on each general-purpose computing device through the IP network, thereby decoupling the test control logic from the simulation execution logic. This facilitates the elastic expansion of the number of simulation endpoints on multiple general-purpose computing devices to support large-scale concurrent test scenarios.
[0036] It should be noted that the test control engine includes a device under test (DUT) adapter, which provides the ability to interact with the management interface of the FC device under test. The management interface includes a command-line interface (CLI), a simple network management protocol (SNMP), or a representational state transfer application programming interface (REST API). Before test execution, the DUT adapter, based on the test case configuration information, converts the parts of the test sequence involving DUT operations into an instruction format recognizable by the management interface and sends them to the DUT to complete the pre-configuration of the test environment (such as creating partitions, setting aliases, enabling or disabling specific protocol functions, etc.). During test execution, the DUT adapter can trigger specific protocol operations or state changes on the DUT according to the test sequence arrangement. After test completion, the DUT adapter collects the operating status and log information of the DUT through the management interface as supplementary data for the test report. In this way, the test control engine can adapt to different types of FC devices without requiring separate test scripts for each device.
[0037] The following is in the above Figures 1 to 4 Based on this, the specific process of the automated conformance testing method for FC devices provided in the embodiments of this application is described in detail.
[0038] See Figure 5 , Figure 5 This is a flowchart illustrating the method provided in an embodiment of this application. Figure 5 As shown, the process may include the following steps: Step S501: The test control engine loads predefined programmable test cases, which define the test sequence and expected response.
[0039] In this embodiment, test cases are pre-written and stored in a test case library in a structured language, such as JavaScript Object Notation (JSON). Each test case defines the sequence of protocol operations to be executed (i.e., the test sequence) and the expected response for each protocol operation.
[0040] For example, a conformance test case for the Fabric Login (FLOGI) login function of an FC switch can include the following test items: Test Item 1: Normal FLOGI Login Verification. The test subsequence (protocol operation) corresponding to this test item includes: the FC endpoint simulation software generates an FLOGI frame and sends it to the FC switch under test. The expected response of the test subsequence is: the expected FLOGI Accept (FLOGI_ACC) response frame is received.
[0041] Test Item 2: Name Server Registration Verification After FLOGI Login. The corresponding test sub-sequence (protocol operation) for this test item includes: After the FC endpoint emulation software completes FLOGI login, it sends a name service query request to the name server of the FC switch under test. The expected response of the test sub-sequence is: the response returned by the name server contains the FC endpoint emulation software's World Wide Name (WWN) and the corresponding Fibre Channel Identifier (FC_ID), indicating that the emulation endpoint has successfully registered with the name server.
[0042] Optionally, as an embodiment, the test case includes N test items, where N is greater than or equal to 1, and each test item corresponds to an independent protocol function verification point. Each test item has a corresponding test subsequence (i.e., at least one protocol operation that needs to be performed). The N test subsequences corresponding to the N test items constitute a test sequence. The test subsequence of at least one test item contains a protocol exception injection instruction, used to verify the processing capability of the FC device under test under abnormal conditions.
[0043] The test sequence includes at least one of the following anomaly injection instructions: constructing an illegal Fibre Channel frame header field, sending a sequence state change request that does not conform to the protocol specification, or simulating a link interruption or frame loss.
[0044] For example, continuing from the previous example, test cases can also include the following test items: Test Item 3: FLOGI login verification in case of conflict with the World Wide Name (WWN).
[0045] The test subsequence (protocol operation) corresponding to this test item includes: the FC endpoint simulation software generates an FLOGI frame carrying an existing WWN and sends it to the FC switch under test. The expected response of the test subsequence is: an FLOGI Link Service Reject (FLOGI_LS_RJT) response frame is expected to be received.
[0046] Test item 4: Verification of illegal frame header field injection.
[0047] The test subsequence corresponding to this test item includes: the FC endpoint emulation software generates an FC frame, and the test control engine, at a preset trigger time, instructs the FC endpoint emulation software to tamper with a specified field in the frame header to an illegal value. The specified field includes at least one of the following: a Routing Control (R_CTL) field, a Frame Type field, or a Source / Destination Fibre Channel Identifier (FC_ID) field, and then sends the abnormal frame to the FC switch under test. The expected response of the test subsequence is: the FC switch under test discards the illegal frame without causing an anomaly in the protocol state machine due to the illegal frame.
[0048] Test item 5: Verification of unexpected sequence state change injection.
[0049] The test subsequence corresponding to this test item includes: the FC endpoint simulation software directly sends a Port Login (PLOGI) request to the FC switch under test before FLOGI is completed, or sends an unexpected sequence state change command during sequence transmission. The expected response of the test subsequence is: the FC switch under test returns a PLOGI Link Service Reject (PLOGI_LS_RJT) response frame, and the subsequent normal FLOGI process is not affected.
[0050] Test item 6: Simulation and verification of link interruption and frame loss.
[0051] The test subsequence corresponding to this test item includes: After the FC endpoint emulation software sends an FLOGI frame, the test control engine instructs the UDP emulation link management module to simulate a link interruption before the switch responds at a preset trigger time, or to discard frames with specific sequence numbers and stop forwarding UDP packets in a specific direction within a specified time window when multiple FC frames are sent consecutively. The expected response of the test subsequence is: after the FC endpoint emulation software detects a link interruption or frame loss, it triggers a retransmission mechanism, and the FC switch under test can correctly process and restore the normal protocol interaction process after receiving the retransmitted frame.
[0052] Using the above method, a single test case can cover the verification of normal processes (FLOGI login, name server registration) as well as various protocol anomaly scenarios (WWN conflict, illegal frame header injection, unexpected sequence state changes, link interruption and frame loss). Among them, test items 4 to 6 correspond to constructing illegal Fibre Channel frame header fields, sending sequence state change requests that do not conform to protocol specifications, and simulating link interruption or frame loss in the anomaly injection instructions, respectively.
[0053] It should be noted that the test subsequences corresponding to all test items in the test case may also exclude exception injection instructions. That is, the test sequence does not include exception injection instructions and only performs automated consistency testing.
[0054] Step S502: The test control engine controls the FC endpoint simulation software to execute the test sequence during the protocol interaction with the FC device under test.
[0055] Before executing the test sequence, the test control engine needs to first determine which one or more FC endpoint emulation software instances will perform the test.
[0056] Optionally, the allocation of FC endpoint emulation software instances includes the following two implementation methods: In one implementation, the test environment is fixed, the topology is known, and it does not change frequently. Testers are assigned a fixed test environment, meaning that the actual FC switch and the general-purpose computing device running FC endpoint emulation software are physically connected to the FC switch.
[0057] Each Ethernet port of the FC switch under test is statically bound to the IP address and UDP port number of the corresponding FC endpoint emulation software instance, and this binding relationship is written to a configuration file in XML format. When the test control engine loads test cases, it directly reads this XML configuration file, obtains the connection parameters such as the IP address and UDP port number of each emulation software instance, and then determines the test operations that the FC switch and FC endpoint emulation software need to execute in the test sequence based on their respective role identifiers. The engine then distributes the required test sequence parts to the designated instances for execution. This method is suitable for scenarios where the test environment is fixed, the topology is known, and changes infrequently.
[0058] In another implementation, the test system also includes a test resource pool, which deploys multiple FC endpoint simulation software instances. Each instance runs on a general-purpose computing device and can be dynamically allocated and reclaimed to simulate different FC endpoint devices.
[0059] Based on the configuration information defined in the test cases (such as the number of emulation initiators, emulation targets, the role of each emulation endpoint, and WWN), the test control engine dynamically allocates idle FC endpoint emulation software instances from the test resource pool. After the instances start, a UDP port number is dynamically assigned to each instance, and the instance's IP address and UDP port number are automatically sent to the FC switch under test via the management interface. A corresponding logical UDP emulation port is created on the FC switch under test and established with the emulation software instance for communication. When the test cases are completed, the test control engine reclaims all allocated instance resources and releases the UDP port numbers for reuse in subsequent test cases.
[0060] Taking a test case that requires simulating 2 simulated initiators and 1 simulated target as an example, the test control engine executes the following allocation process: (1) Parse the configuration information of the test cases and determine that this test requires 2 simulated Initiator instances and 1 simulated Target instance.
[0061] (2) Query the instance status in the test resource pool, and select 2 instances as simulation initiators and 1 instance as simulation targets from the instances that are in an idle state.
[0062] (3) After the instance starts, dynamically assign the local UDP port number to each instance (e.g., 5001 for simulation Initiator1, 5002 for simulation Initiator2, and 6001 for simulation Target).
[0063] (4) Automatically send the IP address, UDP port number and corresponding role information of each instance to the FC switch under test through the management interface, create the corresponding logical UDP simulation port on the FC switch under test and establish communication pairing with the simulation software instance.
[0064] (5) Distribute the test sequence to each target instance for execution.
[0065] After the test cases have finished executing, the test control engine reclaims all allocated instance resources and releases the UDP port numbers for reuse by subsequent test cases. This approach is suitable for automated testing environments with variable test topologies and frequent switching between different test scenarios.
[0066] It should be noted that in the dynamic allocation method, the test control engine is also responsible for maintaining the status information of all instances in the test resource pool, including whether the instance is idle, which test task it is currently assigned to, the IP address of the general computing device where the instance resides, and the currently occupied UDP port number. When an instance is allocated, its status changes from "idle" to "occupied". When the test task is completed and the resources are reclaimed, its status returns to "idle".
[0067] In this embodiment, for a test subsequence containing protocol anomaly injection instructions, the test control engine controls the FC endpoint simulation software to simulate and inject protocol anomalies at preset trigger times during protocol interaction with the FC device under test. Specifically, when the protocol interaction reaches the preset trigger time, the test control engine sends an anomaly injection trigger signal to the FC endpoint simulation software. In response to the trigger signal, the FC endpoint simulation software interrupts the current normal protocol interaction process, constructs an abnormal Fibre Channel frame according to the protocol anomaly injection instructions, and sends the abnormal Fibre Channel frame to the FC device under test. Anomaly injection instructions include, but are not limited to, the anomaly injection instructions listed above.
[0068] In this way, exception injection instructions can be defined programmatically in test cases. The timing of injection, injection location, and specific exception type can all be flexibly configured in test cases. Different exception test scenarios can be freely combined without modifying test code or rebuilding the hardware environment.
[0069] In this embodiment, the test control engine sequentially controls the FC endpoint emulation software to execute each protocol operation. The FC endpoint emulation software generates FC frames conforming to the FC-FS standard according to the instructions of the test control engine, sends them to the FC device under test, and executes corresponding protocol operations according to the test sequence arrangement during the interaction process.
[0070] Step S503: The protocol monitoring and analysis module monitors the fiber channel frames transmitted during the protocol interaction process and determines the monitoring results based on the fiber channel frames.
[0071] In this embodiment, during the protocol interaction between the FC endpoint simulation software and the FC device under test, the protocol monitoring and analysis module captures User Datagram Protocol (UDP) packets carrying the Fibre Channel frames through mirroring ports or protocol sniffing. The captured UDP packets are decapsulated, the Fibre Channel frames are extracted, and the frame header fields and payloads of the Fibre Channel frames are parsed to obtain the protocol interaction type, sequence identifier, and status information. This information constitutes the monitoring results, recording all sent and received FC frames and their detailed content throughout the entire protocol interaction process.
[0072] For example, in an FLOGI login test, the protocol monitoring and analysis module captures an FC frame returned by the FC switch under test. Parsing this frame reveals that its frame type is FLOGI_ACC, its response code is "success," and its assigned FC_ID is 0x010203. The module also records the frame's arrival timestamp and the timing interval between it and the corresponding request frame. This information constitutes the monitoring results and will be used to compare them with the expected responses defined in the test cases.
[0073] Optionally, the monitoring results can be obtained in the following two ways: Method 1: After all test sequences have been executed, the protocol monitoring and analysis module continuously monitors and stores all protocol interaction data during the entire test case execution until all test sequences have been executed, at which point the test control engine obtains the complete monitoring results.
[0074] In Method 2, after each test subsequence of a test item is completed, the protocol monitoring and analysis module returns the monitoring result corresponding to that test subsequence to the test control engine. After obtaining the monitoring result corresponding to that test subsequence, the test control engine immediately compares and judges it, and then continues to execute the next test subsequence.
[0075] It should be noted that the monitoring process is independent of the test execution process, and monitoring does not affect the normal protocol interaction flow. Monitoring results are continuously accumulated during test execution, providing a data foundation for subsequent comparison and judgment.
[0076] Step S504: The test control engine obtains the monitoring results from the protocol monitoring and analysis module, compares the actual response in the monitoring results with the expected response in the test cases, and automatically generates a test report based on the comparison results.
[0077] Optionally, the comparison timing includes the following two implementation methods, which correspond to the two methods of obtaining the monitoring results in step S503: Method 1: Compare all test sequences after execution. After all test sequences have been executed, the test control engine obtains complete monitoring results from the protocol monitoring and analysis module, and then compares the actual response with the expected response for each test subsequence (i.e., each test item) to determine whether each test item passes or fails. This method is suitable for scenarios where real-time feedback is not required during testing, and only the final test conclusion is needed.
[0078] Method 2: Immediate comparison after each test subsequence is completed. After each test subsequence (i.e., each test item) is completed, the test control engine obtains the monitoring results corresponding to that test subsequence returned by the protocol monitoring and analysis module, immediately compares the actual response of the test subsequence with the expected response, and records the judgment result of the test subsequence (i.e., whether the test item passed or failed), before executing the next test subsequence. This method enables real-time judgment of the test, facilitating timely detection of anomalies during the testing process.
[0079] Regardless of the method used, the comparison process is as follows: For each test item, the response type, response code, and status field of the Fibre Channel frame in the actual response of the test subsequence are matched item by item with the corresponding fields defined in the expected response of that test subsequence. When all fields to be matched are consistent, the test subsequence is deemed to have passed (i.e., the test item has passed); when there are inconsistent fields, the name, expected value, and actual value of the inconsistent field are recorded, and the test subsequence is deemed to have failed (i.e., the test item has failed).
[0080] For example, let's compare test item 1 (normal FLOGI login verification): Assume the expected response defined in the test case is "Received FLOGI_ACC response frame, FC_ID is valid (not 0x000000 or 0xFFFFFF)," while the actual response captured by the protocol monitoring and analysis module is "Received FLOGI_ACC response frame, FC_ID=0x010203." The test control engine compares each item: frame type matches (expected FLOGI_ACC, actual FLOGI_ACC), FC_ID validity matches (expected not 0x000000 or 0xFFFFFF, actual 0x010203 is valid). Both are consistent, therefore the test item is deemed to have passed.
[0081] Furthermore, when a test case contains multiple test items, each test item is independently judged as pass / fail. The test control engine aggregates the judgment results of all test items in the same test case and generates a test summary containing the test pass rate, a list of failed test items, and the reasons for failure for that test case; for test items judged as failing, a test log containing a complete protocol interaction sequence diagram and detailed information on exception frames is generated; the test summary and the test log are output as a visual report.
[0082] In addition, when multiple test cases need to be executed, the test control engine executes each test case in sequence, generates a test report for each test case, or summarizes the judgment results of all test cases to generate a comprehensive test report.
[0083] This concludes the process. Figure 5 The process is shown below.
[0084] pass Figure 5 As shown in the flowchart, in this embodiment, firstly, predefined programmable test cases are loaded by the test control engine. These test cases define the test sequence and expected responses, and the FC endpoint simulation software automatically executes the test sequence to interact with the FC device under test. Simultaneously, the protocol monitoring and analysis module automatically monitors the transmitted fiber channel frames during the interaction. After obtaining the monitoring results, the test control engine automatically compares the actual response with the expected response and generates a test report, achieving full automation from test execution and protocol monitoring to result judgment. Compared to traditional technologies where each test script needs to be manually written and test execution and result observation require manual intervention, this embodiment eliminates the need for manual operation and judgment, significantly improving testing efficiency. It also avoids misjudgments due to human error, ensuring the objectivity and consistency of the test results.
[0085] Furthermore, by automatically comparing the actual response with the expected response and generating a test report, the entire testing process can obtain clear test conclusions without human intervention, enabling consistency testing to run continuously in an unattended manner, effectively supporting the automated regression testing needs in agile development and continuous integration scenarios.
[0086] Furthermore, because the test sequence includes protocol anomaly injection instructions, the FC endpoint simulation software can simulate and inject protocol anomalies during protocol interaction with the FC device under test. This eliminates the need for complex manual operations or dedicated analyzers for anomaly injection; instead, it can be predefined and automatically executed through test cases. This not only simplifies and repeats the construction of abnormal test scenarios but also allows for simultaneous verification of normal protocol interactions and abnormal protocol behaviors within the same test flow, expanding the scope of testing.
[0087] The methods provided in the embodiments of this application have been described above. The systems provided in the embodiments of this application are described below: As an example, see Figure 1 This application also provides an automated conformance testing system for Fibre Channel (FC) devices, comprising: a test control engine, a protocol monitoring and analysis module, and at least one FC endpoint simulation software.
[0088] The test control engine is used to load predefined programmable test cases. The test cases define the test sequence and expected response. It controls the FC endpoint simulation software to execute the test sequence during the protocol interaction with the FC device under test, obtains the monitoring results of the protocol monitoring and analysis module, compares the actual response in the monitoring results with the expected response in the test cases, and automatically generates a test report based on the comparison results. The protocol monitoring and analysis module is used to monitor the Fibre Channel frames transmitted during protocol interaction and determine the monitoring results based on the Fibre Channel frames.
[0089] As an example, the test case includes N test items, where N is greater than or equal to 1. Each test item has a corresponding test subsequence. The N test subsequences corresponding to the N test items form a test sequence. At least one test item's test subsequence contains a protocol exception injection instruction. The test control engine controls the FC endpoint simulation software to execute test sequences during protocol interaction with the FC device under test, including: The test control engine controls the FC endpoint simulation software to execute test sequences in order to simulate and inject protocol anomalies during protocol interaction with the FC device under test.
[0090] As an example, the test sequence includes at least one of the following exception injection instructions: Constructing illegal Fibre Channel frame header fields, sending sequence state change requests that do not conform to protocol specifications, and simulating link interruption or frame loss.
[0091] As one example, the testing system also includes a test resource pool, in which multiple FC endpoint simulation software instances are deployed; The test control engine controls the FC endpoint simulation software to execute test sequences during protocol interaction with the FC device under test, including: The test control engine dynamically allocates target FC endpoint simulation software instances from the test resource pool based on the test case configuration information, and distributes the test sequence to the target FC endpoint simulation software instances for execution.
[0092] As one example, the protocol monitoring and analysis module monitors the Fibre Channel frames transmitted during protocol interaction, including: The protocol monitoring and analysis module captures UDP packets carrying Fibre Channel frames through mirrored ports or protocol sniffing. The captured User Datagram Protocol (UDP) messages are decapsulated to extract the Fibre Channel frames. Parse the frame header field and payload of the Fibre Channel frame to obtain the type, sequence identifier and status information of the protocol interaction.
[0093] As an example, the test control engine automatically generates a test report based on the comparison results, specifically including: Summarize the judgment results of all test items in the same test case and generate a test summary including the test pass rate, failed test items and reasons for failure; For test items that are determined to fail, generate test logs containing complete protocol interaction timing diagrams and detailed information on abnormal frames; Output test summaries and test logs as a visual report.
[0094] As one example, the test control engine controls the FC endpoint simulation software to execute a test sequence to simulate and inject protocol anomalies during protocol interaction with the FC device under test, specifically including: When the protocol interaction reaches the preset trigger time, the test control engine sends an exception injection trigger signal to the FC endpoint simulation software. In response to the trigger signal, the FC endpoint simulation software interrupts the current normal protocol interaction process, constructs an abnormal fiber channel frame according to the protocol anomaly injection instruction, and sends the abnormal fiber channel frame to the FC device under test.
[0095] See Figure 6 , Figure 6 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 6 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.
[0096] Based on the same application concept as the above method, this application embodiment also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.
[0097] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.
[0098] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
[0099] This application can be implemented by executing several computer program code flows using an electronic device. The electronic device loads machine-executable instructions into a machine-readable storage medium and then uses its processor to read these machine-executable instructions into memory for execution. A hardware structure diagram of the electronic device, besides... Figure 6 In addition to the processor, memory, network interface, and non-volatile memory shown, electronic devices may also include other hardware depending on their actual functions, which will not be elaborated further.
Claims
1. An automated conformance testing method for Fibre Channel (FC) devices, characterized in that, The method, applied to a test system comprising a test control engine, a protocol monitoring and analysis module, and at least one FC endpoint simulation software, includes: The test control engine loads predefined programmable test cases, which define the test sequence and expected response; The test control engine controls the FC endpoint simulation software to execute the test sequence during the protocol interaction with the FC device under test; The protocol monitoring and analysis module monitors the fiber channel frames transmitted during the protocol interaction process and determines the monitoring results based on the fiber channel frames. The test control engine obtains the monitoring results from the protocol monitoring and analysis module, compares the actual response in the monitoring results with the expected response in the test case, and automatically generates a test report based on the comparison results.
2. The method according to claim 1, characterized in that, The test case includes N test items, where N is greater than or equal to 1. Each test item has a corresponding test subsequence. The N test subsequences corresponding to the N test items constitute the test sequence. At least one of the test items' test subsequences contains a protocol exception injection instruction. The test control engine controls the FC endpoint simulation software to execute the test sequence during protocol interaction with the FC device under test, including: The test control engine controls the FC endpoint simulation software to execute the test sequence in order to simulate and inject the protocol anomaly during the protocol interaction with the FC device under test.
3. The method according to claim 2, characterized in that, The test sequence includes at least one of the following anomaly injection instructions: Constructing illegal Fibre Channel frame header fields, sending sequence state change requests that do not conform to protocol specifications, and simulating link interruption or frame loss.
4. The method according to claim 1, characterized in that, The testing system also includes a testing resource pool, which deploys multiple FC endpoint simulation software instances; The test control engine controls the FC endpoint simulation software to execute the test sequence during protocol interaction with the FC device under test, including: The test control engine dynamically allocates a target FC endpoint simulation software instance from the test resource pool according to the configuration information of the test case, and sends the test sequence to the target FC endpoint simulation software instance for execution.
5. The method according to claim 1, characterized in that, The protocol monitoring and analysis module monitors the fiber channel frames transmitted during the protocol interaction process, including: The protocol monitoring and analysis module captures User Datagram Protocol (UDP) packets carrying the Fibre Channel frames through mirrored ports or protocol sniffing. The captured User Datagram Protocol (UDP) messages are decapsulated to extract the Fibre Channel frames. The frame header field and payload of the fiber channel frame are parsed to obtain the type, sequence identifier and status information of the protocol interaction.
6. The method according to claim 1, characterized in that, The comparison of the actual response with the expected response in the test case includes: The response type, response code, and status field of the Fibre Channel frame in the actual response of each test item in the test case are matched item by item with the corresponding fields defined in the expected response. The test item is considered passed when all fields to be matched are consistent. When inconsistent fields exist, record the inconsistent field name, expected value, and actual value, and determine that the test item has failed.
7. The method according to claim 1, characterized in that, The automatic generation of test reports based on the comparison results includes: The test control engine summarizes the judgment results of all test items in the same test case and generates a test summary that includes the test pass rate, failed test items and reasons for failure. For test items that are determined to fail, generate test logs containing complete protocol interaction timing diagrams and detailed information on abnormal frames; The test summary and test logs are output as a visual report.
8. The method according to claim 2, characterized in that, The simulation and injection of the protocol exception includes: When the protocol interaction reaches a preset trigger time, the test control engine sends an exception injection trigger signal to the FC endpoint simulation software. In response to the trigger signal, the FC endpoint simulation software interrupts the current normal protocol interaction process, constructs an abnormal fiber channel frame according to the protocol abnormal injection instruction, and sends the abnormal fiber channel frame to the FC device under test.
9. An automated conformance testing system for Fibre Channel (FC) devices, characterized in that, The system includes: a test control engine, a protocol monitoring and analysis module, and at least one FC endpoint simulation software; The test control engine is used to load predefined programmable test cases, which define test sequences and expected responses. It controls the FC endpoint simulation software to execute the test sequences during protocol interaction with the FC device under test, obtains the monitoring results of the protocol monitoring and analysis module, compares the actual response in the monitoring results with the expected response in the test cases, and automatically generates a test report based on the comparison results. The protocol monitoring and analysis module is used to monitor the fiber channel frames transmitted during the protocol interaction process and determine the monitoring results based on the fiber channel frames.
10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions that, when executed by a processor, cause the processor to perform the steps of the method as described in any one of claims 1 to 8.
