Firmware integration test method of disk array card, terminal and storage medium

Abstract

The invention relates to the technical field of servers, and particularly provides a firmware integration test method for a disk array card, a terminal and a storage medium, and the method comprises the steps: expanding a management command for a disk array and a volume into a script debugging command matched with a communication protocol of the disk array, and generating an execution sequence for the script debugging command based on a test demand; according to the execution sequence, script debugging commands are sequentially issued to the disk array cards with script analysis layers, and the script analysis layers are used for extracting management commands from the script debugging commands so that the firmware to be tested can execute corresponding operation; and acquiring a firmware log of the to-be-tested firmware, extracting an actual operation of the to-be-tested firmware from the firmware log, and generating a test result based on the actual operation. According to the method, the labor cost and the time cost in the testing process are saved, and the repeated workload of developers or testers is reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of servers, and in particular relates to a firmware integration test method, a terminal and a storage medium for a disk array card. Background Art

[0002] RAID disk array technology is unique in the field of data storage with its wide applicability and far-reaching potential value. The core of this technology - the RAID controller with coordinated optimization of software and hardware, realizes seamless interaction between hardware and firmware through efficient interrupt and queue management mechanisms. When the host sends Admin or IO instructions to the RAID controller, these instructions will go through an accelerated processing flow and eventually be written to the disk accurately and accurately, ensuring rapid and stable data processing.

[0003] In view of the complexity of the internal processing flow of the disk array card firmware, in order to ensure the functional integrity, reliability and high quality standards within a tight R&D cycle, we have adopted a series of comprehensive quality assurance strategies from the module level to the subsystem level, and even across subsystems. Among them, step-by-step testing is an indispensable part, with particular emphasis on detailed verification of each level. However, although unit testing can effectively verify the correctness of the internal logic of the module, it is difficult to fully cover the complete path from the host IO instruction, through hardware processing, firmware subsystem circulation, to the final execution.

[0004] To this end, we further strengthened the integrated testing of functional joints to ensure the accuracy of data transmission between the modules and subsystems of the disk array card firmware and the rigor of process control. In terms of test execution, the traditional method relies on issuing test commands through the serial port, but is limited by the fact that the serial port software in the current mainstream test platform does not support tool input and can only rely on manual operation, which not only greatly reduces the test efficiency, but also increases the risk of human error. In addition, the coupling between test steps is low, and after each test case is executed, the test environment needs to be configured for the next test case, resulting in complicated test steps. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a disk array card firmware integration test method, terminal and storage medium to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a method for testing firmware integration of a disk array card, comprising: Expand the management commands for disk arrays and volumes into script debugging commands that match the communication protocol of the disk array, and generate an execution sequence for the script debugging commands based on test requirements; Sending script debugging commands to the disk array card with the script parsing layer in sequence according to the execution order, wherein the script parsing layer is used to extract management commands from the script debugging commands so that the firmware to be tested performs corresponding operations; A firmware log of the firmware to be tested is obtained, actual operations of the firmware to be tested are extracted from the firmware log, and a test result is generated based on the actual operations.

[0007] In an optional implementation, the management commands for the disk array and volume are extended to script debugging commands that match the communication protocol of the disk array, and an execution sequence is generated for the script debugging commands based on the test requirements, including: Configure test cases and test scenarios matching the test cases based on test requirements, including setting up a disk array, creating a volume, binding a volume, using write commands in multiple write modes, unbinding a volume, deleting a volume, and deleting a disk array; The test cases are decomposed into management command sequences to be executed sequentially, and the management commands in the management command sequences are expanded into script debugging commands that match the communication protocol of the disk array.

[0008] In an optional implementation, configuring the test case and the test scenario matching the test case based on the test requirements includes: Configure a general test case, wherein the general test case includes basic operations and data structures, wherein the basic operations include a test environment building operation, a test environment detection operation, an operation stack, and a customized exception class, and save the general test case to the first database; A configuration parameter file is provided, wherein the parameter file includes test case parameters and operation information corresponding to the test case parameters, and the parameter file is saved in a second database.

[0009] In an optional implementation, configuring a general test case includes: A reverse operation stack is introduced into a general test case, and based on the reverse operation stack, a test environment after the test case is executed is restored to an initial test environment before the test case is executed.

[0010] In an optional implementation, a reverse operation stack is introduced into a general test case, and based on the reverse operation stack, a test environment after the test case is executed is restored to an initial test environment before the test case is executed, including: In the test preparation phase, a test environment is set up and the reverse commands corresponding to the management commands in the target test case are stored in the reverse operation stack; In the test execution phase, the target test cases are executed; Confirm that the target test case is executed, pop the reverse command in the reverse operation stack and execute the reverse command.

[0011] In an optional implementation, the test case is decomposed into a management command sequence to be executed sequentially, and the management commands in the management command sequence are expanded into script debugging commands that match the communication protocol of the disk array, including: The management command is extended based on the specified parameter format, and the parameter values ​​of the extended script debugging command are consistent with the corresponding parameter values ​​of the management command.

[0012] In an optional implementation, obtaining a firmware log of the firmware to be tested, extracting actual operations of the firmware to be tested from the firmware log, and generating a test result based on the actual operations includes: Generate corresponding keywords in advance based on the expected results of the script debugging commands; Use the log acquisition command to obtain the incremental firmware log of the firmware to be tested; Confirming a target script debugging command for generating the incremental firmware log based on the script debugging command execution time corresponding to the generation time of the incremental firmware log, and using a keyword corresponding to the target script debugging command as a verification keyword; Querying the incremental firmware log for verification keywords using an online editor and a command line tool for searching text strings; If all the verification keywords exist in the incremental firmware log, it is determined that the target script debugging command is executed successfully, and an execution success identification value is generated; if all the verification keywords do not exist in the incremental firmware log, an execution abnormality identification value is generated; The test case pass rate of each test scenario is counted based on the success identification value and the exception identification value.

[0013] In an optional embodiment, the method further comprises: Obtain the identification value of the current script debugging command. If the identification value is an execution success identification value, execute the next script debugging command according to the execution order; if the identification value is an execution exception identification value, restore the test scenario based on a preset reverse operation stack, and execute the next test case based on the restored test scenario.

[0014] In a second aspect, the present invention provides a firmware integration test system for a disk array card, comprising: A command extension module, used to extend the management commands for the disk array and volume into script debugging commands that match the communication protocol of the disk array, and generate an execution sequence for the script debugging commands based on the test requirements; A command execution module, used for sending script debugging commands to the disk array card with the script parsing layer in sequence according to the execution order, wherein the script parsing layer is used for extracting management commands from the script debugging commands so that the firmware to be tested performs corresponding operations; The log parsing module is used to obtain the firmware log of the firmware to be tested, extract the actual operation of the firmware to be tested from the firmware log, and generate a test result based on the actual operation.

[0015] In a third aspect, a terminal is provided, including: processor, memory, wherein: The memory is used to store computer programs. The processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned terminal method.

[0016] According to a fourth aspect, a computer storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the methods described in the above aspects.

[0017] The beneficial effects of the present invention are that the firmware integration test method, terminal and storage medium of the disk array card provided by the present invention have the characteristics of all-weather testing, and can be tested at non-working hours such as night or weekends, thereby improving resource utilization; the automated test can be executed repeatedly, saving the manpower cost and time cost in the testing process, and reducing the repetitive workload of developers or testers; the automated test can quickly provide test results, and can be used for daily build version self-testing, so as to discover and solve problems in the earliest possible research and development stage, thereby protecting the quality of the disk array card firmware and accelerating the software development cycle.

[0018] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0021] Figure 2 It is a schematic diagram of the overall framework of the RAID controller.

[0022] Figure 3 It is a schematic principle diagram of a method according to an embodiment of the present invention.

[0023] Figure 4 The figure is a schematic flow chart of host command execution of a method according to an embodiment of the present invention.

[0024] Figure 5 The figure is a schematic flow chart of command management of a method according to an embodiment of the present invention.

[0025] Figure 6 It is a schematic diagram of test case parameter management of a method according to an embodiment of the present invention.

[0026] Figure 7 It is a schematic diagram of a reverse operation stack of a method according to an embodiment of the present invention.

[0027] Figure 8 The present invention is a flowchart of a method according to an embodiment of the present invention based on a reverse operation stack.

[0028] Fig. 9 The figure is a flow chart of a read-write test of a method according to an embodiment of the present invention.

[0029] Fig.10 It is a schematic flow chart of command issuance management of a method according to an embodiment of the present invention.

[0030] Fig.11 is a schematic block diagram of a system according to an embodiment of the present invention.

[0031] Fig.12 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0034] The key terms appearing in the present invention are explained below.

[0035] RAID: The full name is Redundant Array of Independent Disks, which is a storage virtualization technology that combines multiple separate physical disk drives in different ways to form a logical disk drive, thereby achieving the purpose of improving storage capacity, read and write performance, and data security.

[0036] Firmware: Low-level software, embedded software, can be understood as the "soul" of the hardware device, controlling the basic functions and performance of the hardware device.

[0037] Namespace: An independent storage space. Each namespace has its own logical block address range. From the host's point of view, a namespace is an independent storage device. Namespace corresponds to the concept of logical volume in a disk array. Multiple different namespaces can be created on a RAID group.

[0038] SQ: The full name is Submission Queue, which is used to store host commands to be executed.

[0039] CQ: The full name is Completion Queue, which is used to store status information of completed commands.

[0040] RCW: Read Construct Write is a data writing strategy that pre-reads disk data without write requests, copies write request data to the buffer, and then uses this data to reconstruct new verification data. The RCW strategy can avoid verification data contention during write operations.

[0041] RMW: The full name is Read Modify Write, which is a data writing strategy. The advantage of the RMW strategy is that it only reads and writes the data blocks that need to be updated, rather than the entire data stripe, which can reduce disk I / O operations and improve the efficiency of write operations.

[0042] The disk array card firmware integration test method provided by the embodiment of the present invention is executed by a computer device, and accordingly, the disk array card firmware integration test system runs in the computer device.

[0043] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a firmware integration test system for a disk array card. According to different requirements, the order of the steps in the flow chart may be changed, and some may be omitted.

[0044] like Figure 1 As shown, the method includes [1]: Step 110, expanding the management commands for the disk array and the volume into script debugging commands that match the communication protocol of the disk array, and generating an execution sequence for the script debugging commands based on the test requirements; Define management command mapping: Based on the management requirements of the disk array, map common disk array and volume management commands (such as create volume, delete volume, expand volume, data migration, etc.) to specific communication protocol (such as NVMe) command sets.

[0045] Script debugging command design: Based on the characteristics of the communication protocol, a set of script debugging commands are designed. These commands not only contain the instructions required to perform management operations, but also incorporate advanced functions such as debugging information output, error handling, and status checking. Ensure that these commands can be flexibly combined to cope with complex test scenarios.

[0046] Execution sequence logic planning: According to the test plan, design a reasonable execution sequence for the script debugging commands. Considering the dependencies, data consistency requirements and test efficiency, ensure that the execution of each step can provide the necessary premise or data support for the next step.

[0047] Step 120, sending script debugging commands to the disk array card with the script parsing layer in sequence according to the execution order, wherein the script parsing layer is used to extract management commands from the script debugging commands so that the firmware to be tested performs corresponding operations; Script parsing layer configuration: Ensure that the script parsing layer built into or connected to the disk array card is correctly configured and can identify and parse the script debugging commands issued. This layer must be able to decompose complex script commands into basic communication protocol commands.

[0048] Command issuing mechanism: Establish a stable and reliable command issuing channel, and send script debugging commands to the disk array card through the channel according to the execution sequence planned in step 110. Ensure the integrity and correctness of command transmission.

[0049] Execution monitoring and feedback: During the command execution process, the response status of the disk array is monitored in real time, including the execution progress, intermediate results and final status. Feedback information is obtained through the script parsing layer or directly through the communication protocol for subsequent analysis and processing.

[0050] Step 130, obtaining a firmware log of the firmware to be tested, extracting actual operations of the firmware to be tested from the firmware log, and generating a test result based on the actual operations.

[0051] Firmware log collection: Through the log interface or special tools provided by the disk array, the log information generated by the firmware under test during the execution of the script debugging command is fully collected to ensure the integrity and accuracy of the log.

[0052] Actual operation extraction: Accurately extract the actual operation records of the firmware to be tested from the firmware log, including key information such as the success or failure of command execution, execution time, resource usage, error code, etc.

[0053] Test result generation: Based on the actual operation records extracted, a comparative analysis is conducted with the expected test objectives. This is used to evaluate whether the firmware's functional implementation meets the requirements, whether the performance is up to standard, and whether there are any potential problems or defects.

[0054] Problem recording and feedback: For any problems or anomalies found, record them in detail and sort them into categories. According to the nature and severity of the problem, formulate corresponding solutions or suggestions, and promptly feedback them to the firmware development team for repair and optimization.

[0055] Test report preparation: Prepare a detailed test report based on the test results, including test purpose, test environment, test steps, test results, problem analysis, conclusions and suggestions, etc. This provides important reference for subsequent firmware release, upgrade and maintenance work.

[0056] To facilitate understanding of the present invention, the firmware integration test method for the disk array card provided by the present invention is further described below based on the principle of the firmware integration test method for the disk array card of the present invention and in combination with the process of performing integration test on the firmware of the disk array card in the embodiment.

[0057] Take the disk array using NVMe protocol as an example. Figure 2 As shown in the figure, the hardware and firmware interact through interrupts and queues. The host sends Admin or IO commands to the RAID controller, which is accelerated and calculated and finally written to the disk.

[0058] The present invention mainly simulates the host sending Admin or IO commands by extending the manufacturer's custom commands specified in the NVMe protocol, and adds an NVMe shell module as an adaptation layer in the firmware of the disk array, which is responsible for receiving, parsing, forwarding and replying to host commands after completion. In addition, combined with the automated testing framework, the automated integrated testing of the firmware of the disk array card is realized.

[0059] Specifically, the firmware integration test method of the disk array card includes: S1. Integrated test path.

[0060] like Figure 3 As shown in the figure, based on the NVMe protocol, the management commands of the RAID group and namespcae are expanded into commands of the host's NVMe shell debug type, using a fixed parameter format (such as json). The parameter values ​​are consistent with those of the management commands, and the disk array card firmware NVMe shell adaptation layer performs parsing.

[0061] The processing flow of the test command is as follows Figure 4As shown in the figure: the host sends a command, the NVMe shell parses, the RAID controller processes internally, reports the completion of the asynchronous event to the host after the processing is completed, and the host obtains the command execution result. The execution of these commands has a strict order. The next test command can only be issued after the current command is successfully executed. To check whether the current command is successfully executed, a standard NVMe get log command can be issued from the host. The failure or incompleteness of the previous command will affect the execution of the next command, so it must be issued serially and one by one.

[0062] Specifically, in one embodiment of the present invention: Configure test cases and test scenarios matching the test cases based on test requirements, the test cases include setting up disk arrays, creating volumes, binding volumes, using write commands in multiple write modes, unbinding volumes, deleting volumes, and deleting disk arrays. The method for configuring test cases includes: configuring general test cases, the general test cases include basic operations and data structures, the basic operations include test environment construction operations, test environment detection operations, operation stacks, and customized exception classes, and saving the general test cases to a first database; configuring parameter files, the parameter files include test case parameters, and operation information corresponding to the test case parameters, and saving the parameter files to a second database.

[0063] The test cases are decomposed into management command sequences to be executed sequentially, and the management commands in the management command sequences are expanded into script debugging commands that match the communication protocol of the disk array.

[0064] Specifically, the following steps are included: 1. Configure test cases Determine the test requirements. First, clarify the test requirements based on the functional characteristics, performance requirements, and potential failure modes of the disk array. These requirements include, but are not limited to, the formation of the disk array, the creation and management of volumes, the efficiency and accuracy of data writing and reading, and the ability to recover from failures.

[0065] Design test cases. Design comprehensive and representative test cases based on test requirements. These test cases include but are not limited to: Building a disk array: Test the disk array building process under different configurations (such as RAID level, number of disks, capacity, etc.). Creating a volume: Verify whether the volume creation is successful under different conditions (such as space restrictions, RAID type, etc.). Binding a volume: Test the process of assigning a volume to a specific host or virtual machine to ensure the correctness and stability of the binding operation. Write commands using multiple write methods: Test the efficiency and data consistency of write commands through multiple methods such as random write, sequential write, and mixed write. Unbinding a volume: Verify the correctness of the volume unbinding operation to ensure data integrity and system stability. Deleting a volume: Test the volume deletion operation to check whether there is data loss or system abnormality during the deletion process. Deleting a disk array: At the end of the test, verify the disk array deletion operation to ensure that resources are released correctly.

[0066] Configure general test cases. Basic operations: including test environment construction operations (such as hardware connection, software installation, etc.), test environment detection operations (checking hardware status, software version, etc.), operation stack management (recording execution order and status), and customized exception classes (used to handle specific errors that may be encountered during testing). Data structure: define the data structure used in the test process, such as disk configuration table, volume information table, test log table, etc. Save general test cases: save the above general test cases and their related data structures to the first database for subsequent reuse and management.

[0067] Configure parameter files. Test case parameters: Define specific parameters for each test case, such as disk serial number, RAID level, volume size, write command type, etc. Operation information: Configure corresponding operation information for each parameter, describing how to operate these parameters during the test and the expected results. Save parameter files: Save parameter files and their operation information to the second database, which complements the general test cases and together constitutes a complete test case set.

[0068] 2. Decompose test cases into management command sequences Each test case is broken down into a series of management commands that are executed sequentially. These commands are arranged in the logical order of the test flow to ensure the consistency and accuracy of the test process.

[0069] Match the communication protocol: According to the communication protocol of the disk array, the management commands are expanded into script debugging commands that match the protocol. These commands need to contain sufficient parameters and format information so that the disk array can correctly parse and execute them.

[0070] Debug information: Embed debug information output statements in script debug commands to obtain command execution status and return results in real time during the test.

[0071] Error handling: Add error handling logic to script debugging commands to ensure that errors can be captured and appropriate processing measures can be taken when command execution fails.

[0072] In a specific embodiment, the host command executes as follows Figure 5 As shown, the case management module is a management module for test case parameters, the general library module is a general test case management module, and the log recording module is used to record the execution results of the issued commands. The issuance of commands is controlled one by one based on the execution results.

[0073] Specifically, Figure 6 As shown in the figure, the use case management module records the use case parameters and steps according to different use cases in different test scenarios. The general library module contains some basic operations and data structures in the test process, as well as some common operations in the use case execution process, such as setting up the test environment to determine whether the test conditions are ready, operation stack, customized exception class, etc. These common operations are extracted and encapsulated into the general library module to achieve decoupling of test process and test data.

[0074] In order to simplify the test steps and improve the test efficiency, in one embodiment of the present invention, a reverse operation stack is introduced into the general test case, and the test environment after the test case is executed is restored to the initial test environment before the test case is executed based on the reverse operation stack.

[0075] Specifically, in the test preparation phase, a test environment is set up and the reverse commands corresponding to the management commands in the target test case are saved in the reverse operation stack; in the test execution phase, the target test case is executed; after confirming that the execution of the target test case is completed, the reverse command in the reverse operation stack is popped out and the reverse command is executed.

[0076] In a specific embodiment, a stack is introduced into the general library module to record the inverse operations corresponding to the operations successfully executed in the test condition establishment phase. Once an exception occurs during the use case execution, the test condition rollback phase is entered, and the operations saved in the stack are popped out and executed in sequence. This eliminates the need to determine whether the test environment is clean before executing the next use case test, thus simplifying the test process. This ensures that the various scenario use cases are independent of each other and do not affect each other during the test process.

[0077] The automated testing process implemented by the present invention generally includes three steps: preparing test conditions, executing tests, and rolling back test conditions.

[0078] Please refer to Figure 7, build the test environment in the test_setup stage, and save the reverse commands corresponding to the successfully executed commands in the "operation stack". Specifically, if the RAID group is successfully created in the test_setup stage, the operation of deleting the RAID group is pushed into the stack; if the RAID group and the namespace are successfully created, the operations of deleting the namespace and deleting the RAID group are pushed into the stack in sequence; if the RAID group, the namespace, and the namespace are successfully created, the operations of unbinding the namespace, deleting the namespace, and deleting the RAID group are pushed into the stack in sequence.

[0079] like Figure 8 As shown, in the do_test phase, functional testing is performed according to the use case parameters described in the use case management module; In the test_teardown phase, the test condition is rolled back. Regardless of whether there are exceptions in the test_setup and do_test phases, and regardless of the exception type, as long as the stack is not empty, the saved operations in the stack are popped out and executed. This way of implementing the rollback test condition does not require additional commands to determine whether there is a created RAID group in the current test environment, whether there is a created namespace, and whether the namesapce has been successfully bound. The operation is simple.

[0080] S2. Execute the test, such as Fig. 9 shown.

[0081] (1) Deploy test cases in the test case management module. Write test cases according to the functions to be tested and generate a test case description file it_xxx_case.txt, including test case parameters, test case execution steps, and keywords that need to be observed from the firmware log during execution; (2) Add the test case to the automated testing framework and write the test case execution script test_funcN_case.py; (3) Automatically execute test cases. You can run test_funcN_case.py to test a single function for debugging, or you can run test.py to batch test all test cases. This is used for self-testing functions before version release or quality control of daily builds. Batch testing can be performed at night or on weekends. The IO read and write test process includes: creating a RAID group, initializing the RAID group, creating a namespace, binding the namespace, issuing IO commands, unbinding the namespace, deleting the namespace, and deleting the RAID group.

[0082] (4) Check the test results after the test is completed. If the execution fails, the bug scene is saved in the log recording module, and the developer can locate and solve the problem based on the firmware log.

[0083] Log analysis includes the following two aspects: Generate corresponding keywords based on the expected results of the script debugging command in advance; use the log acquisition command to obtain the incremental firmware log of the firmware to be tested; confirm the target script debugging command that generates the incremental firmware log based on the script debugging command execution time corresponding to the generation time of the incremental firmware log, and use the keywords corresponding to the target script debugging command as verification keywords; query the verification keywords from the incremental firmware log using an online editor and a command line tool that searches for text strings; if all the verification keywords exist in the incremental firmware log, determine that the target script debugging command is executed successfully, and generate an execution success identification value; if all the verification keywords do not exist in the incremental firmware log, generate an execution exception identification value; and count the test case pass rate of each test scenario based on the success identification value and the exception identification value.

[0084] Obtain the identification value of the current script debugging command. If the identification value is an execution success identification value, execute the next script debugging command according to the execution order; if the identification value is an execution exception identification value, restore the test scenario based on a preset reverse operation stack, and execute the next test case based on the restored test scenario.

[0085] In a specific embodiment, please refer to Fig.10 The logging module saves all firmware logs during the execution of the test case. The logging module has two important functions. First, it controls the rhythm of issuing test commands according to the firmware log. Second, by matching keywords from the firmware log, it observes whether the processing results of control commands and data in each subsystem and each stage meet expectations (for example, observe whether IO commands are processed according to specific RCW, RMW and other paths), making the test process more white-box and more conducive to problem location. All unexpected situations that may occur in the test scenario are divided into three categories: non-zero results, unequal values, and keywords are not included. Three different types of exceptions, RetNonZero, NotEqual, and NotContain, are customized in the general library module.

[0086] The use case execution control module and the log recording module cooperate with each other to achieve rhythmic command issuance and relative unification of different use case execution processes. In order to make the entire test process independent of the running speed of the test platform, the present invention monitors the incremental log after each command is issued in real time, matches keywords from the disk array card firmware log through sed and grep commands in the automation script, and confirms whether the IO command, control command and data transmission and flow between each subsystem are correct; combines sed and awk commands to obtain the return value of command execution from the disk array card firmware log, and if the return value of the current command is 0, the command corresponding to the next test step is issued. In this way, the rhythm of issuing commands during manual testing is simulated, which has stronger stability.

[0087] Record the execution progress and execution status of the test cases, and obtain the log data in the form of global variables. In the test initialization phase, set the number of test case PASSes for each test scenario to 0, and read the total number of test cases for each test scenario from the test case management module. Update the number of test case PASSes according to the test results, and subtract the number of PASSes from the total number of test cases to get the number of FAILs. After all test cases are executed, highlight the test results.

[0088] In some embodiments, the disk array card firmware integration test system may include multiple functional modules composed of computer program segments. The computer programs of the various program segments in the disk array card firmware integration test system may be stored in a memory of a computer device and executed by at least one processor to perform (see Figure 1 Description) Function to test the firmware integration of disk array cards.

[0089] In this embodiment, the disk array card firmware integration test system can be divided into multiple functional modules according to the functions it performs, such as Fig.11 As shown. The functional modules of the system 1100 may include: a command extension module 1110, a command execution module 1120, and a log parsing module 1130. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0090] A command extension module, used to extend the management commands for the disk array and volume into script debugging commands that match the communication protocol of the disk array, and generate an execution sequence for the script debugging commands based on the test requirements; A command execution module, used for sending script debugging commands to the disk array card with the script parsing layer in sequence according to the execution order, wherein the script parsing layer is used for extracting management commands from the script debugging commands so that the firmware to be tested performs corresponding operations; The log parsing module is used to obtain the firmware log of the firmware to be tested, extract the actual operation of the firmware to be tested from the firmware log, and generate a test result based on the actual operation.

[0091] Optionally, as an embodiment of the present invention, the management commands for the disk array and the volume are extended to script debugging commands that match the communication protocol of the disk array, and an execution sequence is generated for the script debugging commands based on the test requirements, including: Configure test cases and test scenarios matching the test cases based on test requirements, including setting up a disk array, creating a volume, binding a volume, using write commands in multiple write modes, unbinding a volume, deleting a volume, and deleting a disk array; The test cases are decomposed into management command sequences to be executed sequentially, and the management commands in the management command sequences are expanded into script debugging commands that match the communication protocol of the disk array.

[0092] Optionally, as an embodiment of the present invention, configuring a test case and a test scenario matching the test case based on a test requirement includes: Configure a general test case, wherein the general test case includes basic operations and data structures, wherein the basic operations include a test environment building operation, a test environment detection operation, an operation stack, and a customized exception class, and save the general test case to the first database; A configuration parameter file is provided, wherein the parameter file includes test case parameters and operation information corresponding to the test case parameters, and the parameter file is saved in a second database.

[0093] Optionally, as an embodiment of the present invention, configuring a general test case includes: A reverse operation stack is introduced into a general test case, and based on the reverse operation stack, a test environment after the test case is executed is restored to an initial test environment before the test case is executed.

[0094] Optionally, as an embodiment of the present invention, a reverse operation stack is introduced into a general test case, and based on the reverse operation stack, a test environment after the test case is executed is restored to an initial test environment before the test case is executed, including: In the test preparation phase, a test environment is set up and the reverse commands corresponding to the management commands in the target test case are stored in the reverse operation stack; In the test execution phase, the target test cases are executed; Confirm that the target test case is executed, pop the reverse command in the reverse operation stack and execute the reverse command.

[0095] Optionally, as an embodiment of the present invention, the test case is decomposed into a management command sequence to be executed sequentially, and the management commands in the management command sequence are expanded into script debugging commands that match the communication protocol of the disk array, including: The management command is extended based on the specified parameter format, and the parameter values ​​of the extended script debugging command are consistent with the corresponding parameter values ​​of the management command.

[0096] Optionally, as an embodiment of the present invention, obtaining a firmware log of the firmware to be tested, extracting actual operations of the firmware to be tested from the firmware log, and generating a test result based on the actual operations includes: Generate corresponding keywords in advance based on the expected results of the script debugging commands; Use the log acquisition command to obtain the incremental firmware log of the firmware to be tested; Confirming a target script debugging command for generating the incremental firmware log based on the script debugging command execution time corresponding to the generation time of the incremental firmware log, and using a keyword corresponding to the target script debugging command as a verification keyword; Querying the incremental firmware log for verification keywords using an online editor and a command line tool for searching text strings; If all the verification keywords exist in the incremental firmware log, it is determined that the target script debugging command is executed successfully, and an execution success identification value is generated; if all the verification keywords do not exist in the incremental firmware log, an execution abnormality identification value is generated; The test case pass rate of each test scenario is counted based on the success identification value and the exception identification value.

[0097] Optionally, as an embodiment of the present invention, it further includes: Obtain the identification value of the current script debugging command. If the identification value is an execution success identification value, execute the next script debugging command according to the execution order; if the identification value is an execution exception identification value, restore the test scenario based on a preset reverse operation stack, and execute the next test case based on the restored test scenario.

[0098] Fig.12 This is a schematic diagram of the structure of a terminal 1200 provided in an embodiment of the present invention. The terminal 1200 can be used to execute the firmware integration test method of the disk array card provided in an embodiment of the present invention.

[0099] The terminal 1200 may include: a processor 1210, a memory 1220 and a communication unit 1230. These components communicate via one or more buses. Those skilled in the art may understand that the server structure shown in the figure does not limit the present invention. It may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0100] The memory 1220 can be used to store the execution instructions of the processor 1210, and the memory 1220 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 1220 are executed by the processor 1210, the terminal 1200 can perform some or all of the steps in the following method embodiments.

[0101] The processor 1210 is the control center of the storage terminal, and uses various interfaces and lines to connect various parts of the entire electronic terminal. It runs or executes software programs and / or modules stored in the memory 1220, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 1210 can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0102] The communication unit 1230 is used to establish a communication channel so that the storage terminal can communicate with other terminals, receive user data sent by other terminals or send user data to other terminals.

[0103] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0104] Therefore, the present invention has the characteristics of all-weather testing, and can be tested at non-working hours such as at night or on weekends, thereby improving resource utilization; automated testing can be executed repeatedly, saving labor costs and time costs in the testing process, and reducing the repetitive workload of developers or testers; automated testing can quickly provide test results and can be used for daily build version self-testing, so as to discover and solve problems at the earliest possible research and development stage, safeguard the quality of disk array card firmware, and accelerate the software development cycle. The technical effects that can be achieved by this embodiment can be found in the description above and will not be repeated here.

[0105] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0106] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0107] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0108] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0110] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A method for testing the firmware integration of a disk array card, characterized in that: include: Expand the management commands for disk arrays and volumes into script debugging commands that match the communication protocol of the disk array, and generate an execution sequence for the script debugging commands based on test requirements; Sending script debugging commands to the disk array card with the script parsing layer in sequence according to the execution order, wherein the script parsing layer is used to extract management commands from the script debugging commands so that the firmware to be tested performs corresponding operations; A firmware log of the firmware to be tested is obtained, actual operations of the firmware to be tested are extracted from the firmware log, and a test result is generated based on the actual operations.

2. The method according to claim 1, characterized in that Expand the management commands for disk arrays and volumes into script debugging commands that match the communication protocol of the disk array, and generate an execution sequence for the script debugging commands based on test requirements, including: Configure test cases and test scenarios matching the test cases based on test requirements, including setting up a disk array, creating a volume, binding a volume, using write commands in multiple write modes, unbinding a volume, deleting a volume, and deleting a disk array; The test cases are decomposed into management command sequences to be executed sequentially, and the management commands in the management command sequences are expanded into script debugging commands that match the communication protocol of the disk array.

3. The method according to claim 2, characterized in that Configure test cases and test scenarios matching test cases based on test requirements, including: Configure a general test case, wherein the general test case includes basic operations and data structures, wherein the basic operations include a test environment building operation, a test environment detection operation, an operation stack, and a customized exception class, and save the general test case to the first database; A configuration parameter file is provided, wherein the parameter file includes test case parameters and operation information corresponding to the test case parameters, and the parameter file is saved in a second database.

4. The method according to claim 3, characterized in that Configure common test cases, including: A reverse operation stack is introduced into a general test case, and based on the reverse operation stack, a test environment after the test case is executed is restored to an initial test environment before the test case is executed.

5. The method according to claim 4, characterized in that Introducing a reverse operation stack into a general test case, and restoring the test environment after the test case is executed to the initial test environment before the test case is executed based on the reverse operation stack, including: In the test preparation phase, a test environment is set up and the reverse commands corresponding to the management commands in the target test case are stored in the reverse operation stack; In the test execution phase, the target test cases are executed; Confirm that the target test case is executed, pop the reverse command in the reverse operation stack and execute the reverse command.

6. The method according to claim 2, characterized in that Decompose the test cases into a management command sequence that is executed sequentially, and expand the management commands in the management command sequence into script debugging commands that match the communication protocol of the disk array, including: The management command is extended based on the specified parameter format, and the parameter values ​​of the extended script debugging command are consistent with the corresponding parameter values ​​of the management command.

7. The method according to claim 1, characterized in that Obtaining a firmware log of the firmware to be tested, extracting actual operations of the firmware to be tested from the firmware log, and generating a test result based on the actual operations, including: Generate corresponding keywords in advance based on the expected results of the script debugging commands; Use the log acquisition command to obtain the incremental firmware log of the firmware to be tested; Confirming a target script debugging command for generating the incremental firmware log based on the script debugging command execution time corresponding to the generation time of the incremental firmware log, and using a keyword corresponding to the target script debugging command as a verification keyword; Querying the incremental firmware log for verification keywords using an online editor and a command line tool for searching text strings; If all the verification keywords exist in the incremental firmware log, it is determined that the target script debugging command is executed successfully, and an execution success identification value is generated; if all the verification keywords do not exist in the incremental firmware log, an execution abnormality identification value is generated; The test case pass rate of each test scenario is counted based on the success identification value and the exception identification value.

8. The method according to claim 7, characterized in that The method further comprises: Obtain the identification value of the current script debugging command. If the identification value is an execution success identification value, execute the next script debugging command according to the execution order; if the identification value is an execution exception identification value, restore the test scenario based on a preset reverse operation stack, and execute the next test case based on the restored test scenario.

9. A terminal, characterized in that: include: A memory, used for storing a firmware integration test program of a disk array card; The processor is used to implement the steps of the disk array card firmware integration test method according to any one of claims 1 to 8 when executing the disk array card firmware integration test program.

10. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a firmware integration test program for a disk array card, and when the firmware integration test program for a disk array card is executed by a processor, the steps of the firmware integration test method for a disk array card as claimed in any one of claims 1 to 8 are implemented.

Images