HDD firmware evaluation system and HDD firmware evaluation method

Abstract

The process of generating firmware from source code and creating performance evaluation reports on the HDD side can be achieved without allocating human resources. [Solution] The HDD firmware development method comprises a source code storage means 101 for storing the developed source code, a firmware creation means 102 for creating firmware from the source code, an HDD performance information acquisition means 103 for controlling the HDD with the created firmware and acquiring HDD performance information, an HDD performance evaluation report creation means 104 for creating an HDD performance evaluation report, and an overall control means 105 for overall control of storing and accessing the source code in the source code storage means 101, controlling the firmware creation means 102, the HDD performance information acquisition means 103, and the HDD performance evaluation report creation means 104, and creating and outputting an HDD performance evaluation report when the HDD is controlled by the firmware based on the source code.

Description

Technical Field

[0001] This invention relates to an HDD (Hard Disk Drive) firmware development system as storage and an HDD firmware development method.

Background Art

[0002] Conventionally, in HDD firmware development, operations such as introducing evaluation FW (firmware) to the HDD, setting up a performance measurement environment, and aggregating evaluation results were carried out manually by the evaluation personnel. Therefore, when attempting to perform performance evaluations of the latest FW frequently, it was necessary to constantly allocate human resources for the settings and aggregation.

[0003] The firmware responsible for controlling the HDD is an important component that affects the performance of the HDD. To evaluate the performance of the HDD, it is necessary to actually install the firmware on the HDD device after creating the evaluation FW and operate it in the same state as the actual product. An example of the performance evaluation procedure in HDD development is as follows.

[0004] (1) First, the FW developer modifies the source code of the firmware and stores the work product on the server. (2) Next, the FW creator obtains the source code work product from the server, and (3) the FW creator creates the evaluation FW from the source code (the timing for releases to customers, etc. is indefinite). (4) Further, the FW evaluator downloads the evaluation FW to the HDD to be evaluated, and (5) the FW evaluator connects the HDD with the evaluation FW introduced to the evaluation environment and performs a performance evaluation. (6) Finally, the FW evaluator aggregates the performance evaluation results.

[0005] In order to detect performance problems early, it is desirable to perform performance evaluations each time the source code is modified. However, the timing of firmware creation as described in (3) above is unpredictable, making frequent evaluation difficult. Furthermore, downloading evaluation firmware to the HDDs under evaluation as described in (4) above, and performing performance evaluations using the performance evaluation application as described in (5) above, require manual configuration operations, which, along with the aggregation of evaluation results as described in (6), hinders the frequency of evaluations.

[0006] Furthermore, conventional performance evaluations only yielded performance values ​​recorded by the performance measurement application on the measurement environment side, leaving the internal state of the HDD unknown. Therefore, even if performance evaluations are performed more frequently and performance problems are detected early, the lack of information makes analyzing the causes of performance degradation difficult, and it is expected that many evaluation results awaiting analysis will accumulate over time.

[0007] Patent Document 1 discloses a method for shortening the firmware evaluation time in an information processing device. The data management unit of this device stores the first firmware in the first storage unit via the data control unit and the switching unit. To activate the first firmware, the command operation unit restarts the processing unit, switches the connection of the switching unit, evaluates the first firmware, and stores the second firmware in the second storage unit during the evaluation. By processing firmware updates and evaluations in parallel, multiple firmwares can be evaluated in a short time.

[0008] Patent Document 2 discloses a firmware verification system that can efficiently verify the firmware parameters of multiple servers without building a dedicated management server or network. In this system, a system management means equipped with a storage area for storing firmware information and a sharing means for enabling access to the storage area, and a firmware verification means equipped with a connection means for connecting to the storage areas of multiple evaluation devices are communicated together via a network. The firmware verification means obtains verification information from the shared storage area of ​​the master evaluation device after running a verification program and modifies its own firmware settings.

[0009] Patent Document 3 discloses a firmware evaluation system comprising a firmware system having an API unit that stores at least one API (hardware control function), an input / output unit that receives an evaluation string from a PC for evaluating the hardware control function and outputs evaluation result information to the PC, and an evaluation user interface unit that uses the API. According to this, it is possible to provide a firmware evaluation system that can further improve the convenience when evaluating the hardware control function of firmware. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] International Publication No. 2013-145292 [Patent Document 2] Japanese Patent Publication No. 2014-096032 [Patent Document 3] Japanese Patent Publication No. 2007-219893 [Overview of the project] [Problems that the invention aims to solve]

[0011] Although efforts have been made to improve the convenience of conducting evaluations as described above, there was no method that could create firmware from source code, control the HDD with the created firmware to obtain HDD performance information, and generate an HDD performance evaluation report based on this HDD performance information without constantly allocating human resources.

[0012] In the embodiments of the present invention, the objective is to provide a firmware evaluation system and a firmware evaluation method that enable the creation of firmware from source code and the generation of a performance evaluation report on the HDD side without allocating human resources. [Means for solving the problem]

[0013] A firmware evaluation system according to an embodiment of the present invention is characterized by comprising: a source code storage means for storing source code developed for firmware responsible for controlling an HDD; a firmware creation means for creating firmware from the source code; an HDD performance information acquisition means for controlling the HDD with the firmware created by the firmware creation means and acquiring HDD performance information; an HDD performance evaluation report creation means for creating an HDD performance evaluation report based on the HDD performance information; and a central control means for controlling access to the source code storage means, the firmware creation means, the HDD performance information acquisition means, and the HDD performance evaluation report creation means, from the time the source code is stored in the source code storage means, to the output of an HDD performance evaluation report when the HDD is controlled by firmware based on the source code stored in the source code storage means.

[0014] A firmware evaluation method according to an embodiment of the present invention is characterized by comprising: a source code storage step for storing source code developed for firmware responsible for controlling an HDD; a firmware creation step for creating firmware from the source code; an HDD performance information acquisition step for controlling the HDD with the firmware created in the firmware creation step and acquiring HDD performance information; an HDD performance evaluation report creation step for creating an HDD performance evaluation report based on the HDD performance information; and a comprehensive control step that controls access to the source code storage step, the firmware creation step, the HDD performance information acquisition step, and the HDD performance evaluation report creation step, from the time the source code is stored in the source code storage step, to the output of the HDD performance evaluation report when the HDD is controlled by the firmware based on the source code stored in the source code storage step. [Brief explanation of the drawing]

[0015] [Figure 1] A diagram illustrating the configuration of an embodiment of the firmware evaluation system according to the present invention. [Figure 2] A diagram illustrating the configuration of an embodiment in which the firmware evaluation system according to the present invention is configured using a server. [Figure 3] A configuration diagram showing a performance measurement server, which is a means for acquiring HDD performance information in the firmware evaluation system according to the present invention, configured with servers. [Figure 4] A diagram showing an example of the processing content accumulated in the execution queue by the central control means of the firmware evaluation system according to the present invention. [Figure 5] A figure showing an example of a report consisting of numerical sequences output by the firmware evaluation system according to the present invention. [Figure 6] This figure shows an example of a graph-based report output by the firmware evaluation system according to the present invention. [Modes for carrying out the invention]

[0016] Referring to the accompanying drawings, a firmware evaluation system and a firmware evaluation system according to the present invention will be described. In each figure, the same components are denoted by the same reference numerals, and redundant explanations are omitted. FIG. 1 shows a configuration diagram of an embodiment of the firmware evaluation system according to the present invention.

[0017] As shown in FIG. 1, this system has a configuration in which a source code input means 100, a source code storage means 101, a firmware creation means 102, an HDD performance information acquisition means 103, an HDD performance evaluation report creation means 104, and a general control means 105 are connected to a network 200.

[0018] The source code storage means 101 can be constituted by a repository server 101A as shown in FIG. 2, the firmware creation means 102 can be constituted by a build server 102A as shown in FIG. 2, the HDD performance information acquisition means 103 can be constituted by a performance measurement server 103A as shown in FIG. 2, the HDD performance evaluation report creation means 104 can be constituted by an evaluation result storage / report creation server 104A as shown in FIG. 2, and the general control means 105 can be constituted by a performance shuttle run control server 105A as shown in FIG. 2. Further, the source code input means 100 can be constituted by a FW developer computer terminal 100A as shown in FIG. 2.

[0019] The source code storage means 101 stores the source code developed for the firmware responsible for controlling the HDD, and the developed source code is input by the FW developer from the source code input means 100. The firmware creation means 102 creates firmware from the above source code. The HDD performance information acquisition means 103 acquires HDD performance information by controlling the HDD with the firmware created by the firmware creation means 102. The HDD performance evaluation report creation means 104 creates an HDD performance evaluation report based on the HDD performance information.

[0020] The overall control means 105 performs access to the source code storage means 101, controls the firmware creation means 102, the HDD performance information acquisition means 103, and the HDD performance evaluation report creation means 104 after the source code is stored in the source code storage means 101, and based on the firmware according to the source code stored in the source code storage means 101, overall controls until the creation and output of the HDD performance evaluation report when controlling the HDD.

[0021] Note that the performance measurement server 103A, which is the HDD performance information acquisition means 103, is composed of a plurality of performance measurement servers 103A-1, 103A-2,... each equipped with a computer and an HDD, as shown in FIG. 3. The FW to be evaluated is downloaded to each server, and HDD performance information is acquired using FIO (Flexible I / O Tester) as a tool for measuring the performance of the storage system. The HDD performance information has values such as the queuing number, Read / Write error, seek occurrence number, retry occurrence number, etc. These internal state information on the HDD side is managed in units of measurement items, and then held by the HDD performance evaluation report creation means 104 in order to create the HDD performance evaluation report.

[0022] The central control means 105 can be implemented using tools that enable continuous integration (CI), continuous delivery (CD), and continuous deployment, such as Jenkins, and continuously controls processing using the execution queue shown in Figure 4. Specifically, when the source code developed for the firmware responsible for controlling the HDD is stored in the source code storage means 101 from the FW developer's FW developer computer terminal 100A, the central control means 105 adds the processing contents of "source code checkout" TA1 to the execution queue, the processing contents of "change difference check" TA2 to the execution queue, the processing contents of "FW build" TA3 to the execution queue, the processing contents of "FW download" TA4 to the execution queue, the processing contents of "performance measurement" TA5 to the execution queue, the processing contents of "report creation" TA6 to the execution queue, the processing contents of "performance confirmation" TA7 to the execution queue, and the processing contents of "performance degradation alarm generation" TA8 to the execution queue.

[0023] Next, the central control means 105 processes the data in the order it was received. First, in accordance with the processing content of "source code checkout" TA1, the central control means 105 reads the corresponding source code from the repository server 101A, which is the source code storage means 101, and provides it to the build server 102A to perform the "change difference check" TA2 process, which starts the firmware build process, which is the processing content of "FW build" TA3, and waits for it to complete. Once the build server 102A completes the build, it obtains the built firmware from the build server 102A and downloads it to multiple performance measurement servers 103A-1, 103A-2, ... as the processing content of "FW download" TA4, and then instructs the multiple performance measurement servers 103A-1, 103A-2, ... that have downloaded the firmware to perform performance measurement as the processing content of "performance measurement" TA5.

[0024] Multiple performance measurement servers 103A-1, 103A-2, ..., which have received instructions to perform performance measurements, perform performance measurements and obtain the measurement results in performance logs as values ​​such as the number of queued items, read / write errors, number of seek occurrences, and number of retries. The central control means 105 then has the evaluation result storage / report creation server 104A manage this internal state information of the HDDs on a measurement item basis.

[0025] The central control means 105 issues a report creation instruction to the evaluation result storage / report creation server 104A, which obtains and manages the measurement results, at the timing of the execution queue of "report creation" TA6. As already explained, the measurement results are obtained and managed in the performance log as values ​​such as the number of queued items, read / write errors, number of seek occurrences, and number of retry occurrences, so a report is created by arranging these values ​​in chronological order as a numerical column or as a graph. When the central control means 105 receives notification from the evaluation result storage / report creation server 104A that the report creation is complete, it gives the evaluation result storage / report creation server 104A an output instruction from the required output device.

[0026] Figure 5 shows an example of a report with numerical columns. Along with the ID of the firmware evaluated and the date and time of the evaluation, it lists the number of queued items, read / write errors, number of seeks, number of retries, etc. Measurement results are recorded in the performance log and output as a report, making it possible to analyze the internal operation of the HDD when performance degrades, and to address performance degradation caused by the same factor all at once.

[0027] In this embodiment, not only internal state information on the HDD side but also performance changes as seen from a computer connected to the HDD can be acquired by the performance measurement servers 103A-1, 103A-2, ..., stored in the HDD performance evaluation report creation means 104, and then created and output as a report. Figure 6(A) is a graph output as a report showing the time-series change in transfer rate (performance change as seen from a computer connected to the HDD) during performance measurement using firmware with FWID def0001 and firmware with FWID def0002.

[0028] In contrast, Figure 6(B) is a report output showing a graph illustrating the time-series change in the number of seeks, which is a change in the internal state of the HDD, during performance measurements using firmware with FWID def0001 and firmware with FWID def0002.

[0029] In this way, not only can the internal state information of the HDD be output, but also a performance change report as seen from the computer connected to the HDD. Compared to the previous situation where data taken from two locations, the HDD and the computer connected to the HDD, had to be checked separately, it is now possible to visualize and display (output) the data in the same time series. In Figure 6, it can be seen that the peak in the transfer rate in Figure 6(A) and the peak in the number of seek occurrences in Figure 6(B) occur at roughly the same time. This allows us to predict a correlation in the data, and is expected to have the effect of capturing the first event in the analysis of performance degradation factors at an early stage.

[0030] In this example, "number of seeks" was used as an internal state information of the HDD, and "transfer rate" was used as an example of performance change as seen from the computer connected to the HDD. However, other internal state information such as "error rate," "read / write retry count," "amount of data stored in cache memory," and "cache hit rate" can also be used. This makes it possible to determine which part of the HDD's internal operation contributed to the actual performance as seen from the host PC, based on the changes over time.

[0031] In this embodiment, the processing contents of "Report Creation" TA6 are added to the execution queue, and the processing contents of "Performance Verification" TA7 are also added to the execution queue. As performance degradation factors accumulate, the processing contents of "Performance Verification" TA7 are performed using these, and if performance degradation is determined, the processing contents of "Performance Degradation Alarm Generation" TA8 are executed. The performance degradation alarm generation can be performed by sending alarm generation instruction information from the central control means 105 to the FW developer computer terminal 100A, which then performs the action.

[0032] When source code is stored in the source code storage means 101, the overall control means 105 places the processing details into the queue shown in Figure 4, and processing proceeds quickly. As a result, performance evaluation is performed immediately at the time of source code modification, realizing a desirable execution environment from source code modification to performance evaluation that can be achieved without allocating human resources. [Explanation of symbols]

[0033] 100 Source code input means 100A Developer Computer Terminal 101 Source code storage means 101A Repository Server 102 Firmware Creation Method 102A Build Server 103 HDD performance information acquisition means 103A-1, 103A-2, ... Performance measurement servers 104 Method for Creating HDD Performance Evaluation Reports 104A Evaluation Result Storage and Report Generation Server 105 Overall control means 105A Performance Shuttle Run Control Server 200 Networks

Claims

1. A source code storage means for storing the source code developed for the firmware responsible for controlling the HDD, A firmware creation means for creating firmware from the aforementioned source code, HDD performance information acquisition means that controls the HDD using the firmware created by the firmware creation means and acquires HDD performance information, An HDD performance evaluation report creation means for creating an HDD performance evaluation report based on the aforementioned HDD performance information, A central control means controls access to the source code storage means, the firmware creation means, the HDD performance information acquisition means, and the HDD performance evaluation report creation means, from the time the source code is stored in the source code storage means, to the creation and output of an HDD performance evaluation report when the HDD is controlled by the firmware based on the source code stored in the source code storage means. An HDD firmware development system characterized by comprising the following features.

2. The HDD firmware development system according to claim 1, characterized in that the source code storage means, firmware creation means, HDD performance information acquisition means, HDD performance evaluation report creation means, and overall control means are each configured by a server, and these servers are connected by a network.

3. The HDD firmware development system according to claim 2, characterized in that the HDD performance information acquisition means is composed of multiple servers.

4. The HDD firmware development system according to claim 2, characterized in that the overall control means accesses the source code storage means after the source code has been stored in the source code storage means, reads the developed source code, and provides it to the firmware creation means to perform firmware creation.

5. The HDD firmware development system according to claim 4, characterized in that when firmware is created by the firmware creation means, the central control means provides the created firmware to the HDD performance information acquisition means to acquire HDD performance information.

6. The HDD firmware development system according to claim 5, characterized in that the overall control means controls the HDD performance information to be provided to the HDD performance evaluation report creation means when HDD performance information is acquired by the HDD performance information acquisition means.

7. The HDD firmware development system according to claim 6, characterized in that when the HDD performance information is provided to the HDD performance evaluation report creation means, the overall control means causes the HDD performance evaluation report creation means to create an HDD performance evaluation report and output the created HDD performance evaluation report.

8. A source code storage step for storing the source code developed for the firmware responsible for controlling the HDD, A firmware creation step that generates firmware from the aforementioned source code, The HDD performance information acquisition step involves controlling the HDD using the firmware created in the firmware creation step and acquiring HDD performance information. An HDD performance evaluation report creation step, which creates an HDD performance evaluation report based on the aforementioned HDD performance information, A comprehensive control step controls the process from the time the source code is stored in the source code storage step, through access to the source code storage step, the firmware creation step, the HDD performance information acquisition step, and the HDD performance evaluation report creation step, up to the output of the HDD performance evaluation report when the HDD is controlled by the firmware based on the source code stored in the source code storage step. A method for developing HDD firmware, characterized by comprising the following:

9. The HDD firmware development method according to claim 8, characterized in that the source code storage step, the firmware creation step, the HDD performance information acquisition step, the HDD performance evaluation report creation step, and the overall control step are each executed on a server, and these servers are connected by a network.

10. The HDD firmware development method according to claim 9, characterized in that the HDD performance information acquisition step is performed by multiple servers.

11. The HDD firmware development method according to claim 9, characterized in that the overall control step reads the source code developed by accessing the source code storage step after the source code has been stored in the source code storage step, and provides it to the firmware creation step to perform firmware creation.

12. The HDD firmware development method according to claim 11, characterized in that, once firmware is created by the firmware creation step, the overall control step provides the created firmware to the HDD performance information acquisition step to acquire HDD performance information.

13. The HDD firmware development method according to claim 12, characterized in that the overall control step controls the HDD performance information to be provided to the HDD performance evaluation report creation step when HDD performance information is obtained by the HDD performance information acquisition step.

14. The HDD firmware development method according to claim 13, characterized in that when the HDD performance information is provided to the HDD performance evaluation report creation step, the overall control step causes the HDD performance evaluation report creation step to create an HDD performance evaluation report and output the created HDD performance evaluation report.

Images