Address translation anomaly analysis method and system and storage medium

Abstract

The embodiment of the invention provides an address translation anomaly analysis method and system and a storage medium, and belongs to the technical field of data analys.The method comprises the steps that a firmware log analysis thread is executed, and request parameter information and response data information of any admin command in a firmware log of an NVMe accelerator card are extracted; a PCIe message analysis thread is executed, any PCIe message generated by a PCIe protocol analyzer between the host and the NVMe accelerator card is extracted, and the PCIe message represents request data or response data of any admin command or request data or response data of any IO command; and according to the request parameter information and the response data information of any admin command extracted from the firmware log and the extracted any PCIe message, determining whether the NVMe accelerator card has address abnormal translation on any admin command and whether the NVMe accelerator card has address abnormal translation on any IO command. The embodiment of the invention aims to improve the analysis efficiency of the address translation error.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data analysis, and in particular, to a method, system, and storage medium for analyzing an address translation anomaly. Background Art

[0002] With the development of information technologies such as artificial intelligence, the Internet of Things, and cloud computing, the storage bandwidth requirements for data center servers are becoming increasingly higher. In order to improve the hard disk IO performance, researchers have designed many devices to accelerate the hard disk IO process, such as NVMe accelerator cards, which can accelerate the IO command execution process of the NVMe protocol, including IO command transmission and execution.

[0003] During the product development phase of NVMe accelerator cards, many host command address conversion anomalies are often encountered. For example, when the NVMe accelerator card firmware processes admin commands issued by the host, or when the NVMe accelerator card hardware processes IO commands issued by the host, problems such as PRP address conversion errors and command response information DMA address calculation errors may occur. Address conversion errors will cause abnormal behavior of the host connected to the NVMe accelerator card. For example, the host NVMe driver will execute a null pointer, further causing the host to restart.

[0004] When a host exception occurs, such as when the host NVMe driver encounters a null pointer restart, R&D personnel can compare firmware logs and PCIe message logs to analyze the memory DMA process of admin / IO commands between the host and the NVMe accelerator card one by one to identify the address conversion errors. However, due to the difficulty of debugging NVMe, the analysis of address conversion errors is difficult due to the comparison process of R&D personnel, resulting in low efficiency in address conversion error analysis. Summary of the Invention

[0005] The embodiments of the present application provide a method, system, and storage medium for analyzing address translation anomalies, aiming to improve the efficiency of analyzing address translation errors.

[0006] In a first aspect, an embodiment of the present application provides an address translation anomaly analysis method, which is applied to a computer system, wherein the computer system includes a host and an NVMe accelerator card. The method includes: Execute the firmware log analysis thread to extract the request parameter information and response data information of any admin command in the firmware log of the NVMe accelerator card; Execute a PCIe message analysis thread to extract any PCIe message generated by a PCIe protocol analyzer between the host and the NVMe accelerator card, where the PCIe message represents request data or response data of any admin command or request data or response data of any IO command; According to the request parameter information and response data information of any admin command extracted from the firmware log and any PCIe message extracted, determine whether the NVMe accelerator card has an abnormal address conversion for any admin command and whether it has an abnormal address conversion for any IO command.

[0007] Optionally, determining whether the NVMe accelerator card performs abnormal address conversion on any admin command and any IO command based on request parameter information and response data information of any admin command extracted from the firmware log and any PCIe message extracted includes: Determining, based on a message head address of any PCIe message, whether message data of the PCIe message belongs to any one of a queue base address, a queue depth, or an MSI-X address; Determine, based on the message length of any PCIe message, whether to execute any one of the MSI-X address verification process, the NVMe command request check process, the NVMe command response check process, and the PRP address verification process; The MSI-X address verification process is used to determine whether the firmware of the NVMe accelerator card is interrupted; the NVMe command request check process is used to determine whether any admin command request has an abnormal address conversion; the NVMe command response check process is used to determine whether any admin command response or any IO command response is wrong; the PRP address verification process is used to determine whether any IO command request has an abnormal address conversion.

[0008] Optionally, determining, based on a message head address of any PCIe message, whether message data of the PCIe message belongs to any one of a queue base address, a queue depth, or an MSI-X address includes: When the message head address of any PCIe message is the base register address of the request queue or response queue of any admin command, the message data of the PCIe message is used as the queue base address and saved in the queue attribute set; When the message head address of any PCIe message is the depth register address of the request queue or response queue of any admin command, the message data of the PCIe message is used as the queue depth and saved in the queue attribute set; When the message head address of any PCIe message is the MSI-X interrupt register address of the response queue of any admin command, the message data of the PCIe message is used as the MSI-X address and saved in the MSI-X address set.

[0009] Optionally, determining to execute any one of an MSI-X address verification process, an NVMe command request check process, an NVMe command response check process, and a PRP address verification process based on the message length of any PCIe message includes: When the PCIe message length is 4 bytes, the MSI-X address verification process is executed; When the PCIe message length is 64 bytes, execute the NVMe command request check process; When the PCIe message length is 16 bytes, execute the NVMe command response check process; When the PCIe message length is not any of 4, 16, and 64 bytes, the PRP address check process is executed.

[0010] Optionally, executing the MSI-X address verification process includes: Detecting whether the message head address of the PCIe message is located in the MSI-X address set; When the message head address of the PCIe message is located in the MSI-X address set, the firmware of the NVMe accelerator card triggers an MSI-X interrupt; When the message head address of the PCIe message is not in the MSI-X address set, the PRP address verification process is executed.

[0011] Optionally, executing the NVMe command request check process includes: When the message head address of the PCIe message is within the address range of any admin command request queue, the message data of the PCIe message is used as the admin command request, and the admin command request verification process is executed. The admin command request verification process includes: Converting the binary message data of the PCIe message into a target admin command request, comparing the parameter content of the target admin command request with the parameter content of any admin command request in the firmware log, and if no consistent parameter content is found, indicating that any admin command request has an abnormal address conversion; when consistent parameter content is found, determining through the opcode field of the target admin command request that the command type corresponding to the target admin command request is create IO request queue or create IO response queue, extracting the queue information and interrupt vector of the IO request queue or IO response queue from the target admin command request, and adding them to the queue attribute set; When the message head address of the PCIe message is within the address range of any IO command request queue, the message data is used as an IO command request and an IO command request verification process is executed. The IO command request verification process includes: After converting the binary message data of the PCIe message into a target IO command request, determining, according to an operation code field of the target IO command request, whether the command type of the target IO command request is a read command request or a write command request; When the command type of the target IO command request is a read command request or a write command request, obtaining a corresponding PRP table address from the target IO command request and storing it in an IO PRP address set; When the command type of the target IO command request is not a read command request or a write command request, stopping processing the PCIe message; When the message head address of the PCIe message is not within the address range of any admin command request queue and is not within the address range of any IO command request queue, the PRP address verification process is executed.

[0012] Optionally, executing the NVMe command response check process includes: When the message head address of the PCIe message is within the address range of any admin command response queue, the message data of the PCIe message is used as an admin command response, and the admin command response verification process is executed. The admin command response verification process includes: Convert the binary message data of the PCIe message into a target admin command response, extract the queue identifier and queue head pointer of the target admin command response, and check whether the queue identifier exists in the queue attribute set and whether the queue head pointer is less than the queue depth; If the check fails, it indicates that the target admin command response is wrong; if the check passes, the firmware log analysis thread is notified to read the next admin command; When the message head address of the PCIe message is within the address range of any IO command response queue, the message data is used as an IO command response, and an IO command response verification process is executed. The IO command response verification process includes: Converting the binary message data of the PCIe message into a target IO command response, extracting the queue identifier and queue head pointer of the target IO command response, and checking whether the queue identifier exists in the queue attribute set and whether the queue head pointer is less than the queue depth; If the check fails, it indicates that the target IO command response is incorrect. When the message head address of the PCIe message is not within the address range of any admin command response queue and is not within the address range of any IO command response queue, the PRP address verification process is executed.

[0013] Optionally, executing the PRP address verification process includes: When it is checked that the message head address of the PCIe message satisfies the following conditions at the same time: it is inconsistent with the PRP table address corresponding to any admin command read in the firmware log, it is inconsistent with the corresponding PRP table address obtained in the target IO command request, it is not in the admin PRP address set composed of the PRP table address of the admin command, and it is not in the IO PRP address set composed of the PRP table address of the IO command, it is determined that any IO command request has an abnormal address conversion.

[0014] Optionally, the method further includes: Count the number of times any address matches in multiple address matches in the PCIe message analysis thread respectively; Regularly updating the execution priority of the plurality of address matches according to the number of times any address is matched, wherein the more times any address is matched, the higher the execution priority; The PCIe message analysis thread is executed according to the execution priority of multiple address matches.

[0015] In a second aspect, an embodiment of the present application provides a computer system comprising a host and an NVMe accelerator card, wherein the computer system is used to execute the address translation anomaly analysis method described in the first aspect of the embodiment.

[0016] In a third aspect, an embodiment of the present application provides a non-volatile readable storage medium, wherein the non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the address translation anomaly analysis method described in the first aspect of the embodiment is executed.

[0017] Beneficial effects: This method extracts request parameter information and response data information of any admin command in the firmware log of the NVMe accelerator card by executing a firmware log analysis thread; executes a PCIe message analysis thread to extract any PCIe message generated by a PCIe protocol analyzer between the host and the NVMe accelerator card, wherein the PCIe message represents the request data or response data of any admin command or the request data or response data of any IO command; and determines whether the NVMe accelerator card has an abnormal address conversion for any admin command and whether it has an abnormal address conversion for any IO command based on the request parameter information and response data information of any admin command extracted from the firmware log and any extracted PCIe message.

[0018] Based on the request parameter information and response data information of the admin command in the NVMe accelerator card's firmware log and any PCIe message generated by the PCIe protocol analyzer, the address conversion anomaly in the NVMe admin / IO command address conversion process can be reversed and verified, thereby improving the efficiency of address conversion error analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art.

[0020] Figure 1 A schematic diagram of the process flow of an existing host command address translation exception debugging process is shown; Figure 2 A schematic diagram showing the contents of a firmware log of an existing NVMe accelerator card and a PCIe message log of a PCIe protocol analyzer is shown; Figure 3 A flowchart showing the steps of the address translation anomaly analysis method provided by an embodiment of the present application is shown; Figure 4 A flowchart of a process for executing a PCIe message analysis thread provided in an embodiment of the present application is shown; Figure 5 A flowchart of the MSI-X address verification process provided in an embodiment of the present application is shown; Figure 6 A flowchart of a process for executing an NVMe command request check provided in an embodiment of the present application is shown; Figure 7 A flowchart of a process for executing an NVMe command response check provided in an embodiment of the present application is shown; Figure 8 A flowchart of a PRP address verification process according to an embodiment of the present application is shown; Figure 9 A schematic diagram illustrating the execution order of dynamically adjusting address matching provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined with each other and referenced to each other under the premise of no contradiction.

[0023] PCIe: A hardware interface between a host and peripherals. The PCIe bus protocol is implemented through a PCIe hardware link, enabling data exchange between the host and peripherals based on the PCIe interface. It has features such as high-speed transmission, abnormal recovery, and power consumption management.

[0024] MSI: Message interrupt, an interrupt method defined by the PCIe protocol. The device can write message data to the host's MSI address to trigger the host hardware to generate an interrupt signal, thereby notifying the host NVMe device driver to perform corresponding interrupt processing.

[0025] NVMe: An application layer protocol that runs on top of the hardware PCIe protocol. It defines a set of management and IO request / response queue interfaces, enabling the host to submit IO request commands to the device and process IO responses asynchronously with high concurrency.

[0026] Admin commands: NVMe protocol commands used to manage devices, including creating queues, deleting queues, and configuring features.

[0027] IO commands: NVMe protocol commands used to read and write device memory, including read, write, format, and verify.

[0028] MSI-X: A more advanced PCIe protocol interrupt method that defines multiple MSI interrupts through the MSI table. The host NVMe driver can use this interrupt method to assign a unique MSI interrupt number to each admin / IO command response queue. When the device writes the admin / IO command response to the host command response queue, it can use different MSI interrupt numbers to trigger different MSI interrupts to the host, thereby notifying the host to read the admin / IO command response information from different admin / IO command response queues.

[0029] DMA: Direct data transfer, used to move data from one address to another through hardware outside the CPU PRP address: Physical Region Page, which indicates the memory page address of the IO data contained in the IO command.

[0030] PRP table: A PRP address table defined by the NVMe protocol that contains multiple PRP addresses. This table indicates that multiple memory page data are transferred between the host and the NVMe device through DMA at one time. Each PRP table entry is a PRP address.

[0031] Flash: A non-volatile data storage device, generally used to store data inside a storage device (such as a hard disk).

[0032] The PCIe link can provide physical transmission media, data flow control and retransmission, and various data transmission transactions based on PCIe packets. The NVMe protocol is an application-layer data storage protocol with the PCIe link as its underlying layer. NVMe is built on top of PCIe and can support one admin and up to 65,536 I / O command request / response queues. The maximum request / response queue depth can be 65,536, providing a set of simple, easy-to-use, highly concurrent, and highly reliable data transmission interfaces based on queues for application-layer software.

[0033] With the development of information technologies such as artificial intelligence, the Internet of Things, and cloud computing, the storage bandwidth requirements for data center servers are becoming increasingly higher. In order to improve the hard disk IO performance, researchers have designed many devices to accelerate the hard disk IO process, such as NVMe accelerator cards, which can accelerate the IO command execution process of the NVMe protocol, including IO command transmission and execution.

[0034] During the product development phase of NVMe accelerator cards, many host command address conversion anomalies are often encountered. For example, when the NVMe accelerator card firmware processes admin commands issued by the host, or when the NVMe accelerator card hardware processes IO commands issued by the host, problems such as PRP address conversion errors and command response information DMA address calculation errors may occur. Address conversion errors will cause abnormal behavior of the host connected to the NVMe accelerator card. For example, the host NVMe driver will execute a null pointer, further causing the host to restart.

[0035] Reference Figure 1 , shows a flow chart of the existing host command address translation abnormality debugging process. The current common NVMe accelerator card host command address translation abnormality debugging method includes the following steps: S001: The host NVMe device driver submits the admin / IO command to the admin / IO command request queue; S002: The NVMe accelerator card's firmware executes the admin command, and the hardware executes the IO command. When the firmware / hardware polls and finds that the admin / IO command request queue is not empty, it transfers the command from the host through DMA and retrieves the host's PRP address table address, data block number, flash address, and other information from the queue. S003: The NVMe accelerator card firmware / hardware uses DMA to move the PRP table from the host to the local cache, then extracts all entries from the PRP table. Each entry includes a PRP address. Based on the command parameters such as the number of data blocks and the flash address, the NVMe accelerator card firmware / hardware uses DMA to move the IO data from the host memory address pointed to by the PRP address to the local flash address. S004: The NVMe accelerator card firmware writes the response result of the admin command to the host admin command response queue through DMA, then writes the host MSI-X address to trigger an interrupt, and uses the interrupt to notify the host to further read the command response result of the admin command; S005: When processing the admin command, the firmware will record the request parameters of the admin command in the firmware log, including the PRP table address, PRP address, data block length, and flash address, as well as the command response result, including whether the command is successful, the command request queue head pointer (indicating the command address index in the "command request queue" that has been recently processed by the firmware), the command request queue identifier (indicating the unique identifier of the "command request queue" corresponding to the command response information, used to distinguish other "command request queues"), and the command identifier (indicating the unique identifier of the admin command in the "command request queue", used to distinguish other commands in the queue).

[0036] S006: During the NVMe accelerator card hardware execution of IO commands, the PCIe protocol analyzer connected between the host and the NVMe accelerator card monitors all PCIe packets and records all data transmission behaviors on the PCIe link, including monitoring the host reading and writing the device (i.e., the NVMe accelerator card register), monitoring the device reading and writing the host address (various DMA processes such as moving data according to the PRP address), and monitoring the device triggering the MSI interrupt (the NVMe accelerator card triggers the host to generate an interrupt by writing the host MSI-X address).

[0037] Reference Figure 2, showing a schematic diagram of the contents of the firmware log of an existing NVMe accelerator card and the PCIe message log of a PCIe protocol analyzer. The firmware log includes the request parameters corresponding to any admin command and the parameters in the command response result. The request parameters corresponding to the admin command include: PRP table address, PRP address, data block length, and flash address. The parameters in the command response result include whether the command is successful, the command request queue head pointer, the command request queue identifier, and the command identifier; the PCIe message log includes the parameters of any PCIe message. Each PCIe message includes the transmission direction (0 indicates reading the host memory through DMA, 1 indicates writing the host memory through DMA), the message head address (the host memory address as the DMA source / destination address), the message length (DMA length), and multiple message data fragments (representing the binary data of a DMA transfer).

[0038] When a host exception occurs, such as when the host NVMe driver encounters a null pointer restart, R&D personnel can compare firmware logs and PCIe message logs to analyze the memory DMA process of admin / IO commands between the host and NVMe accelerator card one by one, and check which command and parameter (command request, command response, PRP data, MSI, queue head / tail pointer) has an address conversion error, which further causes data to be written to the wrong host address, thereby causing the host to restart.

[0039] Based on the process of comparing each item, the R&D personnel were able to locate the abnormal host command address translation, but the following problems still exist: 1) Debugging NVMe is difficult. Due to the complexity of the NVMe and PCIe protocols, analyzing PCIe protocol messages and converting PCIe binary data into NVMe protocol commands requires in-depth protocol knowledge, making debugging more difficult. 2) Debugging issues is slow. Due to the lack of automated comparison between NVMe accelerator card firmware logs and PCIe messages, as well as the lack of automated conversion of PCIe messages into NVMe commands and checking of command execution, each command execution process requires manual analysis. When the host issues a large number of admin / IO commands, locating issues is slow. 3) Difficulty in summarizing debugging experience. Because debugging methods are not recorded and summarized, when R&D personnel are replaced, they need to re-summarize the steps for locating address anomalies of NVMe accelerator cards, which is not conducive to the dissemination of experience. 4) Debugging steps were poorly optimized. Due to a lack of review and summary of the steps for locating address anomalies in NVMe accelerator cards, R&D personnel followed the same steps to check command parameters and command execution processes one by one in the PCIe message each time they debugged the issue. Some command parameters were not the ones that frequently caused address anomalies, but due to a lack of review and summary, these parameters were frequently checked, resulting in the problem not being located in the optimal parameter checking order.

[0040] Therefore, in order to quickly analyze the problem of host command address conversion exception of the NVMe accelerator card, an address conversion exception analysis method provided in this embodiment can solve the problems of difficult debugging, slow speed and less optimization in the analysis process of host command address conversion exception of the NVMe accelerator card.

[0041] Reference Figure 3 , shows a flowchart of the steps of an address translation anomaly analysis method provided by an embodiment of the present application. The method is applied to a computer system including a host and an NVMe accelerator card. The method may include the following steps: S101: Execute the firmware log analysis thread to extract the request parameter information and response data information of any admin command in the firmware log of the NVMe accelerator card.

[0042] When processing admin commands, the firmware of the NVMe accelerator card stores the request parameter information and response data information of any admin command in the firmware log. The request parameter information includes: PRP table address, PRP address, data block length, and flash address; the response data information includes: whether the command is successful, the command request queue head pointer, the command request queue identifier, and the command identifier.

[0043] In the actual implementation process, the request parameter information and response data information of each admin command in the firmware log of the NVMe accelerator card can be read cyclically. After extracting the request parameter information and response data information of any admin command, when waiting for the verification of the PCIe message analysis thread to be completed and the next admin command to be sent, the request parameter information and response data information of the next admin command can be extracted.

[0044] S102: Execute the PCIe message analysis thread to extract any PCIe message generated by the PCIe protocol analyzer between the host and the NVMe accelerator card, where the PCIe message represents the request data or response data of any admin command or the request data or response data of any IO command.

[0045] PCIe packets are generated by the PCIe protocol analyzer between the host and the NVMe accelerator card and stored in the PCIe packet log of the PCIe protocol analyzer. When the PCIe packet analysis thread is executed, each PCIe packet is read from the PCIe packet log in a loop.

[0046] A PCIe message represents the request data or response data of any admin command or any I / O command. A PCIe message includes the transmission direction, message starting address, message length, and message data. When the transmission direction is 0, it indicates reading the host memory through DMA, and when the transmission direction is 1, it indicates writing to the host memory through DMA. The message starting address is the host memory address of the DMA source / destination address. The message length refers to the length of each DMA. The message data represents the binary data of a DMA transfer.

[0047] S103: Determine whether the NVMe accelerator card has an abnormal address conversion for any admin command and any IO command based on the request parameter information and response data information of any admin command extracted from the firmware log and any PCIe message extracted.

[0048] Reference Figure 4 , shows a processing flow chart for executing a PCIe message analysis thread provided by an embodiment of the present application. In a feasible implementation, first, based on the message head address of the PCIe message, it is determined whether the message data of the PCIe message belongs to any one of the queue base address, queue depth or MSI-X address.

[0049] Specifically, when the message head address of any PCIe message is the base register address of the request queue or response queue of any admin command, the message data of the PCIe message is used as the queue base address and saved in the queue attribute set.

[0050] When the message head address of any PCIe message is the depth register address of the request queue or response queue of any admin command, the message data of the PCIe message is used as the queue depth and saved in the queue attribute set.

[0051] When the message head address of any PCIe message is the MSI-X interrupt register address of the response queue of any admin command, the message data of the PCIe message is used as the MSI-X address and saved in the MSI-X address set.

[0052] Then, based on the relationship between the data length processed in the NVMe protocol and the executed task, it can be determined that the PCIe message is triggering an MSI-X interrupt, executing an admin command request or an IO command request, or executing an admin command response or an IO command response.

[0053] Specifically, the message data length used to trigger the MSI-X interrupt in the NVMe protocol is 4 bytes, the data length of the admin command request or IO command request is 64 bytes, the data length of the admin command response or IO command response is 16 bytes, and the remaining NVMe data transmission is all from the "host memory pointed to by the PRP address" to the device falsh address data transmission.

[0054] Therefore, according to the message length of the PCIe message, the address check sub-process that needs to be executed can be determined. The address check sub-process includes the MSI-X address check process, the NVMe command request check process, the NVMe command response check process and the PRP address check process; the MSI-X address check process is used to determine whether the firmware of the NVMe accelerator card is interrupted; the NVMe command request check process is used to determine whether any admin command request has an abnormal address conversion; the NVMe command response check process is used to determine whether any admin command response or any IO command response is wrong; the PRP address check process is used to determine whether any IO command request has an abnormal address conversion.

[0055] In a feasible implementation, determining to execute any one of the MSI-X address verification process, the NVMe command request check process, the NVMe command response check process, and the PRP address verification process based on the message length of any PCIe message specifically includes: When the PCIe message length is 4 bytes, the MSI-X address verification process is executed.

[0056] When the PCIe message length is 64 bytes, the NVMe command request check process is executed.

[0057] When the PCIe message length is 16 bytes, the NVMe command response check process is executed.

[0058] When the PCIe message length is not any of 4, 16, and 64 bytes, the PRP address check process is executed.

[0059] Reference Figure 5 , shows a flowchart of executing the MSI-X address verification process provided by an embodiment of the present application, executing the MSI-X address verification process includes: Detect whether the message head address of the PCIe message is located in the MSI-X address set; when the message head address of the PCIe message is located in the MSI-X address set, it indicates that the firmware of the NVMe accelerator card has triggered an MSI-X interrupt; when the message head address of the PCIe message is not located in the MSI-X address set, it may be that the firmware of the NVMe accelerator card is reading and writing the host RPR address data, and the PRP address verification process is executed at this time.

[0060] The NVMe command request check process includes the admin command request verification process and the IO command request verification process. Therefore, when executing the NVMe command request check process, address matching is first required to determine whether to execute the admin command request verification process or the IO command request verification process.

[0061] Specifically, the judgment is made based on whether the message first address of the PCIe message is within the address range of any admin command request queue or the address range of any IO command request queue. For example, in the queue attribute set, it is determined whether the message first address of the PCIe message is within the address range determined by the base address and depth of any admin command request queue or any IO command request queue.

[0062] When the message head address of the PCIe message is within the address range of any admin command request queue, the message data of the PCIe message is used as the admin command request, and the admin command request verification process is executed.

[0063] When the message head address of the PCIe message is within the address range of any IO command request queue, the message data is used as the IO command request and the IO command request verification process is executed.

[0064] When the message head address of the PCIe message is not within the address range of any admin command request queue and is not within the address range of any IO command request queue, the PRP address verification process is executed.

[0065] Reference Figure 6 , shows a flowchart of the NVMe command request check process provided by an embodiment of the present application, wherein the admin command request check process includes: The binary message data of the PCIe message is converted into a target admin command request according to the admin command request format defined by the NVMe protocol, and the parameter content of the target admin command request is compared with the parameter content of any admin command request in the firmware log.

[0066] If no consistent parameter content is matched, it indicates that any admin command request has an abnormal address conversion, and an error occurs in the firmware parsing the admin command. For example, the firmware parsing error of the admin command can be printed and the command field with the error can be verified.

[0067] When consistent parameter content is matched, the operation code field of the target admin command request is used to determine whether the command type corresponding to the target admin command request is to create an IO request queue or to create an IO response queue. If the target admin command request is to create an IO request queue or to create an IO response queue, the queue information and interrupt vector of the IO request queue or IO response queue are extracted from the target admin command request and added to the queue attribute set. The queue information includes the base address, depth and identifier of the IO response queue.

[0068] like Figure 6 , the IO command request verification process includes: After converting the binary message data of the PCIe message into a target IO command request according to the IO command request format defined by the NVMe protocol, it is determined that the command type of the target IO command request is a read command request or a write command request based on the operation code field of the target IO command request.

[0069] When the command type of the target IO command request is a read command request or a write command request, it means that the device will subsequently read and write data from the memory pointed to by the host PRP address, first save the PRP table address of the PRP address set, and then take out all PRP addresses from the PRP table in the subsequent process. Therefore, the corresponding PRP table address is obtained from the target IO command request and saved in the IO PRP address set.

[0070] When the command type of the target IO command request is not a read command request or a write command request, it indicates that the target IO command request is a non-read or write command. Non-read or write commands do not involve transporting PRP data, so processing of the PCIe message can be stopped.

[0071] The NVMe command response check process includes the admin command response verification process and the IO command response verification process. Therefore, when executing the NVMe command response check process, address matching is first required to determine whether to execute the admin command response verification process or the IO command response verification process.

[0072] Specifically, the judgment is based on whether the message head address of the PCIe message is within the address range of any admin command response queue or the address range of any IO command response queue. For example, in the queue attribute set, it is determined whether the message head address of the PCIe message is within the address range determined by the base address and depth of any admin command response queue or any IO command response queue.

[0073] When the message head address of the PCIe message is within the address range of any admin command response queue, the message data of the PCIe message is used as the admin command response, and the admin command response verification process is executed.

[0074] When the message head address of the PCIe message is within the address range of any IO command response queue, the message data is used as an IO command response and the IO command response verification process is executed.

[0075] When the message head address of the PCIe message is not within the address range of any admin command response queue and is not within the address range of any IO command response queue, the PRP address verification process is executed.

[0076] Reference Figure 7 , shows a flowchart of the NVMe command response check process provided by an embodiment of the present application, wherein the admin command response check process includes: The binary message data of the PCIe message is converted into a target admin command response according to the admin command response format defined by the NVMe protocol, and the queue identifier and queue head pointer of the target admin command response are extracted. It is checked whether the queue identifier exists in the queue attribute set and whether the queue head pointer is less than the queue depth.

[0077] When the check fails, it indicates that the target admin command response is wrong. In this case, the illegal admin command response content and the admin command content may be printed.

[0078] When the check passes, the firmware log analysis thread is notified to read the next admin command.

[0079] like Figure 7 , the IO command response verification process includes: The binary message data of the PCIe message is converted into a target IO command response according to the IO command response format defined by the NVMe protocol, and the queue identifier and queue head pointer of the target IO command response are extracted. It is checked whether the queue identifier exists in the queue attribute set and whether the queue head pointer is less than the queue depth.

[0080] When the check fails, it indicates that the target IO command response is wrong, and the IO command response content is illegal as well as the IO command content.

[0081] Reference Figure 8 , shows a flowchart of a PRP address verification process provided by an embodiment of the present application, wherein the PRP address verification process includes: When it is checked that the message head address of the PCIe message satisfies the following conditions at the same time: it is inconsistent with the PRP table address corresponding to any admin command read in the firmware log, it is inconsistent with the corresponding PRP table address obtained in the target IO command request, it is not in the admin PRP address set composed of the PRP table address of the admin command, and it is not in the IO PRP address set composed of the PRP table address of the IO command, it is determined that any IO command request has an abnormal address conversion.

[0082] like Figure 8 As shown, specifically, first check whether the message head address of the PCIe message is consistent with the PRP table address of the admin command read from the firmware log.

[0083] If they are consistent, the PCIe message data is added to the admin PRP address set as a PRP table entry, that is, a PRP address. Because the firmware / hardware of the NVMe accelerator card will read all PRP addresses from the host memory pointed to by the RRP table address, this process will be recorded by the firmware / PCIe message log.

[0084] If they are inconsistent, according to the NVMe protocol, the NVMe accelerator card may be reading and writing the host PRP address data, so continue to compare whether the message head address of the PCIe message is consistent with the IO PRP address obtained in the IO command request verification process.

[0085] If it is consistent with the IO PRP address, the message data is used as a PRP table entry and added to the IO PRP address set.

[0086] If it is inconsistent with the IO PRP address, check whether the message first address is in the admin PRP address set. If it is in the admin PRP address set, end; if it is not in the admin PRP address set, check whether the message first address is in the IOPRP address set.

[0087] If it is in the IO PRP address set, then end.

[0088] If it is not in the IO PRP address set, the hardware IO command address exception and the complete IO command content will be printed.

[0089] In a feasible implementation, during the execution of the PCIe message analysis thread, multiple address matches are required, including address matching when executing the main process of the PCIe message analysis thread, address matching when executing the NVMe command request check process, address matching when executing the NVMe command response check process, and address matching when executing the PRP address verification process.

[0090] Specifically, the address matching when executing the main process of the PCIe message analysis thread includes: the message first address is the base register address of the request queue of the admin command, the message first address is the base register address of the response queue of the admin command, the message first address is the depth register address of the request queue of the admin command, the message first address is the depth register address of the response queue of the admin command, the message first address is the MSI-X interrupt register address, the message length is 4, the message length is 16, the message length is 64, and the message length is not 4, 16, or 64 bytes.

[0091] The address matching during the NVMe command request check process includes: the message first address is within the address range of the admin command request queue and the message first address is within the address range of the IO command request queue.

[0092] The address matching during the NVMe command response check process includes: the message first address is within the address range of the admin command response queue and the message first address is within the address range of the IO command response queue.

[0093] The address matching during the PRP address verification process includes: the message first address is consistent with the IO PRP table address, the message first address is in the IO PRP address set, the message first address is consistent with the admin PRP table address, and the message first address is in the adminPRP address set.

[0094] That is, when executing the PCIe message analysis thread, judgment needs to be made to achieve the above-mentioned address matching. For example, for the message data of a PCIe message, when judging the relationship between the message data and the byte length, there are four matching processes: 4 bytes, 16 bytes, 64 bytes, and non-4, 16, 64 bytes. The address matching can be completed after at least one comparison, and at most four comparisons are required to complete the address matching.

[0095] Considering that different matching conditions may have different matching probabilities due to different test focuses in actual products, the address matching order defined in the initial stage is not necessarily the fastest matching order. For example: 1. When repeatedly testing the NVMe accelerator card reset, the host NVMe driver is frequently loaded due to repeated device initialization, and the device's MSI-X address register is frequently configured. Therefore, "message length of 4 bytes" is the most likely matching condition. In this case, matching "message length of 4 bytes" first can speed up the address matching process. 2. When repeatedly testing admin command transmission, since the MSI-X address register is initialized and configured only once, and most admin commands only transmit a command request (64 bytes) and a command response (16 bytes), "message length of 64 / 16 bytes" is the most likely match condition. Therefore, matching "message length of 64 / 16 bytes" first can speed up the address matching process. 3. When repeatedly testing I / O data transfers, the MSI-X address register is initialized and configured only once, and a single I / O command contains a large number of PRP data transfers. Multiple DMA transfers are required for each PRP data transfer. The length of a single DMA transfer is determined by the device's PCIeMax Payload Size register, which is generally 512 bytes. Therefore, the "message length is not 4, 16, or 64 bytes" condition is the most likely match condition. Therefore, matching "message length is not 4, 16, or 64 bytes" first can speed up the address matching process.

[0096] Therefore, in order to improve the address matching process of the PCIe message analysis thread, the matching condition execution order can be dynamically adjusted.

[0097] In a feasible embodiment, the method further includes: The number of times any one of the multiple address matches in the PCIe message analysis thread is executed is counted respectively; based on the number of times any one of the address matches is executed, the execution priority of the multiple address matches is updated periodically, wherein the more times any one of the address matches is executed, the higher the execution priority; and the PCIe message analysis thread is executed according to the execution priority of the multiple address matches.

[0098] That is, if an address match is hit more times, the probability of it being hit subsequently is greater, so the higher the execution priority is, the higher the address match is executed. Prioritizing the execution of the address match can speed up the address matching process of the PCIe message analysis thread.

[0099] Reference Figure 9, showing a schematic diagram of the execution order of dynamically adjusting address matching provided by an embodiment of the present application. For the old address matching order, the address matching order decreases in priority from top to bottom, and when executing the PCIe message analysis thread, matching is performed in order from high to low priority; for example, if the execution priorities of "message length is 4 bytes", "message length is not 4, 16, or 64 bytes", "message length is 16 bytes," and "message length is 64 bytes" decrease in sequence, then when determining to execute any one of the MSI-X address verification process, NVMe command request check process, NVMe command response check process, and PRP address verification process based on the message length of any PCIe message, it is prioritized to determine whether the message length is 4 bytes. If it is not 4 bytes, then continue to determine whether the message length is not 4, 16, or 64 bytes.

[0100] Next, the address matching when executing the main process of the PCIe message analysis thread, the address matching when executing the NVMe command request check process, the address matching when executing the NVMe command response check process, and the address matching when executing the PRP address verification process are counted respectively. The number of address matches refers to the number of successful matches or hits.

[0101] Then, at fixed intervals, the old address matching order is updated to generate a new address matching order, the PCIe message analysis thread is executed according to the new address matching order, and the number of times each address matching condition is re-counted.

[0102] The method provided in this embodiment has at least the following beneficial effects: 1. Developed an NVMe accelerator card host command abnormal address analysis tool. The tool is divided into two parts: a firmware log analysis thread and a PCIe message analysis thread. The firmware log analysis thread is used to extract and analyze the admin command request / response information of the NVMe accelerator card device firmware; the PCIe message analysis thread is used to extract and analyze the PCIe message of the PCIe protocol analyzer between the NVMe accelerator card and the host, translate the binary data of the PCIe message into NVMe IO command request / response information, and then verify whether the device firmware has errors in parsing or executing the host admin command address conversion when processing the admin command by comparing the admin command differences between the firmware log and the PCIe message log; and verify whether the device hardware has errors in executing the host IO command address conversion by comparing the IO command address range issued by the host in the PCIe message log and the IO command address range actually executed by the device hardware.

[0103] 2. Debugging is easy; it can automatically analyze PCIe messages and convert PCIe binary data into NVMe protocol commands for analysis, so debugging is easy; 3. Faster debugging speed: Because the NVMe accelerator card firmware log and PCIe messages can be automatically compared, PCIe messages can be automatically converted into NVMe commands, and the command execution process can be checked, the execution process of each command does not need to be manually analyzed. When the host issues a large number of IO commands, the speed of locating problems can be greatly improved. 4. Debugging experience is easy to summarize. Because the tool records and summarizes the experience, when R&D personnel are replaced, there is no need for the new R&D personnel to re-summarize the steps for locating address anomalies of NVMe accelerator cards. 5. The debugging steps can be dynamically optimized; the address matching order can be dynamically adjusted according to different test scenarios, thereby optimizing the execution time of the algorithm and speeding up the location of abnormal host command address conversion problems.

[0104] An embodiment of the present application also provides a computer system, which includes a host and an NVMe accelerator card, and the computer system is used to execute the address translation anomaly analysis method described in the embodiment.

[0105] An embodiment of the present application further provides a non-volatile readable storage medium, wherein the non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the address translation anomaly analysis method described in the embodiment is performed.

[0106] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0107] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0111] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0112] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0113] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for analyzing an address translation anomaly, characterized in that: Applied to a computer system, the computer system includes a host and an NVMe accelerator card, the method includes: Execute the firmware log analysis thread to extract the request parameter information and response data information of any admin command in the firmware log of the NVMe accelerator card; Execute a PCIe message analysis thread to extract any PCIe message generated by a PCIe protocol analyzer between the host and the NVMe accelerator card, wherein the PCIe message represents request data or response data of any admin command or request data or response data of any IO command; According to the request parameter information and response data information of any admin command extracted from the firmware log and any PCIe message extracted, determine whether the NVMe accelerator card has an abnormal address conversion for any admin command and whether it has an abnormal address conversion for any IO command.

2. The method according to claim 1, characterized in that According to the request parameter information and response data information of any admin command extracted from the firmware log and any PCIe message extracted, determining whether the NVMe accelerator card has an abnormal address conversion for any admin command and whether the NVMe accelerator card has an abnormal address conversion for any IO command includes: According to a message head address of any PCIe message, determining whether message data of the PCIe message belongs to any one of a queue base address, a queue depth or an MSI-X address; According to the message length of any PCIe message, determine to execute any one of the MSI-X address verification process, the NVMe command request check process, the NVMe command response check process and the PRP address verification process; The MSI-X address verification process is used to determine whether the firmware of the NVMe accelerator card is interrupted; the NVMe command request check process is used to determine whether any admin command request has an abnormal address conversion; the NVMe command response check process is used to determine whether any admin command response or any IO command response is wrong; the PRP address verification process is used to determine whether any IO command request has an abnormal address conversion.

3. The method according to claim 2, characterized in that According to the message head address of any PCIe message, determining whether the message data of the PCIe message belongs to any one of the queue base address, the queue depth or the MSI-X address includes: When the message head address of any PCIe message is the base address register address of the request queue or response queue of any admin command, the message data of the PCIe message is used as the queue base address and saved in the queue attribute set; When the message head address of any PCIe message is the depth register address of the request queue or response queue of any admin command, the message data of the PCIe message is used as the queue depth and saved in the queue attribute set; When the message head address of any PCIe message is the MSI-X interrupt register address of the response queue of any admin command, the message data of the PCIe message is used as the MSI-X address and saved in the MSI-X address set.

4. The method according to claim 3, characterized in that According to the message length of any PCIe message, determining to execute any one of the MSI-X address verification process, the NVMe command request check process, the NVMe command response check process and the PRP address verification process includes: When the PCIe message length is 4 bytes, the MSI-X address verification process is executed; When the PCIe message length is 64 bytes, the NVMe command request check process is executed; When the PCIe message length is 16 bytes, the NVMe command response check process is executed; When the PCIe message length is not any one of 4, 16, and 64 bytes, the PRP address check process is executed.

5. The method according to claim 4, characterized in that Executing the MSI-X address verification process includes: Detecting whether the message head address of the PCIe message is located in the MSI-X address set; When the message head address of the PCIe message is located in the MSI-X address set, it indicates that the firmware of the NVMe accelerator card triggers an MSI-X interrupt; When the message head address of the PCIe message is not in the MSI-X address set, the PRP address verification process is executed.

6. The method according to claim 4, characterized in that The NVMe command request check process includes: When the message head address of the PCIe message is within the address range of any admin command request queue, the message data of the PCIe message is used as the admin command request, and the admin command request verification process is executed. The admin command request verification process includes: Convert the binary message data of the PCIe message into a target admin command request, compare the parameter content of the target admin command request with the parameter content of any admin command request in the firmware log, and if no consistent parameter content is matched, it indicates that any admin command request has an abnormal address conversion; when consistent parameter content is matched, determine through the operation code field of the target admin command request that the command type corresponding to the target admin command request is to create an IO request queue or to create an IO response queue, extract the queue information and interrupt vector of the IO request queue or the IO response queue from the target admin command request, and add them to the queue attribute set; When the message head address of the PCIe message is within the address range of any IO command request queue, the message data is used as an IO command request, and an IO command request verification process is executed, wherein the IO command request verification process includes: After converting the binary message data of the PCIe message into a target IO command request, judging, according to the operation code field of the target IO command request, whether the command type of the target IO command request is a read command request or a write command request; When the command type of the target IO command request is a read command request or a write command request, obtaining a corresponding PRP table address from the target IO command request and storing it in an IO PRP address set; When the command type of the target IO command request is not a read command request or a write command request, stop processing the PCIe message; When the message head address of the PCIe message is not within the address range of any admin command request queue and is not within the address range of any IO command request queue, the PRP address verification process is executed.

7. The method according to claim 4, characterized in that Executing the NVMe command response check process includes: When the message head address of the PCIe message is within the address range of any admin command response queue, the message data of the PCIe message is used as the admin command response, and the admin command response verification process is executed. The admin command response verification process includes: Convert the binary message data of the PCIe message into a target admin command response, extract the queue identifier and queue head pointer of the target admin command response, and check whether the queue identifier exists in the queue attribute set and whether the queue head pointer is less than the queue depth; When the check fails, it indicates that the target admin command response is wrong; if the check passes, the firmware log analysis thread is notified to read the next admin command; When the message head address of the PCIe message is within the address range of any IO command response queue, the message data is used as an IO command response, and an IO command response verification process is executed, wherein the IO command response verification process includes: Convert the binary message data of the PCIe message into a target IO command response, extract the queue identifier and queue head pointer of the target IO command response, and check whether the queue identifier exists in the queue attribute set and whether the queue head pointer is less than the queue depth; When the check fails, it indicates that the target IO command response is wrong; When the message head address of the PCIe message is not within the address range of any admin command response queue and is not within the address range of any IO command response queue, the PRP address verification process is executed.

8. The method according to claim 7, characterized in that The PRP address verification process includes: When it is checked that the message head address of the PCIe message satisfies the following conditions at the same time: it is inconsistent with the PRP table address corresponding to any admin command read in the firmware log, it is inconsistent with the corresponding PRP table address obtained in the target IO command request, it is not in the admin PRP address set composed of the PRP table address of the admin command, and it is not in the IO PRP address set composed of the PRP table address of the IO command, it is determined that any IO command request has an abnormal address conversion.

9. The method according to claim 8, characterized in that The method further comprises: Count the number of times any address matches in multiple address matches in the PCIe message analysis thread are executed respectively; According to the number of times any address is matched, the execution priority of the multiple address matches is updated regularly, wherein the more times any address is matched, the higher the execution priority; The PCIe message analysis thread is executed according to the execution priority of multiple address matches.

10. A computer system, characterized in that: The computer system includes a host and an NVMe acceleration card, and the computer system is used to execute the address conversion anomaly analysis method described in any one of claims 1-9.

11. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, wherein the computer program, when executed by a processor, performs the address translation anomaly analysis method according to any one of claims 1 to 9.

Images