Request processing method and device
By obtaining the status information of the log file in the DPU and performing exception recovery, the problem that the upper-level application cannot run normally when the DPU read and write exception occurs, and the rapid exception recovery of the DPU and the stable operation of the upper-level application are achieved.
Patent Information
- Application Number
- CN202411523612.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-02-14
AI Technical Summary
When a read and write exception occurs in the data processor (DPU), the upper application and DPU need to be restarted, causing the upper application to fail to run normally.
When a read and write exception occurs, the DPU obtains the status information recorded by the log file, determines the target read and write request that needs to be recovered abnormally, and performs abnormal recovery based on the request.
Through exception recovery, the DPU can quickly restore read and write exceptions, avoiding the restart of the upper-level application, so that it can still run normally when the DPU read and write exception occurs.
Smart Images
Figure CN119440915A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a request processing method and device. Background Art
[0002] Some electronic devices have a data processing unit (DPU), which can replace the central processing unit (CPU) to provide infrastructure layer services such as network virtualization, storage virtualization, and hardware resource pooling, thereby freeing up more CPU resources to run upper-layer applications.
[0003] In some devices, the DPU can receive read and write requests from upper-layer applications and transmit the read and write requests to the device's memory to meet the upper-layer applications' needs for reading and writing data.
[0004] When the DPU is running, read / write exceptions may occur due to abnormal read / write requests or software crashes. In related technologies, when a DPU read / write exception occurs, both the upper-layer application and the DPU need to be restarted, causing the upper-layer application to be unable to run normally. Summary of the Invention
[0005] To this end, this application discloses the following technical solutions:
[0006] The first aspect of the present application provides a request processing method, comprising:
[0007] When a read / write exception occurs, the data processor obtains status information recorded in a log file, wherein the log file includes multiple read / write requests received by the data processor before the read / write exception occurs and status information corresponding to each read / write request;
[0008] The data processor obtains a target read / write request that needs to be recovered from an abnormality in the log file according to the status information;
[0009] The data processor performs exception recovery according to the target read / write request.
[0010] Optionally, also include:
[0011] The data processor records the received read and write requests in the log file;
[0012] The data processor configures status information corresponding to the read and write requests in the log file according to the processing progress of the read and write requests recorded in the log file.
[0013] Optionally, the log file includes a request linked list, and the request linked list includes multiple data structures for recording read and write requests;
[0014] The data processor records the received read and write requests in the log file, including:
[0015] When receiving a read / write request, the data processor determines a target data structure for recording the read / write request in the request linked list;
[0016] The data processor records the read and write requests in the target data structure.
[0017] Optionally, the data processor configures status information corresponding to the read / write request in the log file according to the processing progress of the read / write request recorded in the log file, including:
[0018] In response to sending the received read / write request to the storage backend, the data processor configures the status information corresponding to the read / write request to a committed state;
[0019] The data processor configures the status information corresponding to the read / write request to a completion status in response to receiving feedback information corresponding to the read / write request sent by the storage backend;
[0020] The target read / write request that needs to be abnormally recovered includes a read / write request whose status information is a submitted status.
[0021] Optionally, the data processor performs exception recovery according to the target read / write request, including:
[0022] The data processor sends the target read and write request to the storage backend;
[0023] Also includes:
[0024] In response to receiving feedback information corresponding to the target read / write request sent by the storage backend, the status information of the target read / write request is updated so that the status information indicates that the target read / write request has completed abnormal recovery.
[0025] Optionally, also include:
[0026] The data processor checks whether the number of read and write requests recorded in the log file meets the target condition;
[0027] If the target condition is not met, the data processor sends a prompt message to the target program that issues the read / write request, so as to trigger the target program to restart;
[0028] When the target condition is met, the data processor is executed to obtain a target read / write request that requires abnormal recovery in the log file according to the status information.
[0029] Optionally, the target conditions include:
[0030] The sum of the number of read and write requests recorded in the log file and the number of read and write requests stored in the hardware queue of the data processor is consistent with the number of read and write requests obtained by the data processor.
[0031] Optionally, also include:
[0032] If the log file does not exist, create a log file;
[0033] Configure the file parameters of the log file.
[0034] Optionally, the log file includes a status linked list and a target index value, and the status linked list includes a plurality of data structures for recording status information corresponding to the read and write requests;
[0035] The method further comprises:
[0036] Initializing the state linked list in the created log file so that each data structure in the state linked list is associated with a subsequent data structure;
[0037] The target index value in the created log file is initialized so that the target index value points to the first data structure of the status linked list.
[0038] A second aspect of the present application provides an electronic device, including a memory and a data processor;
[0039] The memory stores a computer program;
[0040] The data processor executes the computer program and performs the following steps according to the computer program:
[0041] When a read / write exception occurs, status information recorded in a log file is obtained, where the log file includes multiple read / write requests received by the data processor before the read / write exception occurs and status information corresponding to each of the read / write requests;
[0042] Obtaining, according to the state information, a target read / write request requiring abnormal recovery in the log file;
[0043] Perform exception recovery according to the target read / write request. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.
[0045] Figure 1 This is a flowchart of a request processing method provided by an embodiment of the present application;
[0046] Figure 2 This is a schematic diagram of the connection relationship of a data processor provided in an embodiment of the present application;
[0047] Figure 3 This is a schematic diagram of recording read and write requests in a log file provided by an embodiment of the present application;
[0048] Figure 4 This is a schematic diagram of an update status linked list and target index value provided by an embodiment of the present application;
[0049] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0051] This application embodiment provides a request processing method, see Figure 1 , is a flowchart of the method, which may include the following steps.
[0052] S101, when a read / write exception occurs, the data processor obtains status information recorded in a log file, where the log file includes multiple read / write requests received by the data processor before the read / write exception occurs and status information corresponding to each read / write request.
[0053] See Figure 2 Read and write requests can be sent to the data processing unit (DPU) by upper-layer applications that need to read and write data from the storage backend. Upper-layer applications can include applications running on various virtual machines on the host. The storage backend can include one or more memories connected to the DPU.
[0054] The occurrence of a read / write exception may include a crash of the DPU software running on the DPU, or an abnormal read / write request received by the DPU from an upper-layer application.
[0055] When a read / write exception occurs, the DPU can be restarted, and the DPU can start executing the method of this embodiment every time it is started. Optionally, the processing method provided in this embodiment can be executed by the DPU software running on the DPU.
[0056] In this embodiment, the DPU can pre-create a log file. During DPU operation, it can record received read and write requests in real time in this log file. Furthermore, while processing received read and write requests, it can update the read and write request status information in real time in the log file based on the processing progress. Therefore, if a read or write exception occurs and causes the DPU software to restart, the DPU can obtain multiple read and write requests received up to the time of the read or write exception, as well as their status information, from this log file.
[0057] S102: The data processor obtains a target read / write request that needs to be recovered from an abnormality in the log file according to the status information.
[0058] In S102 , the DPU may determine the read and write requests that require abnormal recovery and are recorded in the log file as target read and write requests, and read these target read and write requests from the log file.
[0059] In this embodiment, read / write requests requiring exception recovery may include read / write requests that have not yet been processed by the DPU. After the DPU sends a read / write request to the storage backend, the storage backend may send feedback information corresponding to the read / write request to the DPU. This feedback information indicates that the storage backend has received the corresponding read / write request. When the DPU receives feedback information from the storage backend regarding a read / write request, it can determine that the read / write request has been processed.
[0060] In this embodiment, the read and write requests that have not been processed by the DPU may include read and write requests that the DPU has sent to the storage backend but has not yet received corresponding feedback information from the storage backend.
[0061] S103: The data processor performs abnormal recovery according to the target read / write request.
[0062] The DPU can perform exception recovery by sending the target read and write request to the storage backend. After each target read and write request is sent to the storage backend and feedback information for each target read and write request is received from the storage backend, the DPU can determine that the exception recovery is complete.
[0063] After the abnormal recovery is completed, the DPU can run according to the normal process. At this time, the DPU continues to receive read and write requests from upper-layer applications and forwards the received read and write requests to the storage backend.
[0064] Optionally, if the DPU determines in S102 based on the status information that there is no target read or write request that requires abnormal recovery in the log file, it may also directly run according to the normal process.
[0065] The beneficial effects of this embodiment are:
[0066] When a read / write exception occurs, the target read / write requests requiring exception recovery are obtained from the log file based on the status information recorded in the log file, and then exception recovery is performed based on these target read / write requests. Through the method of this embodiment, the DPU can quickly complete exception recovery when a read / write exception occurs, thereby avoiding restarting the upper-layer application and allowing the upper-layer application to continue operating normally even when the DPU experiences a read / write exception.
[0067] The DPU can determine whether a log file exists at each startup. If the log file does not exist, the DPU can create a log file and then configure the log file parameters.
[0068] As an example, the log file created by the DPU may include the following information:
[0069] Structjournal
[0070] Int version;
[0071] Int status;
[0072] uint32 num;
[0073] Int shmfd;
[0074] uint32 size;
[0075] Char shm_path
[256] ;
[0076] structjournal_chain*chain;
[0077] Union
[0078] uint64 reserve
[32] ;
[0079] Struct{
[0080] Uint16 is_nvme;
[0081] Uint16 nvme_head;
[0082] NvmeCmd*nvme_cmd_cache;
[0083] };
[0084] };
[0085] };
[0086] Among them, the file parameters of the log file may include version, status, num, shmfd, size, is_nvme and shm_path
[256] . Version represents the file version number of the log file, status represents the status of the log file, and the status of the log file can be a trusted state or an untrusted state. Num represents the total number of all read and write requests recorded in the log file. shmfd represents the storage address of the log file in the memory. shm_path
[256] represents the storage path of the log file. Size represents the amount of data in the log file. For example, size equal to 20 can represent that the log file has a size of 20MB. Is_nvme represents whether the read and write requests stored in the log file are read and write requests in the nvme command format.
[0087] Based on the created log file, the DPU can update the log file as follows while receiving and processing read and write requests:
[0088] The data processor records received read and write requests in a log file;
[0089] The data processor configures the status information corresponding to the read and write requests in the log file according to the processing progress of the read and write requests recorded in the log file.
[0090] In some embodiments, the log file may include a request linked list, which may include multiple data structures for recording read and write requests. In conjunction with the aforementioned example, *nvme_cmd_cache is equivalent to the request linked list in the log file. By adding different index values, the data structures in the linked list for recording read and write requests can be obtained. For example, nvme_cmd_cache[0] represents the first data structure in the request linked list, nvme_cmd_cache[1] represents the second data structure in the request linked list, and so on.
[0091] Based on the request list, the DPU can record read and write requests in the log file in the following ways:
[0092] When a read or write request is received, the data processor determines the target data structure for recording the read or write request in the request linked list;
[0093] The data processor records the read and write requests in the target data structure.
[0094] Optionally, the target data structure may be determined by the DPU obtaining a target index value in the log file and determining a data structure corresponding to the target index value in the request linked list as the target data structure.
[0095] In the aforementioned example, nvme_head can represent the target index value. Therefore, each time the DPU receives a read or write request, it can obtain the data structure nvme_cmd_cache[nvme_head] corresponding to the target index value in the request list as the target data structure, and record the received read or write request in the target data structure.
[0096] The DPU may update the target index value each time a read or write request is recorded, so that the next read or write request received may be recorded in another data structure of the request linked list.
[0097] In some embodiments, the log file may further include a status linked list, which may include multiple data structures for recording status information corresponding to read and write requests.
[0098] Combined with the previous example, *chain is equivalent to the status linked list of the log file. Similar to the request linked list, chain[0] can represent the first data structure of the status linked list, chain[1] can represent the second data structure of the status linked list, and so on.
[0099] As an example, each data structure of the state linked list may include the following information:
[0100] structjournal_chain{
[0101] Uint32 nvme_index_status;
[0102] };
[0103] The variable nvme_index_status can be used to record the status information of read and write requests.
[0104] Each data structure in the status linked list corresponds to a data structure with the same index value in the request linked list, i.e., chain[0] corresponds to nvme_cmd_cache[0], and chain[1] corresponds to nvme_cmd_cache[1]. The variable nvme_index_status contained in the data structure of the status linked list can be used to record the status information of the corresponding read and write requests in the request linked list. For example, the status information of the read and write requests recorded by nvme_cmd_cache[1] can be recorded in the variable nvme_index_status contained in chain[1].
[0105] In some optional embodiments, in order to reduce the amount of data in the log file and save storage space, the DPU can use part of nvme_index_status to record the status information of the corresponding read and write requests, and use another part of nvme_index_status to record the index values of other data structures in the status list, so that the various data structures in the status list are associated with each other through the index values recorded by nvme_index_status.
[0106] In combination with the above example, since nvme_index_status is a 32-bit integer variable, the DPU can use bits 1 to 16 of nvme_index_status to represent the index values of other data structures in the status list, and use bits 17 to 32 to represent the status information of the corresponding read and write requests.
[0107] When the log file includes the request list and status list, the DPU can initialize the created log file as follows when creating the log file:
[0108] Initialize the state linked list in the created log file so that each data structure in the state linked list is associated with the subsequent data structure;
[0109] Initialize the target index value in the created log file so that the target index value points to the first data structure of the status linked list and the first data structure of the request linked list.
[0110] That is to say, the DPU can initialize the target index value to 0 when creating a log file, that is, nvme_head=0. At this time, chain[nvme_head] and nvme_cmd_cache[nvme_head] are both the first data structures in the corresponding linked lists.
[0111] At the same time, the DPU can initialize the index value of the nvme_index_status record of each data structure in the status list to the index value pointing to the subsequent data structure, so that each data structure in the status list is associated with the subsequent data structure.
[0112] Illustratively, the index value of the nvme_index_status record of each data structure in the initialized status linked list may be represented by the following Table 1.
[0113] Table 1
[0114]
[0115] In Table 1, chain[0].nvme_index_status represents the nvme_index_status variable contained in chain[0], max represents the total number of data structures contained in the status linked list, and k is any integer from 3 to max-1.
[0116] When initializing the log file, the DPU may set the portion representing status information in each nvme_index_status to an arbitrary initial value. For example, bits 17 to 32 of each nvme_index_status may be set to 0.
[0117] Taking upper-layer applications including virtual machine applications as an example, based on the created and initialized log file, the DPU can obtain read and write requests sent by the upper-layer application and record the read and write requests in the log file as follows:
[0118] The DPU can communicate with each virtual machine based on the NVMe specification. Based on this specification, the virtual machine can identify the DPU as an NVMe controller. NVMe is the abbreviation of NVM Express, and this specification is also called the Non-Volatile Memory Host Controller Interface Specification.
[0119] The virtual machine application can send read and write requests to the NVMe driver layer of the virtual machine to which it belongs according to the NVMe command (NVMe cmd) format specified by the NVMe specification. The NVMe driver layer can write the read and write requests to the submission queue (SQ) of the virtual machine to which it belongs.
[0120] At the same time, the DPU contains a field-programmable gate array (FPGA). When there are read and write requests in the SQ, the FPGA can pull read and write requests from the SQ and store them in the hardware queue maintained by the FPGA. Each time the FPGA stores a read or write request in the hardware queue, it increments the head pointer of the hardware queue by 1.
[0121] When there are read and write requests in the hardware queue, the DPU can pull the read and write requests from the hardware queue. Each time a read and write request is pulled, the DPU can increment the tail pointer of the hardware queue by 1;
[0122] Every time a read or write request is pulled from the hardware queue, the DPU can record the read or write request in the request list of the log file and send the read or write request to the storage backend.
[0123] After issuing a read / write request, the DPU can configure the status information of the read / write request as a submitted status (submit) in the status list of the log file. That is, the data processor can send the received read / write request to the storage backend and configure the status information corresponding to the read / write request as a submitted status.
[0124] Each time the storage backend receives a read / write request, it can send a corresponding feedback message to the DPU to indicate that the storage backend has received the read / write request. Each time the DPU receives feedback information corresponding to a read / write request, it can notify the FPGA to send the feedback information to the virtual machine corresponding to the read / write request and simultaneously configure the status information corresponding to the read / write request in the status linked table to the completed state. In other words, in response to receiving feedback information corresponding to the read / write request sent by the storage backend, the data processor can configure the status information corresponding to the read / write request to the completed state.
[0125] For example, the DPU pulls a read / write request 1 sent by virtual machine A from the hardware queue, records the read / write request 1 in the log file, sends the read / write request 1 to the storage backend, and configures the status information corresponding to the read / write request 1 in the status linked list to the submitted state. Subsequently, the DPU receives feedback information 1 corresponding to the read / write request 1 sent by the storage backend, notifies the FPGA to send the feedback information 1 to virtual machine A, and configures the status information corresponding to the read / write request 1 in the status linked list to the completed state.
[0126] Optionally, after determining that the FPGA has successfully sent the feedback information to the virtual machine, the DPU can also configure the status information of the read / write request corresponding to the feedback information in the status linked table to be in an idle state (free). For example, in the above example, after determining that the FPGA has successfully sent feedback information 1 to virtual machine A, the status information of read / write request 1 can be configured to be in an idle state.
[0127] The virtual machine may have a completion queue (CQ) for receiving feedback information. When sending feedback information, the FPGA may send the feedback information to the CQ of the virtual machine.
[0128] Based on the above-mentioned configuration status information method, the aforementioned target read and write requests that need to be recovered from an exception may include read and write requests whose status information is a committed status.
[0129] That is to say, in the processing method of this embodiment, the DPU can determine one by one whether the status information recorded in each data structure of the status linked list is a committed status. If it is a committed status, the read and write request corresponding to the committed status is read from the request linked list as the target read and write request, and the target read and write request is sent to the storage backend.
[0130] For example, assuming that the DPU finds that the status information represented by bits 17 to 32 of the nvme_index_status variable in chain[2] in the status linked list is the commit status, the DPU reads the read and write request recorded in nvme_cmd_cache[2] in the request linked list as the target read and write request, and sends the target read and write request to the storage backend.
[0131] The DPU can record a read or write request in the log file as follows:
[0132] After pulling a read or write request from the hardware queue, the DPU obtains the data structure of the request list, namely nvme_cmd_cache[nvme_head], based on the current target index value nvme_head of the log file, and then records the pulled read or write request in the nvme_cmd_cache[nvme_head] data structure. Next, the DPU finds the corresponding data structure chain[nvme_head] in the status list and obtains the index value recorded in the data structure in the status list, for example, the index value recorded by the 1st to 16th bits of the nvme_index_status variable in chain[nvme_head] is obtained, and the obtained index value is assigned to the target index value nvme_head of the log file, thereby completing the operation of recording the read or write request in the log file.
[0133] For example, see Figure 3 Before recording the read and write request, nvme_head is equal to 15. The DPU can record a read and write request pulled in nvme_cmd_cache
[15] , and then obtain the index value recorded in chain
[15] . Assuming that the index value of the 1st to 16th bits of the nvme_index_status variable in chain
[15] is 16, that is, chain
[15] .nvme_index_status=16, then the DPU assigns 16 to nvme_head, making nvme_head change from 15 to 16.
[0134] The log file can be specifically stored in the memory. Recording a read / write request in the log file is equivalent to storing the read / write request in the memory. After recording a read / write request, the DPU can obtain the read / write request based on the storage address of the read / write request in the memory. When the status information of a read / write request needs to be configured, the DPU can calculate the index value index corresponding to the read / write request as follows.
[0135] index=(nvme_cmd_addr-nvme_cmd_cache[0])>>sizeof(nvme_cmd_cache[0]).
[0136] Where nvme_cmd_addr represents the memory address of the read / write request, nvme_cmd_cache[0] represents the memory address of the first data structure in the request list, and sizeof(nvme_cmd_cache[0]) represents the memory size occupied by each data structure in the request list. >> This is equivalent to a division operation, that is, nvme_cmd_addr - nvme_cmd_cache[0] is divided by sizeof(nvme_cmd_cache[0]). The index value corresponding to the read / write request calculated in this way is the index value of the data structure in the request list used to record the read / write request.
[0137] For example, the index corresponding to read / write request 1 calculated in the above manner is 5, indicating that read / write request 1 is recorded in the addr-nvme_cmd_cache[5] data structure of the request linked list.
[0138] After obtaining the index corresponding to the read / write request, the DPU can use the index to update the status corresponding to the read / write request in the status linked list. For example, after the DPU sends a read / write request 1 to the storage backend, it can obtain the chain[5] data structure corresponding to the read / write request 1 in the status linked list and configure the status information of the data structure to the submitted status. For example, bits 17 to 32 of the variable nvme_index_status in the data structure are configured as the value representing the submitted status. After the DPU receives the feedback information 1 corresponding to the read / write request 1, it can also change the status information recorded in the chain[5] data structure from the submitted status to the completed status in the above manner.
[0139] Each time the DPU configures the status information corresponding to a read or write request as a completed state, the DPU may update the log file as follows.
[0140] The DPU obtains the index value index of the read / write request in the above manner, obtains the data structure chain[index] corresponding to the index value in the status linked list based on the index value, updates the index value recorded in the data structure so that the index value recorded in the data structure is equal to the current target index value nvme_head, and then updates the target index value so that the target index value is equal to the index value index of the read / write request.
[0141] by Figure 4 As an example, assume that when the status information of read / write request 1 is configured as completed, the target index value nvme_head of the log file is equal to 7, and the index of read / write request 1 is determined to be 5. Then, Figure 4As shown in (2), the data structure chain[5] is obtained, and the 1st to 16th bits of the nvme_index_status variable in chain[5] are updated to binary 7, that is, chain[5].nvme_index_status=7. Then, the target index value is changed to the index of the read / write request 1, that is, the target index value is changed from 7 to 5, nvme_head=index=5.
[0142] Optionally, when the DPU performs exception recovery based on the target read / write request, it can also perform the following steps:
[0143] In response to receiving feedback information corresponding to the target read / write request sent by the storage backend, the status information of the target read / write request is updated so that the status information indicates that the target read / write request has completed abnormal recovery.
[0144] The state indicating that the target read / write request has completed exception recovery may be the aforementioned idle state (free). That is, during exception recovery, the DPU sends target read / write requests to the storage backend one by one. At the same time, each time the DPU receives feedback information corresponding to a target read / write request from the storage backend, it may update the status information of the target read / write request from the submitted state to the idle state in the status linked list.
[0145] On the other hand, when the DPU obtains the target read / write requests according to the status information in S102, it can simultaneously determine the read / write requests whose corresponding status information is completed, and change the status information of these read / write requests from completed to idle.
[0146] In some optional embodiments, the DPU may perform the following steps before performing step S102:
[0147] The data processor checks whether the number of read and write requests recorded in the log file meets the target conditions;
[0148] If the target conditions are not met, the data processor sends a prompt message to the target program that issued the read / write request to trigger the target program to restart;
[0149] If the target conditions are met, step S102 is executed.
[0150] The number of read and write requests recorded in the log file can include the total number of read and write requests with a status of Committed and Completed. For example, if a log file contains 10 read and write requests with a status of Committed and 5 with a status of Completed, the number of read and write requests recorded in the log file can be 15.
[0151] The target program for issuing the read / write request may include an upper-layer application corresponding to the read / write request in the submitted state recorded in the log file.
[0152] For example, if the status information of three read and write requests in the log file is in the committed state, two of which are issued by application 1 and one is issued by application 2, the DPU can send prompt information to application 1 and application 2 to trigger the restart of application 1 and application 2.
[0153] The DPU can also configure the state of the log file based on whether the target conditions are met. If the target conditions are met, the log file state can be set to a trusted state. If the target conditions are not met, the log file state can be set to an untrusted state. When the DPU creates a log file, the log file state can be initialized to a trusted state.
[0154] The DPU may make errors when recording read and write requests and updating status in the log file, resulting in incorrect information being recorded in the log file. By checking in the above manner, it is possible to promptly detect whether incorrect information is recorded in the log file and stop performing exception recovery based on the log file when incorrect information is recorded, thus avoiding other problems caused by using the wrong log file for exception recovery.
[0155] Optionally, target conditions may include:
[0156] The sum of the number of read and write requests recorded in the log file and the number of read and write requests stored in the hardware queue of the data processor is consistent with the number of read and write requests obtained by the data processor.
[0157] The DPU may obtain the head pointer and tail pointer of the FPGA hardware queue, and subtract the two to determine the number of read and write requests stored in the hardware queue, for example, determining that the hardware queue stores 10 read and write requests.
[0158] The DPU can interact with the virtual machine to determine how many read and write requests the DPU has pulled from the virtual machine's submission queue SQ. The number of read and write requests pulled from the SQ is equivalent to the number of read and write requests obtained by the data processor. For example, it can be assumed that the DPU has pulled 20 read and write requests from the SQ.
[0159] After obtaining the above number, the number of read and write requests recorded in the log file and the number of read and write requests stored in the hardware queue can be added. If the total is equal to the number of read and write requests obtained by the data processor, it can be determined that the target condition is met. If the total is not equal to the number of read and write requests obtained by the data processor, it can be determined that the target condition is not met.
[0160] Combined with the previous example, if the number of read and write requests recorded in the log file is equal to 10, then 10+10=20, and the number of read and write requests recorded in the log file meets the target condition. If the number of read and write requests recorded in the log file is equal to 12, then 12+10 is not equal to 20, and the number of read and write requests recorded in the log file does not meet the target condition.
[0161] The present application also provides an electronic device. Figure 5 , the device may include a memory 501 and a data processor 502 .
[0162] The memory stores the computer program;
[0163] The data processor executes the computer program and performs the following steps according to the computer program:
[0164] When a read / write exception occurs, status information recorded in a log file is obtained. The log file includes multiple read / write requests received by the data processor before the read / write exception occurs and status information corresponding to each read / write request.
[0165] According to the status information, the target read and write requests that need to be recovered from the abnormality in the log file are obtained;
[0166] Perform exception recovery based on target read and write requests.
[0167] The working principle of the electronic device of this embodiment can refer to the relevant steps of the request processing method disclosed in the above embodiment, and will not be repeated here.
[0168] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.
[0169] For the convenience of description, the above systems or devices are described as being divided into various modules or units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0170] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.
[0171] Finally, it should be noted that, in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.
[0172] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A request processing method, comprising: When a read / write exception occurs, the data processor obtains status information recorded in a log file, wherein the log file includes multiple read / write requests received by the data processor before the read / write exception occurs and status information corresponding to each of the read / write requests; The data processor obtains, according to the state information, a target read / write request that needs to be abnormally recovered in the log file; The data processor performs exception recovery according to the target read / write request.
2. The method according to claim 1, further comprising: The data processor records the received read and write requests in the log file; The data processor configures the status information corresponding to the read / write request in the log file according to the processing progress of the read / write request recorded in the log file.
3. The method according to claim 2, wherein the log file comprises a request linked list, and the request linked list comprises a plurality of data structures for recording read and write requests; The data processor records the received read and write request in the log file, including: When receiving a read / write request, the data processor determines a target data structure in the request linked list for recording the read / write request; The data processor records the read and write requests in the target data structure.
4. The method according to claim 2, wherein the data processor configures the status information corresponding to the read / write request in the log file according to the processing progress of the read / write request recorded in the log file, comprising: In response to sending the received read / write request to the storage backend, the data processor configures the status information corresponding to the read / write request as a committed status; The data processor configures the status information corresponding to the read / write request as a completion status in response to receiving feedback information corresponding to the read / write request sent by the storage backend; The target read / write request that needs to be abnormally recovered includes a read / write request whose status information is a committed status.
5. The method according to claim 1, wherein the data processor performs abnormal recovery according to the target read / write request, comprising: The data processor sends the target read and write request to the storage backend; Also includes: In response to receiving feedback information corresponding to the target read / write request sent by the storage backend, the status information of the target read / write request is updated so that the status information indicates that the target read / write request has completed abnormal recovery.
6. The method according to claim 1, further comprising: The data processor checks whether the number of read and write requests recorded in the log file meets the target condition; When the target condition is not met, the data processor sends a prompt message to the target program that sends the read / write request to trigger the target program to restart; When the target condition is met, the data processor is executed to obtain a target read / write request step in the log file that requires abnormal recovery based on the status information.
7. The method according to claim 6, wherein the target condition comprises: The sum of the number of read and write requests recorded in the log file and the number of read and write requests stored in the hardware queue of the data processor is consistent with the number of read and write requests obtained by the data processor.
8. The method according to claim 4, further comprising: If the log file does not exist, create a log file; Configure the file parameters of the log file.
9. The method according to claim 8, wherein the log file comprises a state linked list and a target index value, wherein the state linked list comprises a plurality of data structures for recording state information corresponding to the read and write requests; The method further comprises: Initialize the state chain list in the created log file so that each data structure in the state chain list is associated with a subsequent data structure; The target index value in the created log file is initialized so that the target index value points to the first data structure of the status linked list.
10. An electronic device comprising a memory and a data processor; The memory stores a computer program; The data processor executes the computer program and performs the following steps according to the computer program: When a read / write exception occurs, status information recorded in a log file is obtained, wherein the log file includes multiple read / write requests received by the data processor before the read / write exception occurs and status information corresponding to each of the read / write requests; According to the state information, obtaining a target read / write request that needs to be abnormally recovered in the log file; Perform abnormal recovery according to the target read and write request.