Access operation request processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of storage acceleration, in particular to an access operation request processing method, device and equipment and a storage medium, and is used for solving the problem that a remote access system is low in reliability and stability. Determining whether the request information is overtime, if the request information is overtime, retransmitting the access operation request based on the overtime request information, and updating the timestamp of the overtime request information based on the moment of retransmitting the access operation request; wherein the request information is used for indicating identification information of the access operation request; therefore, the intelligent network card can monitor an access operation request loss fault and start a retransmission mechanism based on the request information in the cache space, so that the host does not sense the fault, and the reliability and the stability of the system are improved; and meanwhile, the access operation request and the message data are not directly cached, so that the resource occupation and the logic complexity of the intelligent network card are reduced.

Description

Technical Field

[0001] The present application relates to the field of storage acceleration technology, and in particular to a method, device, equipment and storage medium for processing access operation requests. Background Art

[0002] Currently, due to the scalability, flexibility and low disaster recovery cost of cloud storage, more and more users are placing storage devices in the cloud. In data center scenarios, smart network cards are currently widely used to provide network and storage acceleration for hosts. The storage acceleration service provides transparent cloud storage services for hosts, that is, hosts use storage like local hard disks, without network processing overhead, while enjoying the flexibility and reliability of cloud storage.

[0003] Under the related technology, the smart network card implements the host's local storage access by presenting a standard virtual block device (Virtio blockdevice, virtio_blk) to the host. At the same time, the smart network card sends the received storage data to the cloud through the network to implement cloud storage of data. However, while this application model seamlessly presents cloud storage to the host, it also causes the processing path of storage services to become longer, bringing more unstable factors, such as storage IO loss. Storage IO refers to the input and output operations performed on storage devices in a computer system, causing the host's storage driver to frequently hang. The current solution is to only cold restart the host, which seriously affects the normal operation of the entire system and seriously reduces the reliability and stability of the remote access system. Summary of the invention

[0004] The purpose of the present application is to provide a method, device, equipment and storage medium for processing access operation requests, so as to improve the reliability and stability of a remote access system.

[0005] The specific technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, a method for processing an access operation request is applied to a smart network card, the smart network card is connected to a host, a cache space of the smart network card is used to cache request information of the access operation request, the request information is used to indicate identification information of the access operation request; the method comprises:

[0007] Determining whether the request information has timed out based on a timestamp of the request information in the cache space;

[0008] If the request information times out, resending the access operation request based on the timed out request information;

[0009] The timestamp of the timed-out request information is updated based on the time at which the access operation request is resent.

[0010] By adopting a method for processing access operation requests provided by an embodiment of the present application, the smart network card determines whether the request information has timed out based on the timestamp of the request information cached in the preset cache space. In this way, the smart network card can monitor the access operation request loss failure at the sending source end, thereby starting the retransmission mechanism, so that the host is unaware of the access operation request loss failure, and there is no need to modify the client's operating system. Since the present application can monitor the access operation request loss failure at the sending source end, it can handle any failure including FPGA transmission failure, SOC software failure, network failure, remote storage system failure, etc., thereby improving the reliability and stability of the remote access system.

[0011] Furthermore, in an embodiment of the present application, the request information is generated and cached immediately after the access operation request is obtained from the access operation queue of the host. In this way, the access operation request and message data are not directly cached, which reduces the resource usage and logical complexity of the smart network card, simplifies the fault recovery system, and further improves the stability and reliability of the remote access system.

[0012] In some possible embodiments, determining whether the request information has timed out includes:

[0013] If the difference between the system time and the timestamp is greater than or equal to a threshold, determining that the request information has timed out;

[0014] If the difference between the system time and the timestamp is less than the threshold, it is determined that the request information has not timed out;

[0015] The timestamp is used to indicate the time when the smart network card last sent the access operation request to the outside.

[0016] The above method can automatically start the retransmission mechanism of the smart network card when the sending time of the access operation request reaches a threshold, realize fault self-recovery, and improve the reliability and stability of the remote access system.

[0017] In some possible embodiments, the resending the access operation request based on the timed request information includes:

[0018] Based on the identification information indicated in the timed-out request information, acquiring the message data corresponding to the identification information from the memory of the host, and generating the access operation request;

[0019] Adding the first cache address information of the timed-out request information to a preset field in the access operation request, and resending the access operation request with the added cache address information to the outside;

[0020] Among them, the first cache address information is allocated by the smart network card for the timed request information when the smart network card first obtains the access operation request, and is added to the timed request information, and is added to the access operation request before sending the access operation request to the outside each time.

[0021] In the above method, the smart network card only caches the request information of each access operation request, but does not cache the access operation request and message data. After starting the retransmission mechanism, based on the identification information indicated in the timed-out request information, the message data corresponding to the identification information is re-acquired from the host's memory, and the access operation request corresponding to the timed-out request information is generated, which reduces the resource usage and logic complexity of the smart network card, simplifies the fault recovery system, and further improves the stability and reliability of the remote access system.

[0022] In some possible embodiments, after the access operation request adding the cache address information is resent to the outside, the method further includes:

[0023] If a response message for the access operation request returned by the external device is received, determining whether the request information exists in the cache space based on the second cache address information carried in the response message;

[0024] If it exists and the request information is in a valid state, forwarding the response message to the host and adjusting the request information to an invalid state;

[0025] If it does not exist, or if it exists and the request information is in an invalid state, the response message is discarded.

[0026] In some possible embodiments, determining whether the request information exists in the cache space includes:

[0027] Based on the second cache address information carried in the response message, determining whether request information exists in the cache address corresponding to the second cache address information in the cache space;

[0028] If there is request information, comparing the determined request information with the response message;

[0029] If the identification information indicated in the determined request information matches the identification information carried in the response message, it is determined that the request information exists in the cache space.

[0030] In some possible embodiments, the request information is further used to indicate deduplication information of a response to the access operation request; the deduplication identification information of the response is used to identify duplicate response messages;

[0031] Before determining that the request information exists in the cache space, the method further includes:

[0032] Determine that the first response deduplication information in the determined request information matches the second response deduplication information carried in the response message.

[0033] The above method filters the response messages through the intelligent network card and filters out duplicate response messages, so that the host does not need to process duplicate response messages, avoiding unknown system errors caused by the host receiving duplicate response messages. At the same time, it can also handle the disordered response returned from the outside by comparing the response deduplication information, further improving the reliability and stability of the entire remote access system.

[0034] In some possible embodiments, before determining whether the request information has timed out based on the timestamp of the request information in the cache space, the method further includes:

[0035] Based on the number of virtual block devices of the host and the queue depth of the virtual queue of each virtual block device, the cache space is initialized in the memory, wherein the cache address information of each request information in the cache space adopts low-order mapping.

[0036] In a second aspect, a device for processing access operation requests is applied to a smart network card, the smart network card is connected to a host, a cache space of the smart network card is used to cache request information of the access operation request, the request information is used to indicate identification information of the access operation request; the device includes:

[0037] A determination module, configured to determine whether the request information has timed out based on a timestamp of the request information in the cache space;

[0038] a retransmission module, configured to resend the access operation request based on the timed-out request information if the request information times out;

[0039] The updating module is used to update the timestamp of the timed-out request information based on the time when the access operation request is resent.

[0040] In some possible embodiments, the determining module is specifically used to:

[0041] If the difference between the system time and the timestamp is greater than or equal to a threshold, determining that the request information has timed out;

[0042] If the difference between the system time and the timestamp is less than the threshold, it is determined that the request information has not timed out;

[0043] The timestamp is used to indicate the time when the smart network card last sent the access operation request to the outside.

[0044] In some possible embodiments, the retransmission module is specifically used to:

[0045] Based on the identification information indicated in the timed-out request information, acquiring the message data corresponding to the identification information from the memory of the host, and generating the access operation request;

[0046] Adding the first cache address information of the timed-out request information to a preset field in the access operation request, and resending the access operation request with the added cache address information to the outside;

[0047] Among them, the first cache address information is allocated by the smart network card for the timed request information when the smart network card first obtains the access operation request, and is added to the timed request information, and is added to the access operation request before sending the access operation request to the outside each time.

[0048] In some possible embodiments, after the access operation request with the cache address information added thereto is resent to the outside, the retransmission module is further configured to:

[0049] If a response message for the access operation request returned by the external device is received, determining whether the request information exists in the cache space based on the second cache address information carried in the response message;

[0050] If it exists and the request information is in a valid state, forwarding the response message to the host and adjusting the request information to an invalid state;

[0051] If it does not exist, or if it exists and the request information is in an invalid state, the response message is discarded.

[0052] In some possible embodiments, the retransmission module is specifically used to:

[0053] Based on the second cache address information carried in the response message, determining whether request information exists in the cache address corresponding to the second cache address information in the cache space;

[0054] If there is request information, comparing the determined request information with the response message;

[0055] If the identification information indicated in the determined request information matches the identification information carried in the response message, it is determined that the request information exists in the cache space.

[0056] In some possible embodiments, the request information is further used to indicate deduplication information of a response to the access operation request; the deduplication information of the response is used to identify duplicate response messages;

[0057] Before determining that the request information exists in the cache space, the retransmission module is further used to:

[0058] Determine that the first response deduplication information in the determined request information matches the second response deduplication information carried in the response message.

[0059] In some possible embodiments, before determining whether the request information has timed out based on the timestamp of the request information in the cache space, the determining module is further configured to:

[0060] Based on the number of virtual block devices of the host and the queue depth of the virtual queue of each virtual block device, the cache space is initialized in the memory, wherein the cache address information of each request information in the cache space adopts low-order mapping.

[0061] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor,

[0062] The memory is used to store computer programs or instructions;

[0063] The processor is used to execute the computer program or instructions in the memory so that the method described in any one of the first aspects above is executed.

[0064] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the steps of the method described in any one of the first aspects above are implemented.

[0065] In addition, the technical effects brought about by any one of the implementation methods in the second to fourth aspects can refer to the technical effects brought about by different implementation methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1A A schematic diagram of a system framework of a storage system applicable to a method for processing an access operation request in an embodiment of the present application;

[0067] Figure 1B This is a schematic diagram of the internal architecture of a smart network card in an embodiment of the present application;

[0068] Figure 2 A flowchart of a method for processing an access operation request in an embodiment of the present application;

[0069] Figure 3 A schematic diagram of a system framework of a fault recovery system applicable to a method for processing an access operation request in an embodiment of the present application;

[0070] Figure 4 This is a schematic diagram of a process for determining the timeout of the request information in the cache space in an embodiment of the present application;

[0071] Figure 5 A schematic diagram of a process of obtaining the access operation request in an embodiment of the present application;

[0072] Figure 6 A schematic diagram of a process for obtaining a timed-out retransmission access operation request in an embodiment of the present application;

[0073] Figure 7 This is a schematic diagram of a process of processing a response message in an embodiment of the present application;

[0074] Figure 8 A schematic diagram of a process for determining whether request information corresponding to the access operation request exists in the cache space in an embodiment of the present application;

[0075] Fig. 9 A schematic diagram of a format of request information in an embodiment of the present application;

[0076] Fig.10 A schematic diagram of a cache management process of a cache space (buffer) in an embodiment of the present application;

[0077] Fig.11 A schematic diagram of the format of a request frame of an access operation request (HOST Request) in an embodiment of the present application;

[0078] Fig.12 This is a schematic diagram of the format of a response frame of a response message (Completion) in an embodiment of the present application; Fig.13 A schematic diagram of a failure point that can be avoided in an embodiment of the present application;

[0079] Fig.14 A schematic diagram of the logical architecture of a device for processing access operation requests in an embodiment of the present application;

[0080] Fig.15 It is a schematic diagram of the physical structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0082] It should be noted that the terms "first", "second", "third", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0083] Under traditional technology, the smart network card implements local storage access of the host (HOST) by presenting the standard virtio_blk to the host (HOST), and at the same time, the smart network card sends the received storage data to the cloud through the network to realize cloud storage of data. However, while this application model seamlessly presents cloud storage to the host (HOST), it also causes the processing path of storage services to become longer, bringing more unstable factors, such as storage IO loss. Storage IO refers to the input and output operations on storage devices in a computer system, which makes the storage driver of the host (HOST) frequently hang. The current solution is to cold restart the host (HOST), which seriously affects the normal operation of the entire system and seriously reduces the reliability and stability of the remote access system.

[0084] Since the traditional operating system has a relatively simple access logic for local hard disks, it believes that each access operation request can be processed and will definitely get a response from the "hardware". The operating system does not have a timeout retransmission mechanism, so it will always wait for the "hardware" response.

[0085] However, in virtualization technology, the host (HOST) sees a standard virtio_blk, such as a hard disk or a disk. The smart NIC sends the storage data obtained from the host (HOST) to the processor on the smart NIC for processing, and finally transmits it to the cloud storage through the network. This process will go through several stages: Field Programmable Gate Array (FPGA) transmission (for data processing unit (DPU) smart NIC), on-card processor software transmission, network transmission, etc. If any of these stages fails, such as FPGA transmission failure, operating system failure and restart of the on-card processor, the access operation request will be lost, that is, storage IO will be lost, which will cause the storage driver of the host (HOST) to hang because it cannot wait for the "hardware" response.

[0086] In view of this, in order to solve the problem that traditional solutions seriously reduce the reliability and stability of the entire remote access system, an embodiment of the present application provides a method for processing access operation requests, which is applied to a smart network card. The smart network card is connected to a host. The cache space of the smart network card is used to cache request information of the access operation request, and the request information is used to indicate identification information of the access operation request. The method includes: based on the timestamp of the request information in the cache space, if it is determined that the request information has timed out, then based on the timed-out request information, resending the access operation request, and based on the time of resending the access operation request, updating the timestamp of the timed-out request information; wherein, a plurality of request information are stored in the cache space, and each request information is generated by the smart network card after obtaining the corresponding access operation request from the access operation queue of the host.

[0087] By adopting a method for processing access operation requests provided in an embodiment of the present application, the smart network card determines whether the request information has timed out based on the timestamp of any request information cached in a preset cache space. In this way, the smart network card can monitor the access operation request loss failure at the sending source end, thereby starting a retransmission mechanism, so that the host is unaware of the access operation request loss failure. Since the host is unaware of the access operation request loss failure, there is no need to modify the client's operating system.

[0088] At the same time, in an embodiment of the present application, after obtaining an access operation request from the access operation queue of the host, a request information corresponding to the access operation request is immediately generated and cached, and the request information includes a timestamp and identification information of the access operation request. In this way, the access operation request and message data are not directly cached, which reduces the resource occupancy and logical complexity of the smart network card, simplifies the fault recovery system, and improves the reliability and stability of the entire remote access system.

[0089] The preferred implementation modes of the present application are further described in detail below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0090] Figure 1A Schematic diagram of the system architecture of a storage system applicable to a method for processing access operation requests in an embodiment of the present application. Figure 1A As shown, the storage system includes a host (HOST), a smart network card connected to the host (HOST), and a storage device located in the cloud, wherein the host (HOST) can be an elastic bare metal scenario server or a virtualized scenario server.

[0091] The storage device can be a cloud server or a cluster or multiple clusters composed of multiple cloud servers, such as a Ceph cluster. The storage device communicates with the smart network card through the network, such as transmitting data. The storage device is presented as a standard virtio_blk storage device on the host side through the smart network card, so that the host can perform access operations through direct memory access (DMA), that is, storage IO.

[0092] It should be noted that Figure 1A This is an example introduction to the application scenario of a method for processing an access operation request provided in an embodiment of the present application. In fact, the application scenario to which the method in the embodiment of the present application can be applied is not limited to this.

[0093] In the embodiments of the present application, the above-mentioned smart network card can be a smart network card based on software implementation, such as a multi-core smart network card, based on an application-specific integrated circuit (ASIC) chip containing multiple CPU cores; it can also refer to a DPU smart network card, which is a hardware device that processes data during network transmission, and it can realize functions such as data acceleration, security, and analysis during network transmission. Compared with traditional network cards, DPU smart network cards have higher computing power and richer functions, which can greatly improve the efficiency and security of network transmission.

[0094] See also Figure 1BAs shown, the DPU smart network card in the embodiment of the present application is a programmable smart network card based on FPGA and system on a chip (System on a Chip, SOC). In the embodiment of the present application, only the DPU smart network card is taken as an example to introduce in detail a method for processing an access operation request in the embodiment of the present application. It should be noted that the method in the embodiment of the present application can also realize fault recovery of a multi-core smart network card during transmission.

[0095] After introducing the exemplary application scenarios and the internal architecture of a smart network card in the embodiments of the present application, the preferred implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings. Figure 2 In the process of a method for processing an access operation request shown, only one request message is used as an example for introduction. It should be noted that it does not mean that only one request message in the cache space times out.

[0096] See also Figure 2 As shown, a method for processing an access operation request in an embodiment of the present application is applied to a smart network card. The cache space of the smart network card is used to cache request information of the access operation request. The cache space stores multiple request information. The multiple request information is generated by the smart network card for the access operation request obtained from the access operation queue of the host, and is used to indicate the identification information of the corresponding access operation request. The specific process of the method is as follows:

[0097] Step 200: Determine whether the request information has timed out based on the timestamp of the request information in the cache space.

[0098] The timestamp of the request information is used to indicate the time when the smart network card last sent the access operation request to the outside, and the outside is the storage device to be accessed determined according to the access operation request.

[0099] In the implementation of this application, please refer to Figure 1B As shown, when executing step 200, the FPGA of the smart network card can periodically scan the cache space and determine whether the request information has timed out based on the timestamp of the request information in the cache space; wherein the cache space can be a cache space inside the FPGA chip or a cache space outside the FPGA chip, and this application does not make specific limitations.

[0100] Figure 3 Schematic diagram of the system framework of a fault recovery system applicable to a method for processing access operation requests in an embodiment of the present application. Figure 3As shown, the system framework diagram includes a host (HOST), a smart network card and a storage device. The smart network card includes an FPGA and a SOC. The FPGA includes virtio DMA, blk_lossless_io (storage lossless IO), buffer and other functional modules.

[0101] blk_lossless_io, used to parse the access operation request obtained from the host's access operation queue through virtio DMA, parse the response message returned by the external (i.e., storage device), and periodically scan the buffer to determine whether each request information cached in the buffer has timed out;

[0102] The buffer is used to cache request information, which is generated and cached by the smart network card for access operation requests obtained from the access operation queue of the host.

[0103] It should be noted that for the convenience of description and illustration, only Figure 3 The buffer space in the FPGA of the smart network card is used as an example for introduction.

[0104] See also Figure 3 As shown, in the tx (transmit) direction, the virtio DMA in the FPGA of the smart network card obtains the access operation request from the access operation queue (avail_ring queue) of the host (HOST), and then transmits the obtained access operation request to blk_lossless_io, blk_lossless_io parses the access operation request, generates a request information corresponding to the access operation request, and caches the request information in the buffer in the FPGA of the smart network card, and inserts the cache address information of the request information in the buffer into each request frame of the access operation request, which is recorded as the first cache address information, and then transmits the access operation request with the inserted cache address information to the SOC through the virtio DMA, and the SOC calls the Storage Performance Development Kit (Storage Performance Development Kit, SPDK) for protocol processing, and sends it to the outside through the network after the protocol processing, that is, Figure 3 A storage device is shown.

[0105] Still reading Figure 3As shown, in the rx (HOST receiving) direction, the blk_lossless_io in the FPGA of the smart network card parses the response message returned by the external (i.e., the storage device), reads the buffer based on the cache address information (recorded as the second cache address information) carried in the response message, and compares the identification information indicated in the request information corresponding to the second cache address information in the buffer with the identification information carried in the response message, determines whether the response message is the access operation request corresponding to the request message based on the comparison result, and determines whether to recycle the buffer resources, and forwards the response message to the host (HOST), etc.

[0106] based on Figure 3 The architecture diagram shown in the figure shows that in the specific implementation, blk_lossless_io regularly scans the buffer, see Figure 4 As shown, when executing step 200, the following steps may be specifically performed:

[0107] Step 2001: If the difference between the system time and the timestamp is greater than or equal to a threshold, it is determined that the request information has timed out.

[0108] Step 2002: If the difference between the system time and the timestamp is smaller than the threshold, it is determined that the request information has not timed out.

[0109] In the embodiment of the present application, blk_lossless_io subtracts the current system time from the timestamp of the request information to obtain the AGE of the request information, and then compares the AGE with the threshold. If the AGE of the request information is greater than or equal to the threshold, step 2001 is executed to determine that the request information has timed out; if the AGE of the request information is less than the threshold, step 2002 is executed to determine that the request information has not timed out. The threshold can be determined based on experience or actual scenarios.

[0110] Step 210: If the request message times out, resend the access operation request based on the timed out request message.

[0111] In the present application, see Figure 5 As shown, when executing step 210, the following steps are specifically performed:

[0112] Step 2101: Based on the identification information indicated in the timed-out request information, obtain the message data corresponding to the identification information from the host's memory to generate the access operation request.

[0113] In the present application, see Figure 6As shown, when executing step 2101, blk_lossless_io re-acquires the message data corresponding to the identification information from the memory of the host (HOST) based on the identification information indicated in the timed-out request information (e.g., 1st_DescID, including the location and data length of the transmitted data), and then generates the access operation request based on the message data.

[0114] In an embodiment of the present application, blk_lossless_io only caches the request information of each access operation request, but does not cache the original access operation request and message data. After starting the retransmission mechanism, based on the identification information indicated in the timed-out request information, the access operation request and message data corresponding to the timed-out request information are retrieved from the host (HOST) memory to generate the access operation request. In this way, not only the limited memory resources of the FPGA of the smart network card are saved, but also the execution logic complexity of the FPGA of the smart network card is reduced, and the architecture of the fault recovery system is simplified.

[0115] Step 2102: Add the first cache address information of the timed-out request information to a preset field in the access operation request, and resend the access operation request with the added cache address information to the outside; wherein the first cache address information is allocated by the smart network card for the timed-out request information when the smart network card first obtains the access operation request, and is added to the timed-out request information, and is added to the access operation request each time before the access operation request is sent to the outside.

[0116] In an embodiment of the present application, when executing step 2102, blk_lossless_io adds the first cache address information of the timed-out request information to a preset field in the access operation request, and transmits the access operation request for adding the cache address information to the SOC in the smart network card. The SOC calls SPDK to perform protocol processing on the access operation request for adding the cache address information, and sends it to the outside through the network after protocol processing.

[0117] Step 220: Based on the time of resending the access operation request, update the timestamp of the timed-out request information.

[0118] In the embodiment of the present application, the retransmitted access operation request and the original access operation request both use the same identification information of the access operation request and the same request information. Therefore, when executing step 220, the timestamp of the timed-out request information is updated based on the time when the access operation request is resent.

[0119] In this way, when blk_lossless_io scans the buffer subsequently, if it is determined that the request information for updating the timestamp has timed out again, the retransmission mechanism can still be started to retransmit the access operation request, thereby maximally ensuring that each access operation request initiated by the host (HOST) can obtain a corresponding response message, thereby improving the reliability and stability of the entire remote access system.

[0120] Furthermore, the retransmitted access operation request and the original access operation request both use the same access operation request identification information and the same request information, only changing the timestamp of the request information that times out in the buffer, which greatly reduces the amount of cached data and saves the limited memory resources of the FPGA, thereby improving the reliability and stability of the entire remote access system.

[0121] In the embodiment of the present application, after executing step 220, refer to Figure 7 As shown, the following steps are also performed:

[0122] Step 230: If a response message for the access operation request is received from the outside, determine whether the request information exists in the cache space based on the second cache address information carried in the response message.

[0123] In the specific implementation, when executing step 230, refer to Figure 8 As shown, the following steps are specifically performed to determine whether the request information exists in the cache space:

[0124] Step 2301: Based on the second cache address information carried in the response message, determine whether there is request information in the cache address corresponding to the second cache address information in the cache space.

[0125] In an embodiment of the present application, request information may or may not exist in the cache address corresponding to the second cache address information in the cache space. Therefore, when executing step 230, step 2301 must be executed first to determine whether request information exists in the cache address corresponding to the second cache address information. If so, step 2302 is further executed to determine whether the request information in the cache space is the request information corresponding to the access operation request; if not, it is determined that the request information corresponding to the access operation request does not exist in the cache space, indicating that the access operation request has been responded to, or, an error response.

[0126] Step 2302: If request information exists, the determined request information is compared with the response message.

[0127] Step 2303: If the identification information indicated in the determined request information matches the identification information carried in the response message, it is determined that the request information exists in the cache space.

[0128] Still reading Figure 3 As shown, in the tx (send) direction, when blk_lossless_io inserts the first cache address information of the request information into the access operation request, it can also insert the response deduplication information corresponding to the access operation request; then, the access operation request after the insertion operation is completed is transmitted to the SOC, and the SOC sends it to the outside through the network, wherein the response deduplication information is used to identify duplicate response messages, such as the response deduplication information includes the number of the corresponding access operation request obtained from the access operation queue of the host, so as to identify whether the response message is a duplicate response to which access operation request when the response message returned from the outside is received.

[0129] Still reading Figure 3 As shown, in the rx (HOST receiving) direction, when executing step 2302, blk_lossless_io compares the request information corresponding to the second cache address in the determined buffer with the response message, and determines that the identification information indicated in the request information determined in the buffer matches the identification information carried in the response message. Then, before executing step 2303, it is necessary to further determine that the first response deduplication information in the request information corresponding to the second cache address information in the buffer matches the second response deduplication information carried in the response message before determining that the request information exists in the cache space. Correspondingly, if the first response deduplication information in the determined request information does not match the second response deduplication information carried in the response message, it is determined that the request information does not exist in the cache space.

[0130] In some possible embodiments, after executing step 2301, if there is no request information in the cache address corresponding to the second cache address information in the cache space, it means that the response message is a duplicate response, and it is determined that the request information does not exist in the cache space.

[0131] In other possible embodiments, after executing step 2301, if there is request information in the cache address corresponding to the second cache address information in the cache space, but the identification information indicated in the determined request information does not match the identification information carried in the response message, it is determined that the request information does not exist in the cache space.

[0132] Step 240: If it exists and the request information is in a valid state, the response message is forwarded to the host and the request information is adjusted to an invalid state.

[0133] Fig. 9 This is a schematic diagram of a format of request information in an embodiment of the present application. Fig. 9As shown, the request information includes information validity bit (vld), timestamp (timestamp), access operation request response deduplication information (blk_seq_id) and identification information (1st_DescID: virtio_blk storage device virtual queue descriptor chain head ID), where blk_seq_id is a 32-bit deduplication ID, which can be incremented from 0 and cleared when the smart network card is powered off. Based on experimental tests, it can be seen that the 32-bit blk_seq_id can be suitable for 25G smart network cards, while the higher-speed 100G or 200G smart network cards need to increase the bit width of blk_seq_id. Therefore, it can be flexibly configured according to actual needs in actual use.

[0134] Step 250: If it does not exist, or if it exists and the request information is in an invalid state, discard the response message.

[0135] In the embodiment of the present application, the method of determining whether the request information exists in the cache space has been described in detail in the above content and will not be repeated here.

[0136] The above embodiment is further described in detail below using specific examples.

[0137] still Figure 3 Taking the architecture shown as an example, in the embodiment of the present application, after the host (HOST) is turned on, the smart network card is powered on. During the initialization process of the FPGA memory of the smart network card, the virtual queue Queue Size of the virtio_blk storage device of the host (HOST) is set by running the pre-modified FPGA memory initialization file, and the number of pending access operation requests of the host (HOST) is controlled to achieve the maximum number of access operation requests of the host (HOST) and the deep matching of the FPGA cache space, so as to achieve the purpose that the FPGA can monitor all pending access operation requests during the implementation process. At the same time, by modifying the initialization file of the FPGA memory, the maximum number of access operation requests of the host (HOST) is achieved. The deep matching of the FPGA cache space is also improved, which also improves the reliability of the cache information.

[0138] In the specific implementation, the smart network card initializes the aforementioned cache space in the memory according to the modified FPGA memory initialization file, based on the number of Virtio_blk (virtual block devices) of the host and the queue depth of the virtual queue of each Virtio_blk (virtual block device), wherein the cache address information of each request information in the cache space adopts low-bit mapping. Specifically, the data size of 2 can be realized in the FPGA chip (M+N)The depth of the buffer (buffer), where M is the number of Virtio_blk (virtual block devices), and N is the sum of the Virtqueues (virtual queues) QueueSize (queue depth) of the M Virtio_blk (virtual block devices).

[0139] Fig.10 Schematic diagram of a cache management process of a cache space (taking buffer as an example) in an embodiment of the present application. Fig.10 As shown, after the cache space (i.e., buffer) is initialized, the first cache address information of each request message in the buffer cache is statically mapped by the DPVF field in the channel header of the access operation request (HOST Request) and the response message (SOCCompletion), i.e. Fig.10 q0~qm shown in; the first cache address information N-1 to 0 bits are statically mapped by the preset field (e.g., reserved field: RSVD1) in the DMA header of the access operation request (HOST Request) and the response message (SOC Completion), that is, Fig.10 b0~bn shown in .

[0140] In the present application, see Fig.10 As shown, the FPGA of the smart network card maintains the inner_id pool and is responsible for allocating inner_id to the access operation request obtained from the access operation queue (avail_ring queue) for caching the request information of the access operation request.

[0141] In the specific implementation, please refer to Figure 3 As shown, in the tx (transmit) direction, the virtio DMA in the FPGA of the smart network card obtains an access operation request from the access operation queue (avail_ring queue) of the host (HOST), and then transmits the obtained access operation request to blk_lossless_io, blk_lossless_io parses the access operation request, generates a request information corresponding to the access operation request, allocates a first cache address information for the request information of the access operation request, and caches the request information in the first cache address information in the buffer.

[0142] blk_lossless_io inserts the first cache address information of the request information in the buffer and the response deduplication information (blk_seq_id) into each request frame of the access operation request. Fig.11 The following is a schematic diagram of the format of a request frame for an access operation request (HOST Request) in an embodiment of the present application. Fig.11As shown, the DPVF field in the channel header of the request frame and the reserved field (inner_id in RSVD1) in the DMA header are used to indicate the first cache address information of the request information corresponding to the access operation request, wherein the DPVF field is used to statically map the M+N-1 to N bits of the cache address of the request information corresponding to the access operation request in the buffer, and the inner_id in RSVD1 is used to statically map the N-1 to 0 bits of the first cache address of the request information corresponding to the access operation request in the buffer; the reserved field in the DMA header (blk_seq_id in RSVD1) is used to indicate the response deduplication information of the request information corresponding to the access operation request.

[0143] It should be noted that Fig.11 The format of the request frame shown is for example only. In actual applications, the specific format of the request frame will vary with different DMAs, but a method for processing access operation requests provided in an embodiment of the present application may be adopted to perform field insertion processing on access operation requests obtained from an access operation queue or retransmitted access operation requests, so as to achieve the implementation purpose of the embodiment of the present application.

[0144] Then, after inserting the first cache address information of the request information and the response deduplication information, blk_lossless_io transmits the access operation request with the first cache address information and the response deduplication information to the SOC through virtio DMA. The SOC calls SPDK for protocol processing and sends it to the outside through the network after the protocol processing.

[0145] Correspondingly, since the first cache address information and response deduplication information of the corresponding request information are inserted into the access operation request, the response message returned externally will also contain the second cache address information and response deduplication information of the request information of the access operation request corresponding to the response message. Fig.12 FIG. 1 is a schematic diagram of the format of a response frame of a response message (Completion) in an embodiment of the present application. Fig.12 As shown, the DPVF field in the channel header of the response frame and the reserved field (inner_id in RSVD1) in the DMA header are used to indicate the second cache address information of the request information of the access operation request corresponding to the response message, wherein the DPVF field is used to statically map the M+N-1 to N bits of the second cache address information of the request information in the buffer, and the inner_id in RSVD1 is used to statically map the N-1 to 0 bits of the second cache address information of the request information in the buffer; the reserved field in the DMA header (blk_seq_id in RSVD1) is used to indicate the second response deduplication information of the request information of the access operation request corresponding to the response message. Fig.12As shown in the figure, data[] is the read and Get_ID request read return data, and there is no data[] in the response frame of other types of requests; status is the request completion status.

[0146] It should be noted that Fig.11 The request frame shown is similar to, Fig.12 The format of the response frame shown is only an example. In actual applications, the specific format of the response frame will vary with different DMAs. However, since the request frame carries the cache address information and response deduplication information of the corresponding request information, the corresponding response frame will also carry the cache address information and response deduplication information of the request information of the corresponding access operation request. For different DMAs, a method for processing access operation requests provided in an embodiment of the present application can be adopted to execute subsequent processes to achieve the implementation purpose of the embodiment of the present application.

[0147] In an embodiment of the present application, after receiving a response message returned from the outside for any access operation request, blk_lossless_io parses the response message, and uses the DPVF field in the channel header of the response frame of the response message to statically map out the M+N-1 to N bits of the second cache address information, and uses the reserved field in the DMA header (inner_id in RSVD1) to statically map out the N-1 to 0 bits of the second cache address information; then, based on the second cache address information carried in the response message, it is determined whether there is request information corresponding to the access operation request in the buffer.

[0148] In a specific implementation, based on the second cache address information carried in the response message, it is determined whether there is request information in the cache address corresponding to the second cache address information in the buffer.

[0149] If there is request information and the determined request information is in a valid state, the determined request information is compared with the response message. If the identification information (1st_DescID1) indicated in the determined request information is the same as the identification information (1st_DescID2) carried in the response message, the first response deduplication information (blk_seq_id1) indicated in the determined request information is further compared with the second response deduplication information (blk_seq_id2) carried in the response message to see if they are the same. If it is determined that the first response deduplication information and the second response deduplication information are the same, it is determined that there is request information corresponding to the access operation request in the cache space, the response message is forwarded to the host, and the determined request information is adjusted to an invalid state.

[0150] If there is request information and the determined request information is in an invalid state, the response message is discarded.

[0151] If the request information exists and the determined request information is in a valid state, but the identification information and / or the response deduplication information indicated in the determined request information are not the same, the response message is discarded.

[0152] If the request information does not exist, the response message is discarded.

[0153] Wherein, based on the determined information valid bit (vld) of the request information, it is determined whether the request information is in a valid state.

[0154] Still reading Fig.10 As shown, in scenarios such as network congestion / TCP timeout retransmission, the retransmission response message and the original response message with the same id (inner_id1, blk_seq_id1, 1st_DescID1) arrive one after another. The response message that arrives first triggers the FPGA to release inner_id1, blk_seq_id1, 1st_DescID1, and the FPGA sets the information valid bit (vld) of the request information in the cache address corresponding to the second cache address information carried in the first-arrived response message in the buffer to 0 (invalid), indicating that inner_id1 can be recycled and allocated to the next access operation request (inner_id1, blk_seq_id2, 1st_DescID2).

[0155] If the aforementioned duplicate response message arrives before the FPGA reallocates inner_id1, and it is detected that the request information in the cache address corresponding to the second cache address information carried in the duplicate response message in the buffer is invalid, the duplicate response message is discarded, thereby achieving deduplication of the response message. If the aforementioned duplicate response message arrives after the FPGA reallocates inner_id1, and the blk_seq_id2 and 1st_DescID2 of the request information in the cache address corresponding to the cache address information carried in the duplicate response message in the buffer are different from the blk_seq_id1 and 1st_DescID1 carried in the duplicate response message, that is, the match fails, the duplicate response message is automatically discarded, and deduplication of the response message is also achieved.

[0156] In this way, by setting the response deduplication information, and after receiving the response message returned from the outside for any access operation request, the first response deduplication information in the determined request information is compared with the second response deduplication information carried in the response message, thereby realizing the deduplication function of duplicate response messages. At the same time, when the response messages are out of order in the remote access system (i.e., the external device), the method in the embodiment of the present application can also avoid system abnormalities caused by duplicate responses.

[0157] In the embodiment of the present application, by adopting the above-mentioned processing method of access operation request, after the FPGA of the smart network card obtains the access operation request from the access operation queue of the host, it immediately generates and caches a request information corresponding to the access operation request, and the request information includes a timestamp and identification information of the access operation request, instead of directly caching the access operation request and message data, thereby reducing the resource occupation and logic complexity of the smart network card and simplifying the fault recovery system. In the specific implementation, the FPGA of the smart network card starts the retransmission mechanism when it determines that the current system time and the difference between the timestamp of any request information in the cache space is greater than the threshold, that is, the FPGA is fully implemented in hardware, so that the SOC is unaware and the host (HOST) is unaware, thereby improving the stability and reliability of the remote access system. Because the host is unaware of the access operation request loss failure, there is no need to modify the client's operating system, which improves the flexibility and scalability of the system.

[0158] See also Fig.13 As shown, by adopting a method for processing access operation requests in an embodiment of the present application, the smart network card can monitor the access operation request loss failure at the sending source end and start the retransmission mechanism, which can handle the following failures that occur when the host (HOST) accesses the remote storage device, thereby improving the reliability and stability of the remote access system: FPGA transmission failure, SPDK insufficient memory, SOC abnormal restart, SOC SPDK abnormal exit, SOC and remote network transmission failure, remote storage system failure, etc.

[0159] In the embodiment of the present application, by modifying the initialization file, the number of access operation requests to be processed by the host (HOST) is controlled, and the inner_id is used to correspond one-to-one with the access operation request, so that the intelligent network card monitors all the access operation requests to be processed by the host (HOST). Compared with the timeout retransmission scheme of the access operation request using the HASH table in the related technology, the present application does not have the HASH conflict defect. Therefore, the access operation request to be processed will not be missed due to the HASH conflict. The embodiment of the present application provides a method for processing access operation requests, which has higher reliability and simpler implementation logic, thereby improving the stability of the remote access system.

[0160] Furthermore, in an embodiment of the present application, blk_seq_id is used to deduplicate reply messages and filter reply messages to filter out duplicate reply messages, so that the host (HOST) does not need to process duplicate reply messages, thereby avoiding unknown system errors when the host (HOST) receives duplicate reply messages, and further improving the reliability and stability of the entire remote access system.

[0161] Based on the same inventive concept, refer to Fig.14As shown, in an embodiment of the present application, a device for processing access operation requests is provided, which is applied to a smart network card, the smart network card is connected to a host, and a cache space of the smart network card is used to cache request information of the access operation request, and the request information is used to indicate identification information of the access operation request; the device includes:

[0162] A determination module 1401, configured to determine whether the request information has timed out based on a timestamp of the request information in the cache space;

[0163] A retransmission module 1402, configured to resend the access operation request based on the timed-out request information if the request information times out;

[0164] The updating module 1403 is configured to update the timestamp of the timed-out request information based on the time of resending the access operation request.

[0165] In some possible embodiments, the determining module 1401 is specifically configured to:

[0166] If the difference between the system time and the timestamp is greater than or equal to a threshold, determining that the request information has timed out;

[0167] If the difference between the system time and the timestamp is less than the threshold, it is determined that the request information has not timed out;

[0168] The timestamp is used to indicate the time when the smart network card last sent the access operation request to the outside.

[0169] In some possible embodiments, the retransmission module 1402 is specifically configured to:

[0170] Based on the identification information indicated in the timed-out request information, acquiring the message data corresponding to the identification information from the memory of the host, and generating the access operation request;

[0171] Adding the first cache address information of the timed-out request information to a preset field in the access operation request, and resending the access operation request with the added cache address information to the outside;

[0172] Among them, the first cache address information is allocated by the smart network card for the timed request information when the smart network card first obtains the access operation request, and is added to the timed request information, and is added to the access operation request before sending the access operation request to the outside each time.

[0173] In some possible embodiments, after the access operation request with the cache address information added is resent to the outside, the retransmission module 1402 is further configured to:

[0174] If a response message for the access operation request returned by the external device is received, determining whether the request information exists in the cache space based on the second cache address information carried in the response message;

[0175] If it exists and the request information is in a valid state, forwarding the response message to the host and adjusting the request information to an invalid state;

[0176] If it does not exist, or if it exists and the request information is in an invalid state, the response message is discarded.

[0177] In some possible embodiments, the retransmission module 1402 is specifically configured to:

[0178] Based on the second cache address information carried in the response message, determining whether request information exists in the cache address corresponding to the second cache address information in the cache space;

[0179] If there is request information, comparing the determined request information with the response message;

[0180] If the identification information indicated in the determined request information matches the identification information carried in the response message, it is determined that the request information exists in the cache space.

[0181] In some possible embodiments, the request information is further used to indicate deduplication information of a response to the access operation request; the deduplication information of the response is used to identify duplicate response messages;

[0182] Before determining that the request information exists in the cache space, the retransmission module 1402 is further configured to:

[0183] Determine that the first response deduplication information in the determined request information matches the second response deduplication information carried in the response message.

[0184] In some possible embodiments, before determining whether the request information has timed out based on the timestamp of the request information in the cache space, the determining module 1401 is further configured to:

[0185] Based on the number of virtual block devices of the host and the queue depth of the virtual queue of each virtual block device, the cache space is initialized in the memory, wherein the cache address information of each request information in the cache space adopts low-order mapping.

[0186] See also Fig.15 As shown, an electronic device is provided in an embodiment of the present application, including a memory 1501 and a processor 1502, specifically:

[0187] The memory 1501 is used to store computer programs or instructions.

[0188] The processor 1502 is used to execute the computer program or instruction in the memory, so that any method is executed by a processing device for an access operation request in the above embodiments.

[0189] The specific connection medium between the memory 1501 and the processor 1502 is not limited in the embodiment of the present application. Fig.15 In the embodiment, the memory 1501 and the processor 1502 are connected via a bus 1500. The bus 1500 is Fig.15 The bus 1500 is represented by a bold line, and the connection between other components is only for schematic illustration and is not intended to be limiting. The bus 1500 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0190] The memory 1501 may be a volatile memory, such as a random-access memory (RAM); the memory 1501 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 1501 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1501 may be a combination of the above memories.

[0191] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, any one of the methods in the above embodiments is implemented.

[0192] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0193] Various implementations of the systems and techniques described above in the embodiments of the present application can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] The present application is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0195] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0196] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0197] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for processing access operation requests, applied to a smart network card, characterized in that: The smart network card is connected to a host, and a cache space of the smart network card is used to cache request information of an access operation request, wherein the request information is used to indicate identification information of the access operation request; the method comprises: Determining whether the request information has timed out based on a timestamp of the request information in the cache space; If the request information times out, resending the access operation request based on the timed out request information; The timestamp of the timed-out request information is updated based on the time at which the access operation request is resent.

2. The method according to claim 1, characterized in that The determining whether the request information has timed out includes: If the difference between the system time and the timestamp is greater than or equal to a threshold, determining that the request information has timed out; If the difference between the system time and the timestamp is less than the threshold, it is determined that the request information has not timed out; The timestamp is used to indicate the time when the smart network card last sent the access operation request to the outside.

3. The method according to claim 1 or 2, characterized in that The resending the access operation request based on the timed request information includes: Based on the identification information indicated in the timed-out request information, acquiring the message data corresponding to the identification information from the memory of the host, and generating the access operation request; Adding the first cache address information of the timed-out request information to a preset field in the access operation request, and resending the access operation request with the added cache address information to the outside; Among them, the first cache address information is allocated by the smart network card for the timed request information when the smart network card first obtains the access operation request, and is added to the timed request information, and is added to the access operation request before sending the access operation request to the outside each time.

4. The method according to claim 3, characterized in that After the access operation request with the cache address information added is resent to the outside, the method further includes: If a response message for the access operation request returned by the external device is received, determining whether the request information exists in the cache space based on the second cache address information carried in the response message; If it exists and the request information is in a valid state, forwarding the response message to the host and adjusting the request information to an invalid state; If it does not exist, or if it exists and the request information is in an invalid state, the response message is discarded.

5. The method according to claim 4, characterized in that The determining whether the request information exists in the cache space includes: Based on the second cache address information carried in the response message, determining whether request information exists in the cache address corresponding to the second cache address information in the cache space; If there is request information, comparing the determined request information with the response message; If the identification information indicated in the determined request information matches the identification information carried in the response message, it is determined that the request information exists in the cache space.

6. The method according to claim 5, characterized in that The request information is also used to indicate deduplication information of a response to the access operation request; The response deduplication information is used to identify duplicate response messages; Before determining that the request information exists in the cache space, the method further includes: Determine that the first response deduplication information in the determined request information matches the second response deduplication information carried in the response message.

7. The method according to claim 1, characterized in that Before determining whether the request information has timed out based on the timestamp of the request information in the cache space, the method further includes: Based on the number of virtual block devices of the host and the queue depth of the virtual queue of each virtual block device, the cache space is initialized in the memory, wherein the cache address information of each request information in the cache space adopts low-order mapping.

8. A device for processing access operation requests, applied to a smart network card, characterized in that: The smart network card is connected to the host, and the cache space of the smart network card is used to cache request information of the access operation request, and the request information is used to indicate identification information of the access operation request; The device comprises: A determination module, configured to determine whether the request information has timed out based on a timestamp of the request information in the cache space; a retransmission module, configured to resend the access operation request based on the timed-out request information if the request information times out; The updating module is used to update the timestamp of the timed-out request information based on the time of resending the access operation request.

9. An electronic device, characterized in that: including a processor and memory, The memory is used to store computer programs or instructions; The processor is used to execute the computer program or instructions in the memory so that the method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images