Data reading and writing method and device, electronic equipment and nonvolatile storage medium

Abstract

The invention discloses a data reading and writing method and device, electronic equipment and a nonvolatile storage medium, and is applied to the technical field of storage. The method comprises the following steps: establishing at least one target physical storage medium into a virtual disk, and mapping the virtual disk to a host; and receiving a target read-write operation request obtained by converting the disk data read-write operation request issued to the target physical storage medium by the host. And when the command parameter of the target read-write operation request is normal, and the current storage environment and the storage path and the hardware storage condition corresponding to the target read-write operation request are all in a normal state, performing read-write operation on the target physical storage medium based on the target read-write operation request. According to the method and the device, the problem that the read-write efficiency, security and resource consumption condition of the disk in related technologies cannot meet user requirements can be solved, and the read-write efficiency and security of the disk data can be effectively improved on the basis of reducing the resource consumption and improving the utilization rate of the disk.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a data reading and writing method, device, electronic equipment and non-volatile storage medium. Background Art

[0002] Disks store electronic data through magnetic recording technology and implement read and write operations on computer data by interacting with the host computer.

[0003] In the related art, the host is directly connected to the hard disk to store computer data, that is, the host will directly store the data that needs to be stored on the disk. This data reading and writing method not only consumes a lot of host resources, but also has low data storage efficiency and security. In addition, due to the limitation of the physical link, the same disk can only be used by one host. When it is used by a host that is in sleep, shut down, or has low business volume, there is a problem that the disk cannot be effectively used.

[0004] In view of this, improving the reading and writing efficiency and security of disk data on the basis of reducing resource consumption and improving disk utilization is a technical problem that needs to be solved by technical personnel in this field.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present invention provides a data reading and writing method, device, electronic device and non-volatile storage medium, which can effectively improve the reading and writing efficiency and security of disk data on the basis of reducing resource consumption and improving disk utilization.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] On one hand, the present invention provides a data reading and writing method, which is applied to a target storage device, comprising:

[0009] Preliminarily constructing at least one target physical storage medium into a virtual disk, and mapping the virtual disk to a host;

[0010] Receive a target read / write operation request sent by the host; the target read / write operation request is obtained by converting the disk data read / write operation request sent by the host to the target physical storage medium; when the command parameters of the target read / write operation request are normal, and the current storage environment and the storage path and hardware storage conditions corresponding to the target read / write operation request are all in normal state, perform read / write operations on the target physical storage medium based on the target read / write operation request.

[0011] In a first exemplary embodiment, after receiving the target read / write operation request sent by the host, it also includes: pre-constructing a read / write request structure; the read / write request structure includes a command type, a command lifecycle, physical storage medium identification information and a storage address; the command type includes a read command and a write command; by parsing the target read / write operation request, the target command type, the target command lifecycle, the target physical storage medium identification information and the logical block address are obtained; based on the target command type, the target command lifecycle, the target physical storage medium identification information and the logical block address, a read / write request structure corresponding to the target read / write operation request is generated, so as to perform read / write operations on the target physical storage medium based on the read / write request structure.

[0012] In a second exemplary embodiment, after receiving the target read and write operation request sent by the host, it also includes: pre-building a silent disk dedicated thread; when the virtual disk is in a silent state, storing each target read and write operation request in a request waiting queue; when it is detected that the target read and write operation request in the waiting queue is awakened, calling the silent disk dedicated thread to process the awakened target read and write operation request.

[0013] In a third exemplary embodiment, before performing read and write operations on the target physical storage medium based on the target read and write operation request, it also includes: obtaining the current storage status and the disk status of the virtual disk; if the current storage status is to normally process the read and write requests, and the disk status is online, the current storage environment is in a normal state; obtaining the first path status between the target storage device and the target physical storage medium, and the second path status between the target storage device and the host; when the first path status and the second path status are not in a reset state, the storage paths corresponding to the target read and write operation requests are in a normal state; if the target physical storage medium has no bad sectors and is in a connected state, the hardware storage conditions are in a normal state.

[0014] In a fourth exemplary embodiment, before performing a read or write operation on the target physical storage medium based on the target read or write operation request, it also includes: when at least one of the path states of the target storage device, the target physical storage medium, and the host is in a reset state, an error code indicating that the path state is incorrect is sent to the host; when the target physical storage medium is in a disconnected state, a notification message indicating that data cannot be written or read is sent to the host; if the command type corresponding to the target read or write operation request is a write command, bad sector data and the number of disconnections of the target physical storage medium are obtained; if the storage address of the data to be written contains the sector where the bad sector is located and / or the number of disconnections of the target physical storage medium exceeds a preset threshold, a notification message indicating that data cannot be written is sent to the host.

[0015] In a fifth exemplary embodiment, after receiving the target read and write operation request sent by the host, it also includes: obtaining the read and write data range, command type and data read and write length of the target read and write operation request; obtaining the virtual disk read and write range, authorization information of the target physical storage medium and the allowed data length; when the authorization information is write-protected or read-only, if the command type of the target read and write operation request is a write command, then the command parameters of the target read and write operation request are illegal; if the read and write data range is within the virtual disk read and write range, and the data read and write length is less than the allowed data length, then the command parameters of the target read and write operation request are normal.

[0016] In a sixth exemplary implementation, the current storage environment includes a storage state and a disk state of the virtual disk. After receiving the target read / write operation request sent by the host, it also includes: obtaining the storage state and the disk state of the virtual disk at the current moment; if the storage state is a damaged state or a disconnected state, all data read / write paths between the host and the storage device are disconnected, and the host enters a timeout processing flow; if the storage state is a silent state, processing of the read / write operation request sent by the host is suspended, and the target read / write operation request is stored in a pre-built request waiting queue; if the storage state is a starting state, the target The target read and write operation requests are stored in a pre-built request waiting queue, and after the startup is completed, each target read and write operation request is processed in the order of the time of joining the request waiting queue; if the disk status is a lost state, an error code indicating that the virtual disk is lost is sent to the host, and a virtual disk online monitoring thread is started; when the virtual disk is detected to be online, virtual disk recovery information is sent to the host; if the disk status is a silent state, the target read and write operation requests are stored in a pre-built request waiting queue, a request time monitoring thread is started, and read and write operation requests in the request waiting queue whose waiting time exceeds a preset waiting time threshold are discarded.

[0017] Another aspect of the present invention provides a data reading and writing device, which is applied to a target storage device, comprising:

[0018] A virtualization module is used to pre-assemble at least one target physical storage medium into a virtual disk and map the virtual disk to a host; a request receiving module is used to receive a target read / write operation request sent by the host; the target read / write operation request is obtained by converting a disk data read / write operation request sent by the host to the target physical storage medium; a request processing module is used to perform read / write operations on the target physical storage medium based on the target read / write operation request when the command parameters of the target read / write operation request are normal and the current storage environment and the storage path and hardware storage conditions corresponding to the target read / write operation request are all in normal state.

[0019] The present invention also provides an electronic device, comprising a processor, wherein the processor is used to implement the steps of the data reading and writing method as described in any of the preceding items when executing a computer program stored in a memory.

[0020] Finally, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the data reading and writing method as described in any of the preceding items are implemented.

[0021] The technical solution provided by the present invention has the advantage that after the target storage device completes the virtual disk through the physical hard disk, after mapping the virtual disk to the host, a block device that can be read and written is formed on the host to save the actual business data. The host only needs to convert the disk data read and write operation request originally sent to the physical storage medium into a target read and write operation request sent to the virtual disk, and the target storage device manages and processes the target read and write operation request, reducing the consumption of the host's own computing resources and memory resources, so that the host can focus on the processing of actual business and ensure the reliability of the host's actual application business. The target storage device sends the target read and write operation request to the target physical storage medium only after the command parameters, the current storage environment, the storage path and the hardware storage conditions are successfully verified, which can not only ensure the efficient execution of the data read and write operation request, but also improve the security of the stored data. Since the underlying physical storage medium is not used by the host, it can be used by multiple hosts at the same time, which can effectively improve the utilization rate of the physical storage medium, and finally realize the effective improvement of the read and write efficiency and security of the disk data on the basis of reducing resource consumption and improving the utilization rate of the disk.

[0022] In addition, the present invention also provides a corresponding implementation device, electronic device and non-volatile storage medium for the data reading and writing method, which further makes the method more practical, and the device, electronic device and non-volatile storage medium have corresponding advantages.

[0023] The technical features mentioned above, the technical features to be mentioned below, and the technical features shown separately in the drawings can be arbitrarily combined with each other, as long as the combined technical features are not contradictory. All feasible feature combinations are technical contents clearly recorded in this article. Any of the multiple sub-features contained in the same sentence can be applied independently, and does not have to be applied together with other sub-features. It should be understood that the above general description and the detailed description below are only exemplary and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A flow chart of a data reading and writing method provided by the present invention;

[0026] Figure 2 A schematic diagram of a flow chart of a data reading and writing method provided by the present invention in an exemplary application scenario;

[0027] Figure 3 A schematic flow chart of another data reading and writing method provided by the present invention;

[0028] Figure 4 A structural diagram of a specific implementation of the data reading and writing device provided by the present invention;

[0029] Figure 5 A structural diagram of a specific implementation of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Among them, the terms "first", "second", etc. in the specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations of the two are intended to cover non-exclusive inclusions. The term "exemplary" means "used as an example, embodiment or illustrative". Any embodiment described here as "exemplary" is not necessarily interpreted as being superior or better than other embodiments.

[0031] With the rapid development of big data management, cloud computing, and artificial intelligence technology, the security and effectiveness of data storage are becoming increasingly demanding, which requires improving the utilization of physical storage media and realizing efficient and secure reading and writing of data to physical storage media. Related technologies use servers directly connected to hard disks to store electronic data. In this way, the server needs to consume a large amount of central processing unit resources and memory resources to execute the entire data reading and data writing process. The server has low efficiency, low security, high resource consumption, and low specificity in accessing data. In addition, due to the limitations of the physical link, the same disk can only be used by one host. If the host is in sleep mode or shutdown mode or the business volume is low, the disk bound to the host cannot be effectively utilized, resulting in low disk utilization and sharing.

[0032] In view of this, in order to improve disk utilization and reduce host resource consumption, the present invention uses a special storage device to manage the IO (read and write operation requests) sent from the host to the hard disk, effectively improving the efficiency, safety, redundancy of hard disk management and the rate of hard disk data IO. After introducing the technical solution of the present invention, various non-limiting implementation methods of the present invention are described in detail below. In order to better illustrate the present invention, numerous specific details are given in the specific implementation methods below. Those skilled in the art should understand that the present invention can also be implemented without these specific details. In other examples, methods, means, components and circuits well known to technicians in the field are not described in detail in order to highlight the main purpose of the present invention.

[0033] First see Figure 1 , Figure 1 A flow chart of a data reading and writing method provided in this embodiment is applied to a target storage device. This embodiment may include the following contents:

[0034] S101: pre-assemble at least one target physical storage medium into a virtual disk, and map the virtual disk to a host.

[0035] Among them, the physical storage medium is the storage component of the host bottom layer, such as a mechanical hard disk and a solid-state hard disk. The target physical storage medium refers to the component selected from these storage components. The selection criteria can be the storage component that the host is about to request for a read or write operation, and can also be the storage components that specify the storage component read and write operations using the method provided by the present invention. After the target physical storage medium is determined, any technology that can virtualize multiple physical storage devices into a logical storage device can be used to build a virtual hard disk, and the present invention does not impose any restrictions on this. After the target storage device completes the virtual disk through the physical hard disk, the virtual disk can be mapped to the host through a physical line, forming a block device that can be read and written on the host to save actual business data. The host refers to any computer device that can be connected to the network, including but not limited to servers, personal computers, workstations, smart phones, etc. The host has its own processor, memory and storage device, and can independently run the operating system and application programs.

[0036] S102: Receive a target read / write operation request sent by a host.

[0037] In this step, when the host wants to perform a read / write operation on the target storage medium, it will send a read / write operation request to the target storage medium. For the convenience of description, it can be defined as a disk data read / write operation request. The present invention aims to solve the problem of direct interaction between the host and the target storage medium, such as Figure 2 As shown, a target storage device is introduced, and accordingly, the disk data read and write operation request needs to be converted into the format required by the target storage device. This conversion operation can be performed by the host, such as the host needs to modify the command receiving end information, and send the read and write operation request originally sent to the target physical storage medium to the virtual disk, and then the target storage device obtains the read and write operation request. It can also be performed by the target storage device, that is, the host does not need to make any improvements. The target storage device can set a monitoring thread. When monitoring the disk data read and write operation request sent by the host to the target physical storage medium, the disk data read and write operation request is captured, and the disk data read and write operation request is converted into a target read and write operation request, and then step S103 is executed. In this way, the target storage device will exchange data with the host, and the host will read or write data from the target physical storage medium managed by the storage device.

[0038] S103: When the command parameters of the target read / write operation request are normal, and the current storage environment and the storage path and hardware storage conditions corresponding to the target read / write operation request are all in normal state, the target physical storage medium is read / written based on the target read / write operation request.

[0039] When the target storage device obtains the read / write operation request originally sent by the host to the target storage medium in the previous step, it will first manage the target read / write operation request from multiple aspects such as thread scheduling, memory management, queue control, timeout control, storage device exception handling, etc. Exemplarily, this step performs status detection from command parameters, current storage environment storage path, and hardware storage conditions. When all aspects meet normal conditions, the target storage device will then send the target read / write operation request to the target physical storage medium. Further, if the target read / write operation request can be executed normally, a success message can be returned to the host. For target read / write operation requests that are write operations, host data also needs to be sent to the target storage device. For target read / write operation requests that are read operations, the data on the virtual disk needs to be transferred to the host through the target storage device. If data transmission fails, an error code is returned to the host. If data transmission is successful, the target read / write operation request is successfully completed.

[0040] In the technical solution provided in this embodiment, after the target storage device completes the virtual disk through the physical hard disk, after mapping the virtual disk to the host, a block device that can be read and written is formed on the host to save the actual business data. The host only needs to convert the disk data read and write operation request originally sent to the physical storage medium into a target read and write operation request sent to the virtual disk, and the target storage device manages and processes the target read and write operation request, reducing the consumption of the host's own computing resources and memory resources, so that the host can focus on the processing of actual business and ensure the reliability of the host's actual application business. The target storage device will send the target read and write operation request to the target physical storage medium only after the command parameters, current storage environment, storage path and hardware storage conditions are successfully verified, which can not only ensure the efficient execution of data read and write operation requests, but also improve the security of stored data. Since the underlying physical storage medium is not used by the host, it can be used by multiple hosts at the same time, which can effectively improve the utilization rate of the physical storage medium, and finally achieve the effective improvement of the read and write efficiency and security of disk data on the basis of reducing resource consumption and improving disk utilization.

[0041] In order to further improve the processing efficiency of the target storage device for the read and write operation requests, based on the above embodiment, the present invention also provides a method of using a dedicated structure to process the request, which may include the following contents:

[0042] Pre-construct a read / write request structure; obtain the target command type, target command lifecycle, target physical storage medium identification information and logical block address by parsing the target read / write operation request; generate a read / write request structure corresponding to the target read / write operation request based on the target command type, target command lifecycle, target physical storage medium identification information and logical block address, so as to perform read / write operations on the target physical storage medium based on the read / write request structure.

[0043] In this embodiment, the read / write request structure includes a command type, a command lifecycle, physical storage medium identification information, and a storage address; the command type includes a read command and a write command, the physical storage medium identification information may be, for example, a disk number or a disk id, the storage address may include a start address and an end address, which are used to indicate the location of the data on the storage device, and the address may be an LBA (Logical Block Address) used to describe the block where the data is located on the virtual disk, that is, the storage address may include a start LBA (start_lba) and an end LBA (end_laba), and the command lifecycle may include a command arrival time and a command submission time. Exemplarily, the read / write request structure may include a physical storage medium id, a start LBA, an end LBA, a command type, a command arrival time, and a command submission time. After obtaining the target read / write operation request, the command type, command lifecycle, physical storage medium identification information and storage address are read from the target read / write operation request, and then the read / write request structure is filled accordingly. In this way, a read / write request structure uniquely corresponding to the target read / write operation request is generated. Each read / write request structure corresponds to a read / write operation request resource, and each read / write operation request resource corresponds to a read command or a write command. The lifecycle of the read / write operation request resource is from command generation to command execution completion.

[0044] As can be seen from the above, this embodiment defines a read / write request structure on the target storage device and processes the read / write operation request through the read / write request structure, which is beneficial to improving the execution efficiency of the read / write operation request.

[0045] In the above embodiment, there is no limitation on how to determine whether the command parameters of the read and write operation request are normal. Based on the above embodiment, the present invention provides a method for determining whether the command parameters are normal by determining whether the parameters are legal, which may include the following contents:

[0046] Obtain the read / write data range, command type, and data read / write length of the target read / write operation request; obtain the virtual disk read / write range, authorization information of the target physical storage medium, and the allowed data length; when the authorization information is write-protected or read-only, if the command type of the target read / write operation request is a write command, the command parameters of the target read / write operation request are illegal; if the read / write data range is within the virtual disk read / write range, and the data read / write length is less than the allowed data length, the command parameters of the target read / write operation request are normal.

[0047] In this embodiment, the legality of the parameters of the target read / write operation request is determined by the read / write data range, data read / write length, and authorized operation, such as whether the read / write data range exceeds the data read / write range specified by the virtual disk, whether write protection is enabled, and whether the read / write data length exceeds the range supported by the target storage device. If the command parameters are illegal, the target storage device can set the corresponding error code according to the parameters to tell the host which aspect of the command parameters is illegal, so that the host can adjust the behavior of issuing IO read / write operation requests in a timely manner.

[0048] As can be seen from the above, this embodiment determines whether the command parameters are normal by determining the legality of the parameters, which is conducive to timely discovering abnormal commands, timely repairing, and improving the processing efficiency of read and write operation requests.

[0049] In order to further ensure the stability of the target storage device, based on the above embodiment, the present invention further provides a method of using a dedicated thread to process read and write operation requests in a silent state, which may include the following contents:

[0050] A dedicated thread for silent disk is pre-built; when the virtual disk is in silent state, each target read and write operation request is stored in a request waiting queue; when it is detected that a target read and write operation request in the waiting queue is awakened, the dedicated thread for silent disk is called to process the awakened target read and write operation request.

[0051] In this embodiment, it is considered that the same thread may process the read and write operation requests of multiple virtual disks. When the virtual disk is in a silent state, the read and write operation requests arriving at the virtual disk are not processed, but are added to the waiting queue. In the waiting process of the read and write operation requests, in order to avoid wasting resources, the thread / CPU resources cannot always wait for the read and write operation requests, resulting in the read and write operation requests of other virtual disks being unable to obtain computing resources and being unable to be processed. When the read and write operation requests in the waiting queue are awakened, a read and write operation request storm may occur. In order to prevent the normal processing of the read and write operation requests of other virtual disks from being affected and causing storage performance fluctuations, a dedicated thread can be used to process these re-awakened read and write operation requests to ensure the stability of the target storage device.

[0052] As can be seen from the above, this embodiment uses a dedicated thread in the awakened read and write operation request, which does not affect the normal operation of other virtual disks, effectively avoids storage performance shock, and ensures the stability of the target storage device.

[0053] The above embodiment does not limit how to verify the storage status. The present invention also provides an exemplary storage status verification method, which may include the following contents:

[0054] Get the current storage status and disk status of the virtual disk; if the current storage status is normal processing of read and write requests, and the disk status is online, the current storage environment is in a normal state; get the first path status between the target storage device and the target physical storage medium, and the second path status between the target storage device and the host; when the first path status and the second path status are not in a reset state, the storage paths corresponding to the target read and write operation requests are in a normal state; if the target physical storage medium has no bad sectors and is in a connected state, the hardware storage conditions are in a normal state.

[0055] This embodiment determines whether the storage environment is normal based on the storage status and the disk status of the virtual hard disk, and determines whether the storage path is normal based on the data transmission path of the host-target storage device-target physical storage medium. The damage degree and online status of the physical storage medium that has been stored on the disk are used to determine whether the hardware storage conditions are normal. This ensures that the target read and write operation requests are effectively executed, detects abnormal situations in a timely manner and repairs them, improves the processing efficiency and security of read and write operation requests, and avoids data loss.

[0056] In the above embodiment, after the host sends a read / write operation request, the target storage device will receive the target read / write operation request. Under normal procedures, the management process of the target storage device will be entered, and finally the data will be stored on the actual physical storage medium. If an abnormal process occurs, the corresponding processing process will be entered according to different scenarios. The present invention provides corresponding processing processes for different abnormal situations, such as Figure 3 As shown, it may include the following:

[0057] When at least one of the path states of the target storage device, the target physical storage medium, and the host is in a reset state, an error code indicating that the path state is incorrect is sent to the host.

[0058] When the target physical storage medium is in a disconnected state, a notification message that data cannot be written or read is sent to the host.

[0059] If the command type corresponding to the target read / write operation request is a write command, obtain the bad sector data and disconnection count of the target physical storage medium; if the storage address of the data to be written contains the sector where the bad sector is located and / or the disconnection count of the target physical storage medium exceeds a preset threshold, send a notification message to the host that the data cannot be written.

[0060] Get the current storage status and disk status of the virtual disk; if the storage status is damaged or disconnected, all data read and write paths between the host and the storage device are disconnected, and the host enters the timeout processing process; if the storage status is silent, suspend processing the read and write operation requests issued by the host, and store the target read and write operation requests in the pre-built request waiting queue; if the storage status is starting, store the target read and write operation requests in the pre-built request waiting queue, and when the startup is completed, process each target read and write operation request in the order of the time of joining the request waiting queue.

[0061] If the disk status is lost, an error code indicating that the virtual disk is lost is sent to the host, and the virtual disk online monitoring thread is started; when the virtual disk is detected to be online, virtual disk recovery information is sent to the host; if the disk status is silent, the target read and write operation requests are stored in a pre-built request waiting queue, the request time monitoring thread is started, and the read and write operation requests in the request waiting queue whose waiting time exceeds the preset waiting time threshold are discarded.

[0062] In this embodiment, the storage status includes normal processing of read and write operation requests, damaged or disconnected status, silent status, and starting status. Among them, normal processing of read and write operation requests indicates that in a normal state, the target storage device processes the read and write operation requests normally and enters the next step. For the damaged (disconnected) state, the target storage device is completely unable to process the read and write operation requests, the target storage device cannot reply to the read and write operation requests issued by the host, and the host will enter the timeout processing process. The read and write operation request paths from the host to the target storage device are all disconnected. For the silent state, there are higher priority tasks to be processed inside the target storage device, such as a new storage node that needs to be added to the entire storage cluster, and the target storage device that processes the read and write operation requests needs to suspend processing the internal read and write operation requests issued by the host. However, due to the negotiation mechanism between the host and the target storage device, and considering that if too many error codes are returned, the host will think that the target storage device is in a damaged state and no longer has the ability to process the read and write operation requests, and no longer processes the read and write operation requests through the target storage device, so the error code cannot be returned directly, and it can be added to the request waiting queue. For the starting state, when the target storage device is in the starting state, some functions of the target storage device are not ready yet. Directly sending read and write operation requests to the physical storage medium is likely to cause errors such as data inconsistency, so these read and write operation requests need to be added to the waiting queue. After the target storage device is started and all functions are ready, the read and write operation requests in the waiting queue are taken out in chronological order and sent to the physical storage medium.

[0063] Among them, the virtual disk status may include online status, disconnected status and silent status. When the virtual disk is online, the target storage device can directly submit read and write operation requests to the virtual disk. When the virtual disk is disconnected, the target storage device will directly return an error code to the host. After the host receives error read and write operation request information continuously, it will determine that all paths of the virtual disk are lost. After the virtual disk is back online, the target storage device can report the virtual disk recovery information to the host again. When the virtual disk is in silent state, the read and write operation requests sent to the virtual disk will be added to the waiting queue. If the read and write operation requests in the waiting queue time out, in order to save resources, this part of the content can be discarded.

[0064] Among them, when the data transmission path between the target storage device and the host is in the reset state, the target storage device is unable to process the read and write operation requests sent from this path, and directly returns an error code to the host. Whether the read and write operation request is successful: because the target physical storage medium has bad blocks or the hard disk is disconnected, the read and write operation request data cannot be correctly written to the target physical storage medium, and the data cannot be read from the target physical storage medium. The error code can be directly returned to inform the host that the area cannot be written / read, and the data is sent to other areas of the virtual disk.

[0065] From the above, it can be seen that in the process of using the target storage device to manage the internal read and write operation requests generated by the host in this embodiment, different abnormal scenarios correspond to different processing flows, which can not only ensure that the read and write operation requests are executed efficiently, but also effectively guarantee the reliability of the host's actual application business, reduce the consumption of the host's own computing resources and memory resources, and enable the host to focus on the processing of actual business.

[0066] It should be noted that there is no strict order of execution between the steps in the present invention. As long as they comply with the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 1-Figure 3 This is just a schematic and does not mean that this is the only execution order.

[0067] The above embodiment does not limit how to map the virtual disk to the host. The present invention also provides an exemplary mapping method, which maps the virtual disk to the host through physical lines such as IP (Internet Protocol) network links, FC (Ferrule Connector) fiber links, and RDMA (Remote Direct Memory Access) high-speed network links, which may include the following contents:

[0068] First, configure matching network parameters according to the network requirements of the virtual machine, and use command line tools to map the target physical storage medium to the virtual machine. The network parameters include but are not limited to the network adapter type and IP address. The network adapter type is such as bridge mode, NAT mode or host-only mode, and the IP address is static or dynamic, so that the virtual machine can access external resources through the network and perform data communication. Different virtualization platforms provide different command line tools, which can be used to map physical disks to virtual machines. For example, in VMware (a virtualization platform), the vmkfstools (command line tool) command can be used to map physical disks to virtual machines.

[0069] Secondly, configure the fiber switch, target storage device, and server, create storage areas on the fiber switch, divide the storage device into multiple logical storage areas, set access permissions for each logical storage area, configure LUN (Logical Unit Number) mapping, associate each logical storage area with the server so that the server can access and use the data in the storage device, and complete the FC fiber link mapping. The configuration parameters of the fiber switch include but are not limited to the management IP address, creation of VSAN (virtual storage area network), and configuration of ports. The target storage device configuration includes but is not limited to the management IP address and creation of LUN. The server configuration includes but is not limited to the WWN (World Wide Name) of the HBA (Host Bus Adapter) card and the installation and configuration of the fiber driver.

[0070] Finally, install basic libraries, such as iproute (library name), libibverbs (library name), ibibverbs-utils (library name), and infiniband-diags (library name). Check the RDMA connection status, such as using rdma link show (command) to view the current RDMA connection status, and configure RXE (RoCE (RDMA overConverged Ethernet) based on pure software implementation, to achieve efficient data transmission on lossless Ethernet) on the network card. For example, you can use the rxe_cfg addxgbe0 command to start the RDMA service or view the loaded RDMA-related modules. Send information such as the QP (Queue Pair) number, LID (Local Identifier), GID (Global Identifier), memory key, etc. through the connection, then close the TCP / IP channel and switch to RDMA communication.

[0071] As can be seen from the above, this embodiment maps the virtual disk to the host through different physical lines to ensure the security and integrity of the data.

[0072] The present invention also provides a corresponding device for the data reading and writing method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The data reading and writing device provided by the present invention is introduced below. The device is used to implement the data reading and writing method provided by the present invention. In this embodiment, the data reading and writing device may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors to complete the data reading and writing method disclosed in Example 1. The program module referred to in this embodiment refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of the data reading and writing device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The data reading and writing device described below and the data reading and writing method described above can be referred to in correspondence with each other.

[0073] From the perspective of functional modules, see Figure 4 , Figure 4 The structure diagram of the data reading and writing device provided in this embodiment under a specific implementation mode may include:

[0074] The virtualization module 401 is used to pre-assemble at least one target physical storage medium into a virtual disk, and map the virtual disk to a host.

[0075] The request receiving module 402 is used to receive a target read / write operation request sent by the host; the target read / write operation request is obtained by converting the disk data read / write operation request sent by the host to the target physical storage medium.

[0076] The request processing module 403 is used to perform read and write operations on the target physical storage medium based on the target read and write operation request when the command parameters of the target read and write operation request are normal and the current storage environment and the storage path and hardware storage conditions corresponding to the target read and write operation request are all in normal state.

[0077] Exemplarily, in some implementation schemes of the present embodiment, the above-mentioned device may also include a structure generation module, which is used to pre-construct a read and write request structure; the read and write request structure includes a command type, a command life cycle, physical storage medium identification information and a storage address; the command type includes a read command and a write command; by parsing the target read and write operation request, the target command type, target command life cycle, target physical storage medium identification information and logical block address are obtained; based on the target command type, target command life cycle, target physical storage medium identification information and logical block address, a read and write request structure corresponding to the target read and write operation request is generated, so as to perform read and write operations on the target physical storage medium based on the read and write request structure.

[0078] Exemplarily, in some other implementations of the present embodiment, the above-mentioned device may also include a dedicated thread processing module, which is used to pre-build a silent disk dedicated thread; when the virtual disk is in a silent state, each target read and write operation request is stored in a request waiting queue; when it is detected that the target read and write operation request in the waiting queue is awakened, the silent disk dedicated thread is called to process the awakened target read and write operation request.

[0079] Exemplarily, in some other implementations of this embodiment, the request processing module 403 can also be used to: obtain the current storage status and the disk status of the virtual disk; if the current storage status is normal processing of read and write requests, and the disk status is online, the current storage environment is in a normal state; obtain the first path status between the target storage device and the target physical storage medium, and the second path status between the target storage device and the host; when both the first path status and the second path status are not in a reset state, the storage paths corresponding to the target read and write operation requests are in a normal state; if the target physical storage medium has no bad sectors and is in a connected state, the hardware storage conditions are in a normal state.

[0080] Exemplarily, in some other implementations of the present embodiment, the above-mentioned device may also include an exception handling module, which is used to send an error code indicating that the path status is incorrect to the host when at least one of the path statuses of the target storage device, the target physical storage medium, and the host is in a reset state; when the target physical storage medium is in a disconnected state, send a notification message to the host that data cannot be written or read; if the command type corresponding to the target read / write operation request is a write command, obtain the bad sector data and the number of disconnections of the target physical storage medium; if the storage address of the data to be written contains the sector where the bad sector is located and / or the number of disconnections of the target physical storage medium exceeds a preset threshold, send a notification message to the host that data cannot be written.

[0081] Exemplarily, in some other implementations of the present embodiment, the request processing module 403 may be further used to: obtain the read / write data range, command type and data read / write length of the target read / write operation request; obtain the virtual disk read / write range, authorization information of the target physical storage medium and the allowed data length; when the authorization information is write-protected or read-only, if the command type of the target read / write operation request is a write command, the command parameters of the target read / write operation request are illegal; if the read / write data range is within the virtual disk read / write range and the data read / write length is less than the allowed data length, the command parameters of the target read / write operation request are normal.

[0082] Exemplarily, in some other implementations of the present embodiment, the above-mentioned device may also include an exception handling module, which can be used to obtain the storage status and disk status of the virtual disk at the current moment; if the storage status is a damaged state or a disconnected state, all data read and write paths between the host and the storage device are disconnected, and the host enters the timeout processing process; if the storage status is a silent state, the processing of the read and write operation requests issued by the host is suspended, and the target read and write operation requests are stored in a pre-built request waiting queue; if the storage status is a starting state, the target read and write operation requests are stored in a pre-built request waiting queue, and after the startup is completed, each target read and write operation request is processed in sequence according to the order of the time of joining the request waiting queue; if the disk status is a disconnected state, an error code indicating that the virtual disk is disconnected is sent to the host, and a virtual disk online monitoring thread is started; when the virtual disk is detected to be online, virtual disk recovery information is sent to the host; if the disk status is a silent state, the target read and write operation request is stored in a pre-built request waiting queue, a request time monitoring thread is started, and read and write operation requests in the request waiting queue whose waiting time exceeds a preset waiting time threshold are discarded.

[0083] The data reading and writing device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Figure 5 The following is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention in one implementation manner. Figure 5 As shown, the electronic device includes a memory 50 for storing computer programs; a processor 51 for implementing the steps of the data reading and writing method mentioned in any of the above embodiments when executing the computer program.

[0084] Among them, the processor 51 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the processor 51 may also be a controller, a microcontroller, a microprocessor or other data processing chip. The processor 51 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 51 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 51 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 51 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0085] The memory 50 may include one or more computer non-volatile storage media, which may be non-transitory. The memory 50 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the memory 50 may be an internal storage unit of an electronic device, such as a hard disk of a server. In other embodiments, the memory 50 may also be an external storage device of an electronic device, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 50 may also include both an internal storage unit of an electronic device and an external storage device. The memory 50 may not only be used to store application software and various types of data installed in the electronic device, such as: the code of the program in the process of executing the data reading and writing method, but also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 50 is at least used to store the following computer program 501, wherein, after the computer program is loaded and executed by the processor 51, the relevant steps of the data reading and writing method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 50 may also include an operating system 502 and data 503, and the storage method may be temporary storage or permanent storage. The operating system 502 may include Windows, Unix, Linux, etc. The data 503 may include but is not limited to data corresponding to the data reading and writing results.

[0086] In some embodiments, the electronic device may further include a display screen 52, an input / output interface 53, a communication interface 54 or a network interface, a power supply 55 and a communication bus 56. Among them, the display screen 52 and the input / output interface 53, such as a keyboard, belong to the user interface, and exemplary user interfaces may also include standard wired interfaces, wireless interfaces, etc. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 54 may exemplarily include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, etc., which are generally used to establish a communication connection between the electronic device and other electronic devices. The communication bus 56 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0087] It is understandable that if the data reading and writing method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes but is not limited to: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrically erasable programmable ROM, register, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program codes. Based on this, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the data reading and writing method recorded in any of the above embodiments are performed.

[0088] It is understandable that if the data reading and writing method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, the computer software product does not need to be stored in a physical storage medium. For example, it can be directly transmitted to a computer or other device with information processing capabilities through a wired network or a wireless network to perform all or part of the steps of the methods of each embodiment of the present invention. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product. Based on this, the present invention also provides a computer program product, which stores a computer program. When the computer program is executed by a processor, it performs the steps of the data reading and writing method recorded in any of the above embodiments.

[0089] The above is a detailed introduction to a data reading and writing method, electronic device, non-volatile storage medium, computer program product and server provided by the present invention. The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can refer to each other. The units and algorithm steps of each example described in each disclosed embodiment are executed in electronic hardware or computer software depending on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods for each specific application to implement the described functions, and such implementation should not be considered to exceed the scope of the present invention. Without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A data reading and writing method, characterized in that: Applied to target storage devices, including: Preliminarily constructing at least one target physical storage medium into a virtual disk, and mapping the virtual disk to a host; receiving a target read / write operation request sent by the host; the target read / write operation request is obtained by converting a disk data read / write operation request sent by the host to the target physical storage medium; When the command parameters of the target read / write operation request are normal, and the current storage environment and the storage path and hardware storage conditions corresponding to the target read / write operation request are all in normal state, the target physical storage medium is read / written based on the target read / write operation request.

2. The data reading and writing method according to claim 1, characterized in that: After receiving the target read / write operation request sent by the host, the method further includes: Pre-constructing a read / write request structure; the read / write request structure includes a command type, a command life cycle, physical storage medium identification information and a storage address; the command type includes a read command and a write command; By parsing the target read and write operation request, a target command type, a target command life cycle, target physical storage medium identification information and a logical block address are obtained; Based on the target command type, the target command lifecycle, the target physical storage medium identification information and the logical block address, a read / write request structure corresponding to the target read / write operation request is generated to perform read / write operations on the target physical storage medium based on the read / write request structure.

3. The data reading and writing method according to claim 1, characterized in that: After receiving the target read / write operation request sent by the host, the method further includes: Pre-build dedicated thread for silent disk; When the virtual disk is in a silent state, storing each target read and write operation request in a request waiting queue; When it is detected that the target read / write operation request in the waiting queue is awakened, the silent disk dedicated thread is called to process the awakened target read / write operation request.

4. The data reading and writing method according to claim 1, characterized in that: Before performing a read or write operation on the target physical storage medium based on the target read or write operation request, the method further includes: Obtaining the current storage status and the disk status of the virtual disk; If the current storage state is processing read and write requests normally, and the disk state is online, then the current storage environment is in a normal state; Acquire a first path status between a target storage device and a target physical storage medium, and a second path status between the target storage device and the host; When both the first path state and the second path state are not in a reset state, the storage paths corresponding to the target read / write operation request are in a normal state; If the target physical storage medium has no bad sectors and is in a connected state, the hardware storage condition is in a normal state.

5. The data reading and writing method according to claim 1, characterized in that: Before performing a read or write operation on the target physical storage medium based on the target read or write operation request, the method further includes: When at least one of the path states of the target storage device, the target physical storage medium, and the host is in a reset state, an error code indicating that the path state is incorrect is sent to the host; When the target physical storage medium is in a disconnected state, a notification message is sent to the host indicating that data cannot be written or read; If the command type corresponding to the target read / write operation request is a write command, obtaining bad sector data and disconnection times of the target physical storage medium; If the storage address of the data to be written contains the sector where the bad track is located and / or the number of disconnections of the target physical storage medium exceeds a preset number threshold, a notification message that the data cannot be written is sent to the host.

6. The data reading and writing method according to claim 1, characterized in that: After receiving the target read / write operation request sent by the host, the method further includes: Obtain the read / write data range, command type, and data read / write length of the target read / write operation request; Obtain the virtual disk read and write range, authorization information of the target physical storage medium, and the allowed value of data length; When the authorization information is write-protected or read-only, if the command type of the target read-write operation request is a write command, the command parameters of the target read-write operation request are illegal; If the read / write data range is within the virtual disk read / write range, and the data read / write length is less than the data length allowable value, then the command parameters of the target read / write operation request are normal.

7. The data reading and writing method according to any one of claims 1 to 6, characterized in that: The current storage environment includes a storage state and a disk state of the virtual disk. After receiving the target read / write operation request sent by the host, the method further includes: Obtaining the storage status and disk status of the virtual disk at the current moment; If the storage status is a damaged status or a disconnected status, all data read and write paths between the host and the storage device are disconnected, and the host enters a timeout processing flow; If the storage state is a silent state, suspending processing of the read and write operation request sent by the host, and storing the target read and write operation request in a pre-built request waiting queue; If the storage state is in the starting state, the target read / write operation request is stored in a pre-built request waiting queue, and after the start is completed, each target read / write operation request is processed in sequence according to the time of joining the request waiting queue; If the disk status is disconnected, an error code indicating that the virtual disk is disconnected is sent to the host, and a virtual disk online monitoring thread is started; when the virtual disk is detected to be online, virtual disk recovery information is sent to the host; If the disk state is silent, the target read and write operation request is stored in a pre-built request waiting queue, a request time monitoring thread is started, and read and write operation requests in the request waiting queue whose waiting time exceeds a preset waiting time threshold are discarded.

8. A data reading and writing device, characterized in that: include: A virtualization module, used to pre-assemble at least one target physical storage medium into a virtual disk, and map the virtual disk to a host; A request receiving module, used for receiving a target read / write operation request sent by the host; the target read / write operation request is obtained by converting a disk data read / write operation request sent by the host to the target physical storage medium; The request processing module is used to perform read and write operations on the target physical storage medium based on the target read and write operation request when the command parameters of the target read and write operation request are normal and the current storage environment and the storage path and hardware storage conditions corresponding to the target read and write operation request are all in normal state.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to implement the steps of the data reading and writing method according to any one of claims 1 to 7 when executing a computer program stored in the memory.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data reading and writing method according to any one of claims 1 to 7 are implemented.

Images