Data processing method, device and equipment, computer storage medium and computer program product
By directly applying and using the target storage area in the data storage process between the client and the server, the time delay problem caused by multiple interactions in the prior art is solved, and the efficiency of data storage is improved.
Patent Information
- Application Number
- CN202411814145.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-04-25
AI Technical Summary
In the prior art, multiple interactions between the client and the server are required during the data storage process, resulting in a high time delay.
The client responds to the first write request, applies for the target storage area based on the request information and the attribute information of the storage area, and generates a second write request, and sends it to the server, so that the server directly writes the data to be processed.
It reduces the number of interactions between the client and the server, reduces the time delay in the data storage process, and improves the efficiency of data storage.
Smart Images
Figure CN119883973A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to data processing technology in the computer field, and in particular to a data processing method, device, equipment, computer storage medium and computer program product. Background Art
[0002] With the development of science and technology, the traditional way of storing data through Transmission Control Protocol / Internet Protocol (TCP / IP) can no longer meet the high-performance storage needs of users. Remote Direct Memory Access (RDMA) is a technology that can bypass the operating system kernel of the remote host to directly access data in the memory. It can not only save the resources of the Central Processing Unit (CPU), but also reduce the communication delay of the network system. Based on this, RDMA technology has been increasingly widely used in the field of data storage technology. However, in the data storage scheme of the related technology, after receiving the data write request, the client and the server usually need to interact multiple times to realize data storage, which leads to a high time delay in the data storage process. Summary of the invention
[0003] To solve the above technical problems, the embodiments of the present application hope to provide a data processing method, device, equipment, computer storage medium and computer program product, which solve the problem of high time delay in the data storage process in the related technology and improve the efficiency of data storage.
[0004] The technical solution of this application is implemented as follows:
[0005] A data processing method, the method comprising:
[0006] In response to a first write request for data to be processed, applying for a target storage area based on request information of the first write request and first attribute information of the storage area stored by the client;
[0007] generating a second write request based on the request information and the area information of the target storage area;
[0008] The second write request is sent to the server, so that the server writes the to-be-processed data using the target storage area based on the second write request.
[0009] In the above solution, applying for a target storage area based on the request information of the first write request and the first attribute information of the storage area stored by the client includes:
[0010] Determine a target capacity based on the capacity of the storage area and the data size of the data to be processed; wherein the first attribute information includes the capacity; and the request information includes the data size;
[0011] Based on the target capacity, a target storage area for the data to be processed is determined.
[0012] In the above solution, generating a second write request based on the request information and the area information of the target storage area includes:
[0013] Determine a target address for the data to be processed based on the first attribute information and the target storage area;
[0014] Based on the permission information for the storage area and the target storage area, determining target permission information for the target storage area; wherein the permission information is sent by the server;
[0015] The second write request is generated based on the target address, the target permission information and the data to be processed carried in the request information; wherein the region information includes the target address and the target permission information.
[0016] In the above solution, generating the second write request based on the target address, the target permission information and the data to be processed carried in the request information includes:
[0017] Marking the target address, and encapsulating the marked target address to obtain a first message;
[0018] Encapsulating the target authority information and the data to be processed to obtain a second message;
[0019] Based on the first message and the second message, the second write request is generated.
[0020] In the above scheme, the method further includes:
[0021] receiving feedback information sent by the server in response to the second write request; wherein the feedback information represents a writing result of the data to be processed;
[0022] Based on the feedback information, the mark of the target address is deleted.
[0023] A data processing method, the method comprising:
[0024] Receive a second write request sent by a client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; the target storage area is applied by the client based on the request information and the first attribute information of the storage area of the server;
[0025] After determining that the operation authority exists based on the target authority information in the area information, the data to be processed is written into the storage area corresponding to the target address in the target storage area.
[0026] In the above solution, the receiving the second write request sent by the client includes:
[0027] Receive the second write request sent by the client and carrying the first message and the second message; wherein the first message carries the target address with a mark; and the second message carries the target permission information and the data to be processed.
[0028] In the above scheme, the method further includes:
[0029] Sending feedback information for the second write request to the client; wherein the feedback information represents a writing result of the data to be processed.
[0030] In the above scheme, the method further includes:
[0031] Sending first attribute information of the storage area and permission information for the storage area to the client; wherein the first attribute information includes the address and capacity of the storage area.
[0032] A first data processing device, comprising:
[0033] A first processing unit, configured to respond to a first write request for data to be processed, and apply for a target storage area based on request information of the first write request and first attribute information of a storage area of the server stored by a client;
[0034] a second processing unit, configured to generate a second write request based on the request information and the area information of the target storage area;
[0035] A sending unit is used to send the second write request to the server, so that the server uses the target storage area to write the data to be processed based on the second write request.
[0036] A second data processing device, comprising:
[0037] a second receiving unit, configured to receive a second write request sent by a client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; and the target storage area is applied for by the client based on the request information and the first attribute information of the storage area of the server;
[0038] The third processing unit is configured to write the data to be processed into a storage area corresponding to the target address in the target storage area after determining that the data has the operation authority based on the target authority information in the area information.
[0039] A client comprises: a first processor, a first memory and a first communication bus;
[0040] The first communication bus is used to realize the communication connection between the first processor and the first memory;
[0041] The first processor is used to execute the data processing program in the first memory to implement the steps of the above-mentioned data processing method.
[0042] A server, comprising: a second processor, a second memory, and a second communication bus;
[0043] The second communication bus is used to realize the communication connection between the second processor and the second memory;
[0044] The second processor is used to execute the data processing program in the second memory to implement the steps of the above-mentioned data processing method.
[0045] A distributed storage system, comprising:
[0046] The client is used to respond to a first write request for data to be processed, apply for a target storage area based on request information of the first write request and first attribute information of a storage area of the server stored by the client; generate a second write request based on the request information and area information of the target storage area; send the second write request to the server, so that the server uses the target storage area to write the data to be processed based on the second write request;
[0047] The server is used to receive a second write request sent by a client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; the target storage area is applied for by the client based on the request information and the first attribute information of the storage area of the server; after determining that the operation authority is available based on the target authority information in the area information, the data to be processed is written to the target storage area.
[0048] A computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the above-mentioned data processing method.
[0049] A computer program product comprises a computer program, wherein the computer program implements the above data processing method when executed by a processor.
[0050] The data processing method, apparatus, device, computer storage medium and computer program product provided by the embodiments of the present application, the client can respond to a first write request for the data to be processed, and apply for a target storage area based on the request information of the first write request and the first attribute information of the storage area of the server stored in the client, and then generate a second write request based on the request information and the area information of the target storage area, and then send the second write request to the server, so that the server uses the target storage area to write the data to be processed based on the second write request. In this way, as long as the client receives the write request for processing the data to be processed, it can directly apply for the target storage area according to the first attribute information and request information of the storage area of the server stored in advance, and then send the area information of the target storage area to the server, so that the server directly writes the data to be processed, instead of requiring multiple interactions between the client and the server to write the data as in the related art, thereby solving the problem of high time delay in the data storage process in the related art and improving the efficiency of data storage. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;
[0052] Figure 2 A flowchart of another data processing method provided in an embodiment of the present application;
[0053] Figure 3 A flowchart of a data processing method provided for another embodiment of the present application;
[0054] Figure 4 A schematic diagram of the structure of a first data processing device provided in an embodiment of the present application;
[0055] Figure 5 A schematic diagram of the structure of a second data processing device provided in an embodiment of the present application;
[0056] Figure 6 A schematic diagram of the structure of a client provided in an embodiment of the present application;
[0057] Figure 7 A schematic diagram of the structure of a server provided in an embodiment of the present application;
[0058] Figure 8 A schematic diagram of a distributed storage system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0059] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0060] It should be understood that the "embodiments of the present application" or "the aforementioned embodiments" mentioned throughout the specification mean that specific features, structures or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, "in the embodiments of the present application" or "in the aforementioned embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.
[0061] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0062] The present application embodiment provides a data processing method, which can be applied to a client, the client is connected to a server, and the server includes a storage area, referring to Figure 1 As shown, the method may include the following steps:
[0063] Step 101 : respond to a first write request for data to be processed, and apply for a target storage area based on request information of the first write request and first attribute information of the storage area stored in the client.
[0064] The storage area of the server includes the target storage area.
[0065] In an embodiment of the present application, the storage area may refer to the memory area of the server; the request information may include the data to be processed and the attribute information of the data to be processed; the first attribute information may include the capacity and address of the storage area, etc.; the attribute information of the data to be processed may include parameters such as the data type, data format and data size of the data to be processed.
[0066] In the embodiment of the present application, after receiving the first write request sent by the application, the client will respond to the first write request, and then determine the target capacity according to the attribute information of the data to be processed and the first attribute information of the storage area, and then determine the area required to process the data to be processed (i.e., the target storage area) according to the target capacity. It should be noted that the first write request can be one or more, that is, the application can send only one write request through the terminal, or it can send multiple write requests at the same time. At this time, the client will also receive multiple write requests at the same time.
[0067] Step 102: Generate a second write request based on the request information and the area information of the target storage area.
[0068] In an embodiment of the present application, the region information may include address information of the target storage region and permission information for the target storage region. Specifically, the region information may be determined based on the first attribute information of the storage region, the permission information for the storage region, and the target storage region, and then a second write request may be generated based on the request information and the region information.
[0069] It should be noted that the permission information for the storage area is sent to the client in advance by the server.
[0070] Step 103: Send the second write request to the server, so that the server uses the target storage area to write the data to be processed based on the second write request.
[0071] In an embodiment of the present application, the client can carry the request information of the first write request and the area information of the target storage area in the second write request and send it to the server, so that after receiving the second write request, the server can write the data to be processed to the corresponding target storage area based on the area information.
[0072] The data processing method provided by the embodiment of the present application is that as long as the client receives a write request for processing the data to be processed, it can directly apply for the target storage area according to the first attribute information and request information of the storage area of the server stored in advance, and then send the area information of the target storage area to the server, so that the server can directly write the data to be processed, instead of requiring multiple interactions between the client and the server to write the data as in the related art. This solves the problem of high time delay in the data storage process in the related art and improves the efficiency of data storage.
[0073] Based on the above embodiments, the embodiments of the present application provide a data processing method, which can be applied to a server, referring to Figure 2 As shown, the method may include the following steps:
[0074] Step 201: Receive a second write request sent by a client.
[0075] The second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; the target storage area is applied for by the client based on the request information and the first attribute information of the storage area of the server.
[0076] In an embodiment of the present application, after receiving a first write request issued by an application for processing data to be processed, the client can respond to the first write request and apply for a target storage area for the data to be processed based on the first attribute information of the storage area and the request information of the first write request, and then carry the request information and the area information of the target storage area in a second write request and send it to the server.
[0077] Step 202: After determining that the operation authority exists based on the target authority information in the area information, write the data to be processed into the target storage area.
[0078] In an embodiment of the present application, after receiving the second write request, the server can obtain the pending data carried in the write request information and the operation permission information for the target storage area (i.e., target operation permission). Thereafter, after determining that the client has the operation permission for the target storage area based on the target permission information, the pending data obtained from the second write request is written to the target storage area determined by the client.
[0079] The data processing method provided by the embodiment of the present application is that after the server receives the request information carrying the first write request and the second write request carrying the area information of the applied target storage area, it can determine whether it has the operation permission to the target storage area according to the target permission information in the area information, and directly write the to-be-processed data to the target storage area after determining that it has the operation permission to the target storage area. In this way, only one interaction is required between the client and the server to realize data writing, instead of multiple interactions between the client and the server to realize data writing as in the related art. This solves the problem of high time delay in the data storage process in the related art and improves the efficiency of data storage.
[0080] Based on the above embodiments, the embodiments of the present application provide a data processing method, referring to Figure 3 As shown, the method may include the following steps:
[0081] Step 301: The server sends first attribute information of a storage area and permission information for the storage area to a client.
[0082] The first attribute information includes the address and capacity of the storage area; the storage area is a storage area used for data writing.
[0083] In an embodiment of the present application, the server can apply for part of the area from its own memory area as a storage area for data writing, that is, divide a memory pool (i.e., storage area) of a specified capacity from its own memory area for data writing, and send the address, capacity, and target permission information of the determined storage area to the client. It should be noted that the instruction includes the capacity of the memory pool to be established, and the capacity is determined based on the attribute information of the historical transmission data and the memory capacity of the server itself, wherein the attribute information of the historical transmission data may include data volume, data size, data type, etc.
[0084] In a feasible implementation, the client can send an instruction to the server in advance to instruct the establishment of a memory pool (i.e., a storage area), and after receiving the instruction, the server will divide an area of specified capacity and address from its own memory area according to the information carried in the instruction as a storage area for writing data. After the storage area is successfully divided, the server will return a successful registration message. It should be noted that when the server returns the registration information to the client, it will send the permission information for the storage area and the first attribute information representing the capacity and address of the storage area to the client.
[0085] Step 302: The client receives first attribute information of the storage area and permission information for the storage area sent by the server.
[0086] The storage area is a storage area in the server for writing data; the first attribute information represents the address and capacity of the storage area.
[0087] In an embodiment of the present application, after receiving the first attribute information and permission information for the storage area sent by the server, the client will store the first attribute information and permission information to obtain operation permissions for the storage area and determine the target address for the data to be processed based on the first attribute information when receiving a request to write data.
[0088] Step 303: The client responds to the first write request and determines a target capacity based on the capacity of the storage area and the size of the data to be processed.
[0089] Among them, the first attribute information includes capacity; the request information includes data size.
[0090] In the embodiment of the present application, the client can determine the target capacity required for writing the data to be processed in combination with the capacity of the storage area and the data size of the data to be processed. In one achievable manner, if the data size of the data to be processed is smaller than the capacity of the target storage area, the determined target capacity is also smaller than the capacity of the storage area; if the data size of the data to be processed is greater than or equal to the capacity of the storage area, the target capacity is the capacity of the storage area.
[0091] Step 304: The client determines a target storage area for the data to be processed based on the target capacity.
[0092] In the embodiment of the present application, after determining the target capacity, the client may determine a target storage area with the target capacity for the data to be processed.
[0093] Step 305: The client determines a target address for the data to be processed based on the first attribute information and the target storage area for the data to be processed.
[0094] In the embodiment of the present application, after determining the target storage area with the target capacity, the client will determine the target storage area with the same capacity (i.e., the target capacity) from the storage area, and then determine the target address of the target storage area according to the address of the storage area. It should be noted that the target storage area is an area for writing the data to be processed.
[0095] Step 306: The client determines target permission information for the target storage area based on the permission information for the storage area and the target storage area.
[0096] Among them, the permission information is sent by the server.
[0097] In the embodiment of the present application, after determining the target storage area for the data to be processed, the client can determine the permission information for the target storage area based on the permission information of the storage area sent by the server. Specifically, if it is determined that all areas in the storage area for the server have write permission, then it can be determined that the target storage area must have write permission; then, if not all areas in the storage area of the server have write permission, it is necessary to determine whether the area with write permission includes the target storage area. If the target storage area is included, it can be determined that the target storage area has write permission; correspondingly, if part of the target storage area is included or the target storage area is not included, it can be determined that the target storage area does not have write permission.
[0098] Step 307: The client generates a second write request based on the target address, target permission information, and the data to be processed carried in the request information.
[0099] The area information includes the target address and target authority information.
[0100] In the embodiment of the present application, step 307 can be implemented through steps 307a to 307c.
[0101] Step 307a: The client marks the target address and encapsulates the marked target address to obtain a first message.
[0102] In an embodiment of the present application, the first message may refer to a control message, and the control message may be used to instruct the server to start its own hardware device to perform data processing operations.
[0103] In one achievable manner, the client marks the target address for the data to be processed in the address of the storage area, and encapsulates the marked target address and information such as the write instruction to obtain a control message. The write instruction can be used to instruct the server to directly perform a data write operation. It should be noted that the specific marking method of the target address is not specifically limited here.
[0104] In an embodiment of the present application, by marking the target address, the client can use the target storage area corresponding to the target address only for writing the current data to be processed until the data to be processed is written, and will not allocate the target storage area to other data to be processed before the data writing is completed.
[0105] Step 307b: The client encapsulates the target authority information and the data to be processed to obtain a second message.
[0106] In an embodiment of the present application, the second message may refer to a data message. Specifically, the client may encapsulate the target permission information for the target storage area and the related information of the data such as the data to be processed, and obtain a data message for providing the data to be processed and the data operation authority. In an achievable manner, according to the requirements of the RMDA-based computer system interface extension (Internet Small Computer System Interface Extensions for RDMA, iSER) protocol adopted by the RDMA technology, the related information of the data may also include the target address for the data to be processed.
[0107] Step 307c: The client generates a second write request based on the first message and the second message.
[0108] In an embodiment of the present application, the client will carry the control message (ie, the first message) and the data message (ie, the second message) together in a new write request (ie, the second write request).
[0109] Step 308: The client sends a second write request to the client.
[0110] In an embodiment of the present application, the client sends a second write request carrying the first message and the second message to the server, so that the server performs a write operation on the data to be processed according to the control message and the data message.
[0111] Step 309: The server receives a second write request sent by the client and carrying the first message and the second message.
[0112] The first message carries a marked target address; the second message carries target authority information and data to be processed for a target storage area.
[0113] In an embodiment of the present application, after receiving the second write request sent by the client, the server can parse the first message and the second message in the second write request, obtain the marked target address of the target storage area carried in the first message, and the target permission information and data to be processed for the target storage area carried in the second message.
[0114] Step 310: After determining that the server has the operation authority based on the target authority information in the area information, the server writes the data to be processed into the target storage area.
[0115] In an embodiment of the present application, after obtaining a control message (i.e., the first message), the server learns from the write instruction carried in the control message that a write operation must be performed on the data to be processed, and obtains from the control message the target address of the storage area where the data to be processed is to be written. Thereafter, based on the target permission information for the target storage area, it determines that the client has operation authority over the target storage area, and writes the data to be processed obtained from the data message into the target storage area with the target address in the storage area obtained from the control message in accordance with the write instruction.
[0116] It should be noted that steps 311 to 312 may be further executed after step 310 .
[0117] Step 311: The server sends feedback information regarding the second write request to the client.
[0118] The feedback information represents the writing result of the data to be processed.
[0119] In an embodiment of the present application, after the server writes the data to be processed into the target storage area in the storage area according to the second write request, the server can use the RDMA technology to carry the writing result of the data to be processed in the feedback information and send it to the client. It should be noted that if the data to be processed is written successfully, the feedback information is the information that the data is written successfully; if the data to be processed fails to be written, the feedback information includes not only the information that the data fails to be written, but also the error description when the input write fails.
[0120] Step 312: The client receives feedback information sent by the server regarding the second write request, and deletes the mark of the target address based on the feedback information.
[0121] The feedback information represents the writing result of the data to be processed.
[0122] In an embodiment of the present application, after receiving the feedback information sent by the server, the client will synchronously send the feedback information to the application that issued the first write request, and at the same time delete the mark on the target address of the marked target storage area to continue to allocate the target storage area to other data to be processed.
[0123] According to the data processing method provided by the embodiment of the present application, the server can send the first attribute information of the target storage area and the permission information for the target storage area to the client, so that the client can receive in advance the target storage area and the corresponding permission information for data writing sent by the server, and after receiving the first write request for processing the data to be processed, the target address, permission information and the data to be processed can be directly sent to the server so that the server can write the data, instead of requiring multiple interactions between the client and the server to write the data as in the related art. This solves the problem of high time delay in the data storage process in the related art and improves the efficiency of data storage.
[0124] Based on the above embodiments, the embodiments of the present application provide a first data processing device, which can be applied to Figure 1 and 3 In the data processing method provided in the corresponding embodiment, refer to Figure 4 As shown, the first data processing device 4 may include: a first processing unit 41, a second processing unit 42 and a sending unit 43, wherein:
[0125] The first processing unit 41 is used to respond to a first write request for the data to be processed, and apply for a target storage area based on the request information of the first write request and the first attribute information of the storage area of the server stored by the client;
[0126] A second processing unit 42, configured to generate a second write request based on the request information and the area information of the target storage area;
[0127] The sending unit 43 is used to send the second write request to the server, so that the server uses the target storage area to write the data to be processed based on the second write request.
[0128] In other embodiments of the present application, the first processing unit 41 is further configured to perform the following steps:
[0129] Determine the target capacity based on the capacity of the storage area and the data size of the data to be processed; wherein the first attribute information includes the capacity; and the request information includes the data size;
[0130] Based on the target capacity, determine the target storage area for the data to be processed.
[0131] In other embodiments of the present application, the second processing unit 42 is further configured to perform the following steps:
[0132] Determine a target address for the data to be processed based on the first attribute information and the target storage area;
[0133] Determine target permission information for the target storage area based on the permission information for the storage area and the target storage area; wherein the permission information is sent by the server;
[0134] A second write request is generated based on the target address, the target permission information, and the data to be processed carried in the request information; wherein the region information includes the target address and the target permission information.
[0135] In other embodiments of the present application, the second processing unit 42 is further configured to perform the following steps:
[0136] Marking the target address, and encapsulating the marked target address to obtain a first message;
[0137] Encapsulating the target authority information and the data to be processed to obtain a second message;
[0138] A second write request is generated based on the first message and the second message.
[0139] In other embodiments of the present application, the second processing unit 42 is further configured to perform the following steps:
[0140] receiving feedback information for the second write request sent by the server; wherein the feedback information represents a write result of the data to be processed;
[0141] Based on the feedback information, the mark of the target address is deleted.
[0142] It should be noted that the specific implementation process of the steps performed by each unit in the embodiment of the present application can be referred to Figure 1 and 3 The implementation process of the data processing method provided in the corresponding embodiment will not be repeated here.
[0143] The first data processing device provided in the embodiment of the present application can directly apply for a target storage area based on the first attribute information and request information of the storage area of the server stored in advance, as long as the client receives a write request for processing the data to be processed, and then send the area information of the target storage area to the server so that the server can write data, instead of requiring multiple interactions between the client and the server to write data as in the related art. This solves the problem of high time delay in the data storage process in the related art and improves the efficiency of data storage.
[0144] Based on the above embodiments, the embodiments of the present application provide a second data processing device, which can be applied to Figure 2 and 3 In the data processing method provided in the corresponding embodiment, refer to Figure 5As shown, the second data processing device 5 may include: a second receiving unit 51 and a third processing unit 52, wherein:
[0145] The second receiving unit 51 is used to receive a second write request sent by the client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; the target storage area is applied by the client based on the request information and the first attribute information of the storage area of the server;
[0146] The third processing unit 52 is used to write the data to be processed into the target storage area after determining that the operation authority exists based on the target authority information in the area information.
[0147] In other embodiments of the present application, the second receiving unit 51 is further configured to perform the following steps:
[0148] A second write request sent by a client and carrying a first message and a second message is received; wherein the first message carries a marked target address; and the second message carries target authority information and data to be processed.
[0149] In other embodiments of the present application, the third processing unit 52 is further configured to perform the following steps:
[0150] Send feedback information for the second write request to the client; wherein the feedback information represents the writing result of the data to be processed.
[0151] In other embodiments of the present application, the third processing unit 52 is further configured to perform the following steps:
[0152] Sending first attribute information of the storage area and permission information for the storage area to the client; wherein the first attribute information includes the address and capacity of the storage area.
[0153] It should be noted that the specific implementation process of the steps performed by each unit in the embodiment of the present application can be referred to Figure 2 and 3 The implementation process of the data processing method provided in the corresponding embodiment will not be repeated here.
[0154] The second data processing device provided by the embodiment of the present application can determine whether the server has the operation permission to the target storage area according to the target permission information in the area information after receiving the request information of the first write request and the second write request carrying the area information of the applied target storage area, and directly write the data to be processed to the target storage area after determining that it has the operation permission to the target storage area. In this way, only one interaction is required between the client and the server to realize data writing, instead of multiple interactions between the client and the server to write data as in the related art. This solves the problem of high time delay in the data storage process in the related art and improves the efficiency of data storage.
[0155] Based on the above embodiments, the embodiments of the present application provide a client, which can be applied to Figure 1 and 3 In the data processing method provided in the corresponding embodiment, refer to Figure 6 As shown, the client 6 may include: a first processor 61, a first memory 62 and a first communication bus 63, wherein:
[0156] The first communication bus 63 is used to realize the communication connection between the first processor 61 and the first memory 62;
[0157] The first processor 61 is used to execute the data processing program in the first memory 62 to implement the following steps:
[0158] In response to a first write request for the data to be processed, applying for a target storage area based on request information of the first write request and first attribute information of the storage area stored by the client;
[0159] generating a second write request based on the request information and the area information of the target storage area;
[0160] The second write request is sent to the server, so that the server performs write processing on the data to be processed using the target storage area based on the second write request.
[0161] In other embodiments of the present application, the first processor 61 is used to execute the data processing program in the first memory 62 to apply for a target storage area based on the request information of the first write request and the first attribute information of the storage area stored by the client, so as to implement the following steps:
[0162] Determine the target capacity based on the capacity of the storage area and the data size of the data to be processed; wherein the first attribute information includes the capacity; and the request information includes the data size;
[0163] Based on the target capacity, determine the target storage area for the data to be processed.
[0164] In other embodiments of the present application, the first processor 61 is used to execute the data processing program in the first memory 62, based on the request information and the area information of the target storage area, to generate a second write request to implement the following steps:
[0165] Determine a target address for the data to be processed based on the first attribute information and the target storage area;
[0166] Determine target permission information for the target storage area based on the permission information for the storage area and the target storage area; wherein the permission information is sent by the server;
[0167] A second write request is generated based on the target address, the target permission information, and the data to be processed carried in the request information; wherein the region information includes the target address and the target permission information.
[0168] In other embodiments of the present application, the first processor 61 is used to execute the data processing program in the first memory 62, based on the target address, the target permission information and the data to be processed carried in the request information, to generate a second write request to implement the following steps:
[0169] Marking the target address, and encapsulating the marked target address to obtain a first message;
[0170] Encapsulating the target authority information and the data to be processed to obtain a second message;
[0171] A second write request is generated based on the first message and the second message.
[0172] In other embodiments of the present application, the first processor 61 is used to execute the data processing program in the first memory 62, and may also implement the following steps:
[0173] receiving feedback information for the second write request sent by the server; wherein the feedback information represents a write result of the data to be processed;
[0174] Based on the feedback information, the mark of the target address is deleted.
[0175] It should be noted that the specific description of the steps performed by the first processor can be referred to Figure 1 and 3 The data processing method provided in the corresponding embodiment will not be described in detail here.
[0176] The client provided in the embodiment of the present application can directly apply for the target storage area according to the first attribute information and request information of the storage area of the server stored in advance, as long as the client receives a write request for processing the data to be processed, and then send the area information of the target storage area to the server so that the server can write the data, instead of requiring multiple interactions between the client and the server to write the data as in the related art. This solves the problem of high time delay in the data storage process in the related art and improves the efficiency of data storage.
[0177] Based on the above embodiments, the embodiments of the present application provide a server, which can be applied to Figure 2 and 3 In the data processing method provided in the corresponding embodiment, refer to Figure 7 As shown, the server 7 may include: a second processor 71, a second memory 72, and a second communication bus 73, wherein:
[0178] The second communication bus 73 is used to realize the communication connection between the second processor 71 and the second memory 72;
[0179] The second processor 71 is used to execute the data processing program in the second memory 72 to implement the following steps:
[0180] receiving a second write request sent by a client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; the target storage area is applied by the client based on the request information and the first attribute information of the storage area of the server;
[0181] After determining that the operation authority exists based on the target authority information in the area information, the data to be processed is written to the target storage area.
[0182] In other embodiments of the present application, the second processor 71 is used to execute the data processing program in the second memory 72 to receive the second write request sent by the client, so as to implement the following steps:
[0183] A second write request sent by a client and carrying a first message and a second message is received; wherein the first message carries a marked target address; and the second message carries target authority information and data to be processed.
[0184] In other embodiments of the present application, the second processor 71 is used to execute the data processing program in the second memory 72, and may also implement the following steps:
[0185] Send feedback information for the second write request to the client; wherein the feedback information represents the writing result of the data to be processed.
[0186] In other embodiments of the present application, the second processor 71 is used to execute the data processing program in the second memory 72, and may also implement the following steps:
[0187] Sending first attribute information of the storage area and permission information for the storage area to the client; wherein the first attribute information includes the address and capacity of the storage area.
[0188] It should be noted that the specific description of the steps performed by the second processor can be referred to Figure 2 and 3 The data processing method provided in the corresponding embodiment will not be described in detail here.
[0189] The server provided by the embodiment of the present application can determine whether it has the operation permission to the target storage area according to the target permission information in the area information after receiving the request information of the first write request and the second write request carrying the area information of the applied target storage area, and directly write the to-be-processed data to the target storage area after determining that it has the operation permission to the target storage area. In this way, only one interaction is required between the client and the server to realize data writing, instead of multiple interactions between the client and the server to realize data writing as in the related art. This solves the problem of high time delay in the data storage process in the related art and improves the efficiency of data storage.
[0190] Based on the above embodiments, the embodiments of the present application provide a distributed storage system, which can be applied to Figures 1 to 3 In the data processing method provided in the corresponding embodiment, refer to Figure 8 As shown, the distributed storage system 8 may include: a client 81 and a server 82, wherein:
[0191] The client 81 is used to respond to a first write request for the data to be processed, apply for a target storage area based on the request information of the first write request and the first attribute information of the storage area of the server stored by the client; generate a second write request based on the request information and the area information of the target storage area; send the second write request to the server, so that the server uses the target storage area to write the data to be processed based on the second write request;
[0192] Server 82 is used to receive a second write request sent by the client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; the target storage area is applied for by the client based on the request information and the first attribute information of the storage area of the server; after determining that the operation authority is available based on the target authority information in the area information, the data to be processed is written to the target storage area.
[0193] In an embodiment of the present application, after receiving a write request for processing the data to be processed, the client can directly apply for a target storage area for the data to be processed based on the first attribute information and request information of the storage area of the server stored in advance, and send the request information of the first write request and the area information of the target storage area to the server. Thereafter, after receiving the request information carrying the data to be processed and the second attribute information of the data to be processed, the server can directly write the data based on the request information and the relevant information of the target storage area corresponding to the data to be processed. That is to say, the distributed file system provided in the embodiment of the present application only requires one interaction to realize data writing, instead of requiring multiple interactions between the client and the server to write data as in the related art, thereby reducing the time delay in the data writing process and improving the efficiency of data storage.
[0194] The distributed storage system provided by the embodiment of the present application can send the first attribute information of the storage area of the server to the client in advance, so that when the client receives the first write request for the data to be processed, it can directly send the area information of the target storage area requested according to the first attribute information and the request information to the server, so that the server can write the data, which solves the problem of high time delay in the data storage process in the related technology and improves the efficiency of data storage.
[0195] Based on the foregoing embodiments, the embodiments of the present application provide a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement Figures 1 to 3 The corresponding embodiments provide steps of the data processing method.
[0196] Based on the above embodiments, an embodiment of the present application provides a computer program product, the computer program product includes a computer program, the computer program is executed by a processor to achieve Figures 1 to 3 The corresponding embodiments provide steps of the data processing method.
[0197] It should be noted that the above-mentioned computer-readable storage medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM) and other memories; it can also be various electronic devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.
[0198] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0199] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0200] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0201] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0202] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0203] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
Claims
1. A data processing method, characterized in that: Applied to a client, the client is connected to a server, the server includes a storage area, and the method includes: In response to a first write request for data to be processed, applying for a target storage area based on request information of the first write request and first attribute information of the storage area stored by the client; generating a second write request based on the request information and the area information of the target storage area; The second write request is sent to the server, so that the server uses the target storage area to write the data to be processed based on the second write request.
2. The method according to claim 1, characterized in that Applying for a target storage area based on the request information of the first write request and the first attribute information of the storage area stored by the client includes: Determine a target capacity based on the capacity of the storage area and the data size of the data to be processed; wherein the first attribute information includes the capacity; and the request information includes the data size; Based on the target capacity, a target storage area for the data to be processed is determined.
3. The method according to claim 1, characterized in that The generating a second write request based on the request information and the area information of the target storage area includes: Determine a target address for the data to be processed based on the first attribute information and the target storage area; Based on the permission information for the storage area and the target storage area, determining target permission information for the target storage area; wherein the permission information is sent by the server; The second write request is generated based on the target address, the target permission information and the data to be processed carried in the request information; wherein the region information includes the target address and the target permission information.
4. The method according to claim 3, characterized in that The generating the second write request based on the target address, the target permission information and the data to be processed carried in the request information includes: Marking the target address, and encapsulating the marked target address to obtain a first message; Encapsulating the target authority information and the data to be processed to obtain a second message; Based on the first message and the second message, the second write request is generated.
5. The method according to claim 4, characterized in that The method further comprises: receiving feedback information sent by the server in response to the second write request; wherein the feedback information represents a writing result of the data to be processed; Based on the feedback information, the mark of the target address is deleted.
6. A data processing method, characterized in that: Applied to a server, the method comprises: Receive a second write request sent by a client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; the target storage area is applied by the client based on the request information and the first attribute information of the storage area of the server; After determining that the operation authority exists based on the target authority information in the area information, the to-be-processed data is written into the target storage area.
7. The method according to claim 6, characterized in that The receiving a second write request sent by the client includes: receiving the second write request sent by the client and carrying a first message and a second message; wherein the first message carries a target address with a mark; and the second message carries the target permission information and the data to be processed; Accordingly, the method further includes: Sending feedback information for the second write request to the client; wherein the feedback information represents a writing result of the data to be processed.
8. The method according to claim 6, characterized in that The method further comprises: Sending first attribute information of the storage area and permission information for the storage area to the client; wherein the first attribute information includes the address and capacity of the storage area.
9. A first data processing device, characterized in that: The device comprises: A first processing unit, configured to respond to a first write request for data to be processed, and apply for a target storage area based on request information of the first write request and first attribute information of a storage area of the server stored by a client; a second processing unit, configured to generate a second write request based on the request information and the area information of the target storage area; A sending unit is used to send the second write request to the server, so that the server uses the target storage area to write the data to be processed based on the second write request.
10. A second data processing device, characterized in that: The device comprises: a second receiving unit, configured to receive a second write request sent by a client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; and the target storage area is applied for by the client based on the request information and the first attribute information of the storage area of the server; The third processing unit is configured to write the to-be-processed data into the target storage area after determining that the operating authority exists based on the target authority information in the area information.
11. A client, characterized in that: The client comprises: a first processor, a first memory and a first communication bus; The first communication bus is used to realize the communication connection between the first processor and the first memory; The first processor is used to execute the data processing program in the first memory to implement the steps of the data processing method according to any one of claims 1 to 5.
12. A server, characterized in that: The server comprises: a second processor, a second memory and a second communication bus; The second communication bus is used to realize the communication connection between the second processor and the second memory; The second processor is used to execute the data processing program in the second memory to implement the steps of the data processing method according to any one of claims 6 to 8.
13. A distributed storage system, characterized in that: The system comprises: The client is used to respond to a first write request for data to be processed, apply for a target storage area based on request information of the first write request and first attribute information of a storage area of the server stored by the client; generate a second write request based on the request information and area information of the target storage area; send the second write request to the server, so that the server uses the target storage area to write the data to be processed based on the second write request; The server is used to receive a second write request sent by a client; wherein the second write request is generated by the client based on the request information of the first write request for the data to be processed and the area information of the target storage area; the target storage area is applied for by the client based on the request information and the first attribute information of the storage area of the server; after determining that the operation authority is available based on the target authority information in the area information, the data to be processed is written to the target storage area.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the data processing method according to any one of claims 1 to 5 or 6 to 8.
15. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the computer program implements the data processing method according to any one of claims 1 to 5 or 6 to 8.