Data processing method, memory and distributed system

Abstract

The invention provides a data processing method, a memory and a distributed system. According to the method, a cross-node data processing service is unloaded from a CPU to a first memory to be executed, a service request issued by a processor can be received based on cross-node data communication of the memory, and service processing sub-requests are distributed to other memories according to the service type of the service request. And processing results of the service processing sub-requests fed back by other memories in response to the service processing sub-requests are obtained. Based on the data processing capability of the memory, the memory can realize data processing tasks in the memory, and meanwhile, the memory also executes service requests according to processing results fed back by other memories. According to the method, the cross-node data processing service in the distributed system is realized, the processing capability of a single memory is prevented from becoming the performance bottleneck of the distributed system, the performance overhead of a CPU (Central Processing Unit) is reduced, and the data processing performance and efficiency of the distributed system are improved.

Description

Technical Field

[0001] This application relates to the field of information and communication technology, and in particular to a data processing method, a memory, and a distributed system. Background Technology

[0002] Cross-node data processing is a widespread business requirement in distributed systems, typically executed jointly by the processor and network interface card (NIC) of a storage node. However, because the bandwidth of a processor is often lower than that of an SSD and NIC, the processor is not very efficient when performing related data processing tasks.

[0003] To improve processor data processing efficiency, a new type of SSD (Solid State Drive) with integrated compute units has recently been proposed. This allows some data computation tasks in data processing to be offloaded from the processor to the compute units of the SSD. However, SSDs with integrated compute units can only perform data computation tasks within the SSD itself and cannot handle data processing tasks between multiple SSDs within a storage node or across storage nodes.

[0004] We hope there can be an improved solution that can better enable cross-node data processing through SSDs. Summary of the Invention

[0005] This specification describes one or more embodiments of a data processing scheme that reduces CPU performance overhead and improves the performance and efficiency of data processing in a distributed system by offloading cross-node data processing tasks from the CPU to memory.

[0006] In a first aspect, this application provides a data processing method applied to a first memory, the method comprising: receiving a service request issued by a processor; the service request including a service type and service processing parameters; wherein the service request is used to request the first memory to process requested service data according to the service processing parameters; sending a service processing sub-request to a second memory according to the service type; and executing the service request according to the processing result fed back by the second memory in response to the service processing sub-request.

[0007] In this application, the memory possesses cross-node data communication and data processing capabilities. Based on the memory's cross-node data communication, it can receive service requests issued by the processor and distribute service processing sub-requests to other memories according to the service type of the service request, thereby obtaining the processing results of the service processing sub-requests from other memories. Based on the memory's data processing capabilities, the memory can implement data processing tasks within the memory itself. Simultaneously, the memory also executes service requests based on the processing results fed back by other memories. This method realizes cross-node data processing services in a distributed system, avoiding the processing capacity of a single memory becoming a performance bottleneck in the distributed system, reducing CPU performance overhead, and improving the performance and efficiency of data processing in the distributed system.

[0008] In some possible implementations, the first memory includes a peer-to-peer internet module; sending a service processing sub-request to the second memory according to the service type includes: sending the service processing sub-request to the second memory via the peer-to-peer internet module according to the service type.

[0009] In this application, the data communication capability between the memory provided by the peer-to-peer interconnection network module and multiple computing nodes and other memories, the first memory can distribute business processing sub-requests so that multiple memories can cooperate to complete business requests, thereby improving the performance and efficiency of data processing in the distributed system.

[0010] In some possible implementations, sending a service processing sub-request to the second memory according to the service type includes: dividing the service request into a first service processing sub-request and a second service processing sub-request according to the service type; sending the second service processing sub-request to the second memory; the second service processing sub-request carrying the service processing parameters, so that the second memory executes the second service processing sub-request according to the service processing parameters, wherein the first service processing sub-request is a request processed by the first memory.

[0011] In this application, the data processing capability based on memory can divide a business request into multiple business processing sub-requests. Through the cross-node communication capability of memory, multiple business processing sub-requests are distributed to multiple corresponding memory locations, and multiple memory locations jointly execute a business request. This achieves distributed processing of business requests, avoids the processing capability of a single memory becoming a performance bottleneck of the distributed system, and improves the performance and efficiency of data processing in the distributed system.

[0012] In some possible implementations, sending a service processing sub-request to the second memory according to the service type includes: sending a service processing sub-request to the second memory according to the service type, including: determining, according to the service type, that a first service data is located in the first memory and a second service data is located in the second memory; generating a third service processing sub-request and sending the third service processing sub-request to the second memory to request the second service data to be retrieved from the second memory.

[0013] In this application, based on the cross-node data communication capability of memory, cross-node data transfer can be realized without the processor needing to read and write network data, thus freeing up processor resources and improving the performance and efficiency of data processing in the distributed system, as well as the read and write performance of the distributed system.

[0014] In some possible implementations, the service request is executed based on the processing result fed back by the second memory in response to the service processing sub-request, including: obtaining the second service data fed back by the second memory; obtaining the first service data from the storage area of ​​the first memory; and processing the first service data and the second service data according to the service processing parameters to obtain processed service data.

[0015] In this application, based on the cross-node data communication and data processing capabilities of the memory, the memory can obtain business data on other memories and process it, thereby improving the performance and efficiency of data processing in the distributed system.

[0016] In some possible implementations, the service request may also include a destination address, and the method may further include storing the processed service data in the memory corresponding to the destination address.

[0017] In this application, based on the ability of memory to communicate across nodes, processed business data can be input to other memories, realizing data transfer across nodes in a distributed system and improving the read and write performance of the distributed system.

[0018] In some possible implementations, the processed service data is stored in the memory corresponding to the destination address, and then the process further includes deleting the first service data and the second service data according to the service processing parameters.

[0019] In this application, based on the ability of memory to communicate across nodes, the memory can request to delete business data in other memories, thereby reducing data redundancy in the distributed system and improving the performance and efficiency of data processing in the distributed system.

[0020] In some possible implementations, the service type includes data reconstruction, and the service processing parameters include the algorithm configuration of erasure coding and EC check matrix used when performing EC check during data reconstruction; or the service type includes garbage collection, and the service processing parameters include the execution order of data marking and data deletion when performing garbage collection; or the service type includes data compression, and the service processing parameters include the compression algorithm configuration used when performing data compression; or the service type includes data filtering and screening, and the service processing parameters include the screening conditions issued by the computing node.

[0021] In this application, based on the data processing capabilities of memory, corresponding data reconstruction, garbage collection, data compression, and data filtering and screening operations can be performed according to the business processing parameters of different business types. This can offload data processing tasks that restrict processor performance to memory for execution, freeing up processor resources and improving the data processing performance of distributed systems.

[0022] Secondly, this application provides a memory, the memory comprising: a communication interface; a controller, which receives a service request issued by any processor through the communication interface; the service request includes a service type and service processing parameters; wherein the service request is used to request the memory to process the requested service data according to the service processing parameters; to send a service processing sub-request to a second memory according to the service type; and to obtain the processing result fed back by the second memory in response to the service processing sub-request, and execute the service request.

[0023] In this application, a business request issued by any processor can be received through a communication interface. By integrating a data processing module into the controller, cross-node data processing tasks in a distributed system can be implemented, improving the performance, efficiency, input, and output performance of the distributed system's data processing.

[0024] In some possible implementations, the memory further includes a peer-to-peer interconnection network module; the controller sends a service processing sub-request to the second memory according to the service type, specifically for sending the service processing sub-request to the second memory through the peer-to-peer interconnection module.

[0025] In this application, by setting a peer-to-peer interconnection network module in the memory, data communication between the memory and multiple computing nodes and other memories is realized. The first memory can distribute business processing sub-requests, enabling multiple memories to cooperate in completing business requests, thereby improving the performance and efficiency of data processing in the distributed system. In some possible embodiments, the controller sends business processing sub-requests to the second memory according to the business type, specifically for:

[0026] Based on the service type, it is determined that the first service data is located in the first memory and the second service data is located in the second memory; a third service processing sub-request is generated and sent to the second memory to request the second service data to be retrieved from the second memory.

[0027] In some possible implementations, the controller executes the service request based on the processing result fed back by the second memory in response to the service processing sub-request, specifically by: obtaining the second service data fed back by the second memory; obtaining the first service data from the storage area of ​​the first memory; and processing the first service data and the second service data according to the service processing parameters to obtain processed service data.

[0028] Thirdly, this application provides a distributed system, which includes a first memory and a second memory, and a plurality of other memories. The first memory receives a service request issued by a processor, the service request including a service type and service processing parameters. The service request is used to request the first memory to process the requested service data according to the service processing parameters. According to the service type, a service processing sub-request is sent to the second memory. The processing result fed back by the second memory in response to the service processing sub-request is obtained, and the service request is executed.

[0029] In some possible implementations, the first memory includes a peer-to-peer internet module, and the first memory sends a service processing sub-request to the second memory according to the service type, specifically for: sending the service processing sub-request to the second memory through the peer-to-peer internet module.

[0030] In some possible implementations, the first memory sends a service sub-request to the second memory according to the service type, specifically for: determining that the first service data is located in the first memory and the second service data is located in the second memory; and sending the service processing sub-request to the second memory to request the second service data to be retrieved from the second memory.

[0031] In some possible implementations, the first memory sends a service sub-request to the second memory according to the service type, specifically for: determining, according to the service type, that the first service data is located in the first memory and the second service data is located in the second memory; generating a third service processing sub-request and sending the third service processing sub-request to the second memory to request the second service data to be obtained from the second memory.

[0032] Fourthly, this application proposes a computer-readable storage medium including computer program instructions, which, when executed by a memory, cause the memory to perform the aforementioned data processing method.

[0033] Fifthly, this application proposes a computer program product containing instructions that, when executed by a memory, cause the memory to perform the aforementioned data processing method.

[0034] It should be understood that the beneficial effects achieved by the technical solutions and corresponding possible implementations of the third, fourth and fifth aspects of the embodiments of this application can be referred to the above-described technical effects of the first and second aspects, and will not be repeated here. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of a cross-node data processing scenario in a distributed system in the prior art;

[0036] Figure 2 This is a schematic diagram of a data processing scenario in a distributed system provided in an embodiment of this application;

[0037] Figure 3 This is a schematic diagram of the data structure for business requests related to data processing.

[0038] Figure 4 This is a diagram illustrating the first method of generating business processing sub-requests based on business requests;

[0039] Figure 5 This is a diagram illustrating the second method of generating business processing sub-requests based on business requests;

[0040] Figure 6 This is a schematic diagram illustrating a data processing scenario where business data related to business requests is distributed across multiple storage devices.

[0041] Figure 7 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0042] Figure 8 This is a schematic diagram of the structure of a memory provided in an embodiment of this application. Detailed Implementation

[0043] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0044] To facilitate understanding of the technical solution of this application, the relevant terms used in this document are explained below.

[0045] Garbage collection: Used to centralize valid data and release storage space occupied by garbage data in a distributed system. Garbage collection effectively manages the lifecycle of data in a distributed system, ensuring data reliability and efficient use of storage space.

[0046] Data reconstruction: Used to restore the integrity and availability of data and maintain the reliability of system data in the event of data loss in a distributed system. Illustratively, the causes of data loss can include hardware failure, software defects, misoperation, network problems, etc.

[0047] Cross-node data processing is a widely used business requirement in distributed systems. Typically, the data processing tasks are executed by the processor and network interface card of a storage node in the distributed system.

[0048] Indicative Figure 1 The diagram illustrates a scenario of cross-node data processing in a known distributed system, such as... Figure 1 As shown, a known distributed system includes storage nodes 1 to n, each of which includes a network interface card (NIC), a processor, and memory. Schematic, storage node 1 includes memory 1 and memory 2. Storage node n includes memory 3 and memory 4. This is in the case where business data related to business requests is distributed across memory 1 and memory 3 of storage nodes.

[0049] First, the processor of storage node 1 needs to communicate with other storage nodes via the network interface card, for example... Figure 1 The storage node n shown sends a service request to request the reading of service data on storage node n.

[0050] Then, the processor of storage node n responds to the service request, obtains the service data from memory 3, and feeds back the aforementioned service data to storage node 1.

[0051] The processor of storage node 1 receives the service data fed back by storage node n in response to the service request through the network card, and obtains the service data on its own memory 1.

[0052] Then, the processor of storage node 1 performs data reconstruction or garbage collection related data processing tasks on the above business data to obtain the processed business data.

[0053] Finally, the processor of storage node 1 stores the processed business data to the memory 4 of storage node n and / or its own memory 2 via the network card.

[0054] However, because the bandwidth of a processor is often lower than that of an SSD and a network card, the processor is not efficient when performing network data read / write, data reconstruction, and garbage collection-related data processing tasks. Processor performance typically becomes the bottleneck for data processing tasks in a distributed system, limiting the system's read / write performance.

[0055] To improve the data processing efficiency of processors, a new type of SSD (Solid State Drive) with integrated computing units has recently been proposed. Storage nodes can use commands to offload the data computation portion of data processing tasks from the processor to the computing units of the SSD.

[0056] However, SSDs with integrated computing units can only perform data computation within the SSD itself, and can only accelerate the data computation process for data reconstruction and garbage collection. They cannot perform data migration of processed business data, or realize data processing between multiple SSDs within a storage node, or across storage nodes.

[0057] Therefore, how to better realize cross-node data processing business through SSDs has become an urgent technical problem to be solved.

[0058] This application provides a data processing method. This method offloads cross-node data processing tasks from the CPU to a first memory for execution. Based on memory-based cross-node data communication, it can receive service requests issued by the processor and distribute service processing sub-requests to other memories according to the service type of the service request, thereby obtaining the processing results of the service processing sub-requests from other memories. Based on the data processing capabilities of the memory, the memory can implement its own data processing tasks, and simultaneously execute service requests based on the processing results from other memories. This method realizes cross-node data processing in a distributed system, avoids the processing capacity of a single memory becoming a performance bottleneck, reduces CPU performance overhead, and improves the performance and efficiency of data processing in the distributed system.

[0059] For example, Figure 2 This is a schematic diagram of a data processing scenario in a distributed system provided in an embodiment of this application, such as... Figure 2 As shown, the distributed system includes multiple computing nodes and multiple storage devices. The multiple computing nodes, including computing node 1 to computing node n, can be, but are not limited to, computing devices such as desktop computers, laptops, and servers.

[0060] Multiple storage devices include independently configured storage devices 200 (first storage) to 300 (second storage), which may include, but are not limited to, hard disk drives (HDDs), solid-state drives (SSDs), and hybrid drives. The SSD interfaces include: Serial ATA (SATA), M.2 (PCI Express M.2 Specification), U.2 (SFF-8639), and Peripheral Component Interconnect Express (PCIE) interfaces. M.2 interface SSDs mainly include: SSDs supporting the Serial ATA (SATA) transmission channel and using the Advanced Host Controller Interface (AHCI) standard protocol; and SSDs supporting the PCIE transmission channel and using the Non-Volatile Memory Express (NVME) standard protocol.

[0061] The compute node 100 includes a processor 110 and a network interface card 120.

[0062] Processor 110 is the computing and control core of computing node 100. Processor 110 may include one or more processing units. For example, processor 110 may include one or more of the following: application processor (AP), modem, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0063] Network interface card 120 is used to implement computing nodes.

[0064] 100 provides communication connections with other computing nodes and memory 200 to memory 300. Network interface card 120 may include one or more of the following: Ethernet network interface card, wireless network interface card, and fiber optic network interface card.

[0065] In this application, processor 110 is used to generate data processing service requests, which are associated with at least one of the memories 200 to 300. Processor 110 can send the service request to a memory for execution. Each service request includes at least one service processing sub-request.

[0066] Processor 110 can also send multiple service requests to all or more of the memories 200 to 300 according to a load balancing strategy, so that all or more memories can jointly execute the multiple service requests. These memories may or may not be the memories involved in the service requests.

[0067] The memory 200 includes a peer-to-peer interconnection module 210, a controller 220, a data processing module 230, and a storage medium 240.

[0068] The peer-to-peer (P2P) network module 210 is a network communication module based on the Peer-to-Peer (P2P) protocol. By setting up a peer-to-peer (P2P) network module on top of the storage devices, multiple storage devices can achieve data communication between multiple computing nodes and multiple storage devices in a distributed system without relying on computing nodes. Each storage device can both provide services to other storage devices and enjoy the services provided by other storage devices.

[0069] The memory 200 can communicate with other memories, as well as compute nodes 1 to n, via the peer-to-peer interconnection module 210.

[0070] Controller 220 is the main control chip for controlling and managing the read and write operations of memory 200. It is responsible for handling all data read and write requests, error correction, bad block management, and other tasks of memory 200. This includes, but is not limited to, microcontroller units (MCUs), complex programmable logic devices (CPLDs), and field-programmable gate array (FPGA) chips that support serial (SATA, Serial Advanced Technology Attachment) interfaces, high-speed serial computer expansion bus (PCIe, peripheral component interconnect express) interfaces, and non-volatile memory host controller (NVMe) interfaces.

[0071] In this application, the controller 220 can receive service requests issued by the processor 110 through the peer-to-peer interconnection network module 210. The service requests include service type and service processing parameters.

[0072] Controller 220 can send a service processing sub-request to memory 300 according to the service type of the service request. Controller 220 can also receive the processing result fed back by memory 300 in response to the service processing sub-request through peer-to-peer interconnection module 210, and execute the above-mentioned service request.

[0073] The data processing module 230, as the data processing unit of the memory 200, has the capabilities of Erasure Coding (EC), Redundant Arrays of Independent Disks (RAID), Cyclic Redundancy Check (CRC), data comparison, and data compression. In actual implementation, the data processing module 230 can be a separate MCU in the memory 200, or it can be integrated as a circuit module into the controller 220. This application does not limit the implementation method of the data processing module 230.

[0074] In this application, the data processing module 230 is used to process the business data related to the business processing sub-request according to the business processing parameters, obtain the processed business data, and feed back the processing result of the business processing sub-request to the controller 220.

[0075] Storage medium 240 is used to store business data and / or processed business data. Depending on the type of memory, the storage medium may include, but is not limited to, disk arrays, and non-volatile memory such as NAND flash memory chips and NOR flash memory chips; this application does not limit the scope of the storage medium.

[0076] Indicatively, in some possible implementations, the controller 220 can divide a service request into a first service processing sub-request and a second service processing sub-request based on the service type. The first service processing sub-request is a request processed by the memory 200, and the second service processing sub-request is a request processed by the memory 300. Both the first and second service processing sub-requests carry service processing parameters.

[0077] The controller 220 sends the service processing parameters in the first service processing sub-request to the data processing module 230, so that the data processing module 230 processes the first service processing sub-request according to the service processing parameters in the first service processing sub-request, and receives the processing result of the first service processing sub-request fed back by the data processing module 230.

[0078] The controller 220 sends a second service processing sub-request to the memory 300 and receives the processing result of the second service processing sub-request from the memory 300.

[0079] The controller 220 is also used to generate the processing result of the above-mentioned service request based on the processing results of the first service processing sub-request and the second service processing sub-request, and send the processing result of the service request to the processor 110.

[0080] In some possible implementations, the service data related to the aforementioned service request may include service data (first service data) located in memory 200 and service data (second service data) located in memory 300.

[0081] Controller 220 can generate a third business sub-request based on the business type of the business request.

[0082] The controller 220 sends a third service processing sub-request to the memory 300 to request the second service data to be retrieved from the memory 300, and sends the second service data to the data processing module 230.

[0083] The controller 220 sends a service request to the data processing module 230, requesting the data processing module 230 to obtain the first service data from the storage area of ​​the memory 200 itself, process the first service data and the second service data according to the service processing parameters of the service request, and feed back the processing result of the service request to the controller 220.

[0084] As mentioned earlier, a business request may include the business type and business processing parameters related to the business data. Depending on the business type, the business request may also include the source address and / or the destination address corresponding to the source address.

[0085] The following content will explain the business requests for data processing.

[0086] For example, Figure 3 This is a diagram illustrating the data structure of a business request for data processing, such as... Figure 3 As shown, the service requests issued by processor 110 can also include a service request pipeline composed of multiple service requests. Illustratively, the aforementioned service request pipeline may include service request 1, service request 2, and service request 3. Illustratively, each service request includes a service type, service processing parameters, source address, and destination address.

[0087] As one possible implementation, the memory 200 can divide multiple service requests in the service request pipeline into multiple service processing sub-requests according to the service type in the service request, and send the multiple service processing sub-requests to multiple corresponding memories.

[0088] Indicative Figure 4 This is a diagram illustrating the first method of generating business processing sub-requests based on business requests, as shown below. Figure 4 As shown, the business request pipeline includes Figure 3 The examples shown are business request 1, business request 2, and business request 3.

[0089] The memory 200 can generate a service processing sub-request 1 based on service request 1. Service processing sub-request 1 includes the same service type, service processing parameters, source address field, and destination address as service request 1. The source address of the service request includes the IP address of the memory associated with the service data and the storage medium address of that memory. The destination address of the service request includes the IP address of the memory associated with the processed service data and the storage medium address of that memory.

[0090] It is understood that the service processing sub-request 2, which requires processing by memory 2 (not shown in the figure), and the service processing sub-request 3, which requires processing by memory 300, also include the same service type, service processing parameters, source address, and destination address as the corresponding service request.

[0091] As one possible implementation, the memory 200 can also use the controller 220 to refine a certain service request, divide the source address of the service request, and generate corresponding first service processing sub-requests and second service sub-requests.

[0092] For example, Figure 5The diagram shown illustrates the second method of generating business processing sub-requests based on business requests, such as... Figure 5 As shown, memory 200 will Figure 3 The storage medium address corresponding to the source address of the service request 3 shown is divided into storage medium address 1 and storage medium address 2, and service processing sub-request 4, which needs to be processed by memory 200, and service processing sub-request 5, which needs to be processed by memory 300, are generated.

[0093] In business processing sub-request 4, the business type, business processing parameters, and destination address are the same as in business request 3. The destination address corresponds to the IP address of memory 300 and the address of the storage medium; the source address's IP address corresponds to the IP address of memory n, and the address of the storage medium is storage medium address 1 of memory n.

[0094] The service type, service processing parameters, and destination address in service processing sub-request 5 are the same as those in service request 3. The destination address corresponds to the IP address of memory 300 and the address of the storage medium; the source address's IP address corresponds to the IP address of memory n, and the address of the storage medium is storage medium address 2 of memory n.

[0095] The following content will introduce the service type, service processing parameters, source address, and destination address.

[0096] The service type describes the type of service request. Illustratively, service types can include garbage collection, data reconstruction, data compression, data filtering, and other services. Based on the service type, the storage system can send a service processing sub-request to at least one storage system (target storage system) related to the service request. For example… Figure 4 The business processing sub-requests 2 and 3 shown are either Figure 5 The business processing sub-request 5 is shown. The target storage can perform data migration, data marking, data deletion, and data compression operations on the data (business data) related to the business request based on the business processing parameters in the above business processing sub-request.

[0097] Business processing parameters are used to describe the specific parameters when processing business data in a business request. For example, they specify the corresponding operations to be performed on the processed business data based on the business type, including the following four cases.

[0098] I. When a business request's business type is data reconstruction, the business processing parameters include the erasure coding and EC checksum algorithm configuration used during the EC (Erasure Coding) verification of the data reconstruction operation. The EC checksum algorithm configuration refers to the parameters or settings used to customize and adjust the behavior of an EC checksum matrix when implementing or using it. Business processing parameters may also include the data operations to be performed after completing the business processing sub-request. For example, after completing the data reconstruction business processing sub-request, the memory can store the processed business data at the destination address corresponding to the business processing sub-request.

[0099] II. When the service type is garbage collection and / or data compression, the service processing parameters include the execution order of data marking and data deletion during garbage collection. When data compression is also required after garbage collection, the service processing parameters also include the algorithm configuration of the compression algorithm used for the data compression operation. The algorithm configuration refers to the parameters or settings used to customize and adjust the behavior of a compression algorithm when implementing or using it. After completing the service request for garbage collection and / or data compression, the memory 200 can delete the service data from the source address corresponding to the service processing sub-request.

[0100] Third, when the service type is to delete duplicate data, the storage can delete the service data in the source address of the service request according to the service processing parameters of the service request.

[0101] IV. When the service type is data filtering or screening, the service processing parameters include the screening conditions issued by the computing device 10. The memory can obtain the service data of the service request from the source address of the memory related to the service data according to the service processing parameters, determine the relevant data that meets the screening conditions in the above service data according to the above screening conditions, and send the screening result of the service data to the processor.

[0102] The source address specifies the source address of the service data for a particular service request. The source address points to the IP address of at least one memory associated with the service data, as well as the address of the storage medium of that memory.

[0103] Indicative, such as Figure 2 As shown, the storage related to business data includes storage 200 and storage 300. The source address of the business request points to the IP address of storage 200 and the storage medium address of storage 200, respectively, and points to the IP address of storage 300 and the storage medium address of storage 300.

[0104] The destination address specifies the destination address of the service data for a particular service request. The destination address points to the IP address of at least one memory associated with the processed service data, as well as the address of the storage medium of that memory.

[0105] Indicative, such as Figure 2 As shown, the memory associated with the processed service data includes memory 200 and memory 300. The destination address associated with the processed service data points to the IP address of memory 200 and the storage medium address of memory 200, respectively, and points to the IP address of memory 300 and the storage medium address of memory 300.

[0106] It is worth noting that when the business type of the business request and the business processing sub-request is to delete duplicate data, the destination address field may not be included in the business request and the business processing sub-request.

[0107] As an example and not a limitation, the addresses of the storage media corresponding to the source and destination addresses of the business data in this application may include, but are not limited to, logical addresses of storage media in the form of LBA, Key, Object ID, etc. This application does not limit them.

[0108] Next, through the following content and in combination Figure 2 This paper introduces data processing methods for waste recycling.

[0109] When the type of the business request is garbage collection, the business request includes the business type, business processing parameters, source address related to the business data, and destination address related to the processed business data.

[0110] As mentioned earlier, the business processing parameters for garbage collection include the data marking and data deletion execution order during garbage collection operations. When data compression is also required after garbage collection, the business processing parameters also include the algorithm configuration of the compression algorithm used for the data compression operation.

[0111] The business data determined by controller 220 based on the business request includes first business data related to the first business processing sub-request and second business data related to the second business processing sub-request.

[0112] The memory 200 can utilize the controller 220 to divide the service request into a first service processing sub-request processed by the memory 200 (first memory) and a second service processing sub-request processed by the memory 300 (second memory) according to the service type of garbage collection. The first and second service processing sub-requests include the service type and service processing parameters.

[0113] The source and destination IP addresses of the first service data point to memory 200, and the storage medium address points to the address in storage medium 240 of memory 200; the source and destination IP addresses of the second service data point to memory 300, and the storage medium address points to the address in storage medium 340 of memory 300.

[0114] The memory 200 can use the controller 220 to send the service processing parameters in the first service processing sub-request to the data processing module 230 according to the service type of the first service processing sub-request.

[0115] The data processing module 230 obtains the first service data from the storage medium 240, marks invalid data in the first service data according to the service processing parameters of the first service processing sub-request, and deletes the marked invalid data to obtain the processed service data. In some possible implementations, after the data processing module 230 completes the garbage collection of the service data, it also needs to compress the processed service data according to the compression algorithm specified in the service processing parameters, and feed back the processing result of the first service data to the controller 220.

[0116] By way of example and not limitation, the compression algorithm proposed in this application may be one or more of lossless compression algorithms and lossy compression algorithms. This application does not limit the scope of the application.

[0117] The data processing module 230 can input the processed first business data to the address in the storage medium 240 pointed to by the destination address.

[0118] In some possible implementations, when the source address of the service processing parameter points to an address in the storage medium 240, after executing the first service processing sub-request of the garbage collection type, the controller 220 may also delete the service data (first service data) in the storage medium 240.

[0119] Based on the service type of the second service processing sub-request, the memory 200 sends the second service processing sub-request to the memory 300 via the peer-to-peer Internet module 210, requesting the memory 300 to process the second service processing sub-request according to the service processing parameters in the second service processing sub-request.

[0120] Please continue reading. Figure 2 On the right side, the memory 300 can use the peer-to-peer interconnection network module 310 to receive the second service processing sub-request sent by the memory 200 and send the second service processing sub-request to the controller 320.

[0121] The controller 320 parses the second service processing sub-request to obtain the service type and service processing parameters of the second service processing sub-request, and sends the service processing parameters to the data processing module 330.

[0122] The data processing module 330 obtains the second business data from the storage medium 340, processes the second business data in the same way as the data processing module 230, and feeds back the processing result of the second business data to the controller 230.

[0123] The data processing module 330 can input the processed second business data to the address in the storage medium 340 pointed to by the destination address.

[0124] It is understandable that after executing the second business processing sub-request of the garbage collection type, the controller 220 may also request the controller 320 to delete the business data (second business data) in the storage medium 340.

[0125] The controller 320 sends the feedback result of the second service processing sub-request to the memory 200 via the peer-to-peer interconnection network module 310. The feedback result of the second service processing sub-request includes the processing result of the second service data.

[0126] Please continue reading. Figure 2 On the left side, the memory 200 can use the peer-to-peer interconnection network module 210 to receive the feedback result of the second service processing sub-request sent by the memory 300, and send the feedback result of the second service processing sub-request to the controller 220.

[0127] The controller 220 generates the processing result of the business request corresponding to garbage collection based on the processing result of the first business data and the processing result of the second business data, and sends the above processing result to the processor 110 via the network card 120 through the interconnection network module 210.

[0128] Next, through the following content and in combination Figure 6 This section introduces data processing methods for data reconstruction. Please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram illustrating a data processing scenario where business data related to business requests is distributed across multiple storage devices. For example... Figure 6 As shown, after parsing the service request, the controller 220 determines that the first service data is located in the memory 200 (first memory) and the second service data is located in the memory 300 (second memory).

[0129] The IP addresses of the source and destination addresses related to the first service data point to memory 200, and the storage medium address points to the storage area of ​​memory 200. The IP addresses of the source and destination addresses related to the second service data point to memory 300, and the storage medium address points to the storage area of ​​memory 300.

[0130] When the business request type is data reconstruction, the business request includes the business type, business processing parameters, the source address related to the business data, and the destination address related to the processed business data. The business processing parameters include the erasure coding used during EC (Erasure Coding) verification of the data reconstruction operation, and the algorithm configuration of the EC checksum matrix.

[0131] The controller 220 generates a third service processing sub-request based on the service request to request the acquisition of second service data from the second memory. The third service sub-request includes the source address related to the second service data.

[0132] The memory 200 can use the controller 220 to send a third service request to the memory 300 via the peer-to-peer interconnection module 210, based on the IP address of the source address, to obtain the processing result fed back by the memory 300 in response to the third service processing sub-request. The processing result fed back by the third service processing sub-request includes second service data.

[0133] In response to the third service processing sub-request, the memory 300 uses the controller 320 to obtain the second service data from the storage medium 340 of the memory 300 according to the storage medium address of the source address, and sends the second service data to the memory 200 through the peer-to-peer interconnection module 310.

[0134] The memory 200 uses the controller 220 to receive the second service data through the peer-to-peer interconnection network module 210, and sends the second service data to the data processing module 230.

[0135] The data processing module 230 obtains the first service data from the storage medium 240, and processes the first service data and the second service data according to the erasure code and EC check matrix algorithm configuration used in the data reconstruction operation specified by the service processing parameters, to obtain the processed service data and the check code of the processed service data, and feeds back the processing results of the first service data and the second service data to the controller 220.

[0136] By way of example and not limitation, the verification algorithms proposed in this application include, but are not limited to, one or more of CRC, EC check, parity check, and checksum. When EC check is selected, the business processing parameters also include the EC check matrix used by the EC check.

[0137] The data processing module 230 can send the processed business data and the verification code of the processed business data to the address in the storage medium 240 according to the destination address related to the first business data.

[0138] The controller can send the processed service data and the checksum of the processed service data to the storage area of ​​the memory 300 through the peer-to-peer interconnection network module 310, based on the IP address of the destination address and the storage medium address.

[0139] The controller 220 generates the processing result of the service request based on the processing result of the first service data and the processing result of the second service data, and sends the above processing result to the processor 110 via the network card 120 through the interconnection network module 210.

[0140] It is worth noting that, Figure 2 and Figure 6 The embodiments shown are merely illustrative. For example, the division of hardware modules in the memory is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or components may be combined or integrated into another component, or some features may be ignored or not executed. At the same time, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0141] Next, based on the content described above, a data processing method provided by an embodiment of this application will be introduced. It is understood that this method is proposed based on the content described above, and some or all of the content of this method can be found in the description above.

[0142] Please see Figure 7 , Figure 7 This is a flowchart illustrating a data processing method provided in an embodiment of this application, as shown below. Figure 7 As shown, data processing in a distributed system can be implemented using S710 to S730. It can be understood that, when performing data processing in a distributed system, this method is... Figure 2 At least one of the memory 200 to memory 300 shown is executed.

[0143] S710: Receives service requests from any processor.

[0144] As mentioned earlier, distributed systems include Figure 2 The computing nodes 1 to n and the independently configured memory 200 to 300 are shown.

[0145] When a computing node needs to perform data processing tasks, it can offload the data processing-related task requests to a memory through its own processor, and the peer-to-peer interconnection network module 210 of that memory will receive the task requests.

[0146] As one possible implementation, the processor 110 can also, based on a load balancing strategy, allocate... Figure 3 Service requests 1, 2, and 3 are sent to all or more memories in memories 200 to 300, and are executed jointly by all or more memories. These memories may or may not be the memories involved in the service requests.

[0147] Schematic example, taking computing node 1 as an example, computing node 1 can use processor 110 to send data processing service requests from network interface card 200 to peer-to-peer interconnection network module 210 in memory 200 (first memory). In some possible implementations, processor 110 can also send a service pipeline consisting of multiple service requests to peer-to-peer interconnection network module 210 in memory 200.

[0148] The memory 200 can parse one or more service requests and determine the service type and service processing parameters of one or more service requests.

[0149] The business types of the aforementioned business requests and business processing sub-requests include data reconstruction, and the business processing parameters include the erasure code and the algorithm configuration of the EC check matrix used when performing the data reconstruction operation.

[0150] The business type includes garbage collection, and the business processing parameters include the execution order of data marking and data deletion when performing garbage collection operations.

[0151] The business type includes data compression, and the business processing parameters include the configuration of the compression algorithm used to perform the data compression operation.

[0152] The business types include data filtering and screening, and the business processing parameters include the screening conditions issued by the processor.

[0153] The service request may also include a source address related to the service data; depending on the service type, the service request may also include a destination address related to the processed service data. The source address points to the IP address and storage medium address of at least one memory related to the service data; the destination address points to the IP address and storage medium address of at least one memory related to the processed service data.

[0154] S720: Send a service processing sub-request to the second solid-state storage according to the service type.

[0155] In some possible implementations, the memory 200 can generate multiple corresponding service processing sub-requests based on the service type in the service request and multiple service requests in the service request pipeline, so as to request multiple memories to jointly execute the multiple service requests in the aforementioned service request pipeline.

[0156] For example, memory 200 is based on Figure 3 The illustrated service request pipeline includes service request 1, which generates service processing sub-request 1 (first service processing sub-request) processed by memory 200, and other service processing sub-requests processed by memory, such as service processing sub-request 2 processed by memory 2 and service processing sub-request 3 (second service processing sub-request) processed by memory 300.

[0157] The business type of the aforementioned business requests can be one of the following: data reconstruction, garbage collection, data compression, data filtering, screening, and deduplication. The business types corresponding to different business requests can be the same or different. The business processing sub-requests generated based on the business requests include the business type and business processing parameters of that business request.

[0158] The memory 200 can process the business data (first business data) related to the business processing sub-request 1 according to the business processing parameters of the business processing sub-request 1, and obtain the processing result of the business processing sub-request 1.

[0159] The memory 200 can send the service processing sub-request 3 to the memory 300 according to the service type of the service processing sub-request 3.

[0160] In response to the service processing sub-request 3, the memory 300 executes the service processing sub-request 3 according to the service processing parameters of the service processing sub-request 3, processes the service data related to the service processing sub-request 3, and feeds back the processing result of the service processing sub-request 3 to the memory 200.

[0161] It is understandable that the memory 200 can send the service processing sub-request 2 to the memory 2 in the same way in order to obtain the processing result of the service processing sub-request 2 fed back by the memory.

[0162] In some possible implementations, the memory 200 may also divide a service request into multiple service processing sub-requests according to the service type of the service request issued by the processor, so as to request multiple memories to jointly execute the service request.

[0163] Schematic, memory 200 will Figure 3The illustrated service request 3 is divided into service processing sub-request 4 (first service processing sub-request) processed by memory 200 (first memory) and service processing sub-request 5 (second service processing sub-request) processed by memory 300 (second memory). Each service processing sub-request includes the service type and service processing parameters of the service request. The source addresses of all service processing sub-requests point to the storage area of ​​the memory associated with the service request.

[0164] The memory 200 can generate a third service processing sub-request based on the service type of the service processing sub-request 4, in order to request the second service data to be obtained from the memory 300, and process the second service data according to the service processing parameters of the service processing sub-request 4 to obtain the processing result of the service processing sub-request 4.

[0165] The memory 200 can send the service processing sub-request 5 to the memory 300 according to the service type of the service processing sub-request 5.

[0166] In response to service processing sub-request 5, memory 300 executes service processing sub-request 5 according to the service processing parameters of service processing sub-request 5, processes the service data related to service processing sub-request 5, and feeds back the processing result of service processing sub-request 5 to memory 200.

[0167] In some possible implementations, the service request includes first service data and second service data, wherein the destination address and source address of the first service data point to the storage area of ​​the first memory, and the second service data points to the storage area of ​​the second memory.

[0168] Schematic illustration: Memory 200 can generate a third service processing sub-request based on the service type in the service request, to request the retrieval of second service data from memory 300. The third service processing sub-request includes the source address related to the second service data.

[0169] The memory 300 obtains the second service data from the source address in the third service processing sub-request and feeds the second service data back to the memory 200.

[0170] The first memory generates multiple business sub-requests based on the business request, and sends these sub-requests to multiple memories via the interconnection network module. The multiple memories then jointly execute a business request, achieving distributed data processing, freeing up CPU performance, and improving the performance, efficiency, and reliability of data processing in the distributed system.

[0171] S730: Execute the service request based on the processing result fed back by the second memory in response to the service processing sub-request.

[0172] As described in step S720, the memory 200 can receive the processing result fed back by the second memory in response to the second service processing sub-request.

[0173] As one possible implementation, in the case where memory 200 processes service processing sub-request 1, memory 2 processes service processing sub-request 2, and memory 300 processes service processing sub-request 3.

[0174] Memory 200 can obtain the processing result of service processing sub-request 1, as well as the processing results fed back by memory 2 and memory 300 in response to the service processing sub-request.

[0175] As one possible implementation, in the case where memory 200 processes service processing sub-request 4 and memory 300 processes service processing sub-request 5.

[0176] The memory 200 received the processing result of the service processing sub-request 4, and the memory 300 received the processing result in response to the service processing sub-request 5.

[0177] As one possible implementation, in the case where the destination address and source address of the first service data of the service request point to the storage area of ​​the first memory, and the second service data points to the storage area of ​​the second memory.

[0178] The memory 200 obtains the first service data from the storage area of ​​the first memory pointed to by the source address according to the service processing parameters of the service request.

[0179] Memory 200 receives the second service data fed back by memory 300 in response to the third service processing sub-request.

[0180] The memory 200 can process the first service data and the second service data according to the service processing parameters in the service request to obtain the processing result of the service request.

[0181] The memory 200 can generate a processing result for the corresponding service request based on the processing result of the service processing sub-request, and send the processing result of the task request to the processor 110. Alternatively, the memory 200 can send the processing result of each service processing sub-request to the processor 110 via the peer-to-peer interconnection module 210 after completion. This application does not limit this aspect.

[0182] It is worth noting that when the memory 200 needs to process multiple service processing sub-requests, the controller 220 can also determine the corresponding service priorities of the multiple service processing sub-requests according to the service type of the multiple service processing sub-requests, and process the multiple service requests in sequence based on the service priorities of the multiple service processing sub-requests.

[0183] This application provides a data processing method. This method offloads cross-node data processing tasks from the CPU to a first memory for execution. The first memory sends a sub-request for processing a service from the service request to a second memory based on the service type in the service request. The second memory then processes the sub-request according to data processing parameters and receives feedback from the second memory. This method enables cross-node data processing in a distributed system, avoids the processing capacity of a single memory becoming a performance bottleneck, reduces CPU performance overhead, and improves the performance and efficiency of data processing in the distributed system.

[0184] Secondly, this application discloses a memory, illustratively, Figure 8 This is a schematic diagram of the structure of a memory provided in an embodiment of this application, such as... Figure 8 As shown, the memory 80 includes a communication interface 81 and a controller 82.

[0185] Communication interface 81 serves as the data communication interface for memory 80. In this application, communication interface 81 may include, but is not limited to, wireless communication data communication interfaces implemented using Bluetooth, WLAN (Wireless Local Area Network), 4G, and 5G protocols. It may also include wired data communication interfaces implemented using high-speed serial computer expansion bus standards (PCIe, peripheral component interconnect express), M.2 NVME / SATA protocols, SATA protocols, and Universal Serial Bus (USB). This application does not limit this specific implementation.

[0186] The controller 82 receives service requests from any computing node via communication interface 81. These service requests include a service type and service processing parameters. Specifically, the service request instructs the second memory to process the requested service data according to the service processing parameters; sends a service processing sub-request to the second memory according to the service type; and obtains the processing result from the second memory in response to the service processing sub-request, and executes the aforementioned service request.

[0187] In some possible implementations, the memory 80 also includes a peer-to-peer interconnection module 83, and the controller 82 is further configured to send service processing sub-requests to the second memory via the peer-to-peer interconnection module 83.

[0188] In one possible implementation, the controller 82 is configured to determine, based on the service type, that the first service data is located in the memory 80 and the second service data is located in the second memory. The controller 82 is also configured to generate a third service processing sub-request and send the third service processing sub-request to the second memory to request the retrieval of the aforementioned second service data from the second memory.

[0189] In some possible implementations, the controller 82 is also used to obtain second service data fed back from the second memory, obtain first service data from the storage area of ​​the memory 80, process the first service data and the second service data according to service processing parameters, and obtain processed service data.

[0190] In actual implementation, memory 80 may also include storage medium 84.

[0191] Storage medium 84 is used to store the first business data and / or the processed first business data.

[0192] Thirdly, this application discloses a distributed system, which includes... Figure 2 Multiple memories, including the first memory 200 and the second memory 300, are shown.

[0193] The first memory 200 receives a service request from the processor. This service request includes a service type and service processing parameters. The service request instructs the first memory 200 to process the requested service data according to the service processing parameters. Based on the service type, a service sub-request is sent to the second memory 300. The processing result from the second memory 300 in response to the service sub-request is obtained, and the service request is executed.

[0194] In some possible implementations, the first memory 200 may also include a peer-to-peer internet module 210, which is used to send service processing sub-requests to the second memory 300 via the peer-to-peer internet module 210.

[0195] In one possible implementation, the first memory 200 is used to determine, based on the service type, that first service data is located in the first memory 200 and second service data is located in the second memory 300. The first memory 200 is also used to generate a third service processing sub-request and send the third service processing sub-request to the second memory 300 to request the retrieval of the aforementioned second service data from the second memory 300.

[0196] In some possible implementations, the first memory 200 is used to obtain the second service data fed back by the second memory 300, obtain the first service data from the storage area of ​​the first memory 200, and process the first service data and the second service data according to the service processing parameters to obtain the processed service data.

[0197] In addition to the methods, apparatus, and electronic devices described above, embodiments of this application may also provide a computer program product, comprising computer program instructions. When executed by a processor, the computer program instructions cause the processor to perform the steps of the data processing methods described in the various embodiments of this application in the "Method" section of this specification. The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The computer program code can be in source code form, object code form, executable file, or some intermediate form. The computer program code can be executed entirely on a user's computing device, partially on a user's device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0198] Furthermore, embodiments of this application may also provide a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the data processing method according to various embodiments of this disclosure as described in the "Method" section above. The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, a computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0199] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0200] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0201] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of the various embodiments of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the specific details described above.

[0202] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0203] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0204] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

[0205] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The specific embodiments described above have further detailed the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0206] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, Applied to a first memory, the method includes: The system receives a service request from the processor; the service request includes a service type and service processing parameters; wherein the service request is used to request the first memory to process the requested service data according to the service processing parameters. According to the service type, send a service processing sub-request to the second memory; The service request is executed based on the processing result fed back by the second memory in response to the service processing sub-request.

2. The method according to claim 1, characterized in that, The first storage device includes a peer-to-peer internet module; according to the service type, a service processing sub-request is sent to the second storage device, including: According to the service type, the service processing sub-request is sent to the second memory through the peer-to-peer Internet module.

3. The method according to claim 1 or 2, characterized in that, The step of sending a service processing sub-request to the second memory according to the service type includes: Based on the business type, the business request is divided into a first business processing sub-request and a second business processing sub-request. The second service processing sub-request is sent to the second memory; the second service processing sub-request carries the service processing parameters, so that the second memory executes the second service processing sub-request according to the service processing parameters, wherein the first service processing sub-request is a request processed by the first memory.

4. The method according to claim 1 or 2, characterized in that, According to the service type, a service processing sub-request is sent to the second memory, including: Based on the business type, it is determined that the first business data is located in the first memory and the second business data is located in the second memory. A third service processing sub-request is generated and sent to the second memory to request the acquisition of the second service data from the second memory.

5. The method according to claim 4, characterized in that, Based on the processing result fed back by the second memory in response to the service processing sub-request, the service request is executed, including: Obtain the second service data fed back by the second memory; The first service data is obtained from the storage area of ​​the first memory; The first business data and the second business data are processed according to the business processing parameters to obtain the processed business data.

6. The method according to claim 5, characterized in that, The service request also includes a destination address, and the method further includes: The processed business data are stored in the memory corresponding to the destination address.

7. The method according to claim 6, characterized in that, After storing the processed business data into the memory corresponding to the destination address, the method further includes: According to the business processing parameters, delete the first business data and the second business data.

8. The method according to claim 4, characterized in that, The business type includes data reconstruction, and the business processing parameters include the erasure coding and EC check matrix algorithm configuration used when performing EC check during data reconstruction operations; or The business type includes garbage collection, and the business processing parameters include the execution order of data marking and data deletion when performing garbage collection operations; or The service type includes data compression, and the service processing parameters include the configuration of the compression algorithm used to perform the data compression operation; or The business types include data filtering and screening, and the business processing parameters include the screening conditions issued by the processor.

9. A memory, characterized in that, The memory includes: Communication interface; The controller receives a service request from any processor through the communication interface; the service request includes a service type and service processing parameters; wherein, the service request is used to request the memory to process the requested service data according to the service processing parameters; according to the service type, it sends a service processing sub-request to the second memory; and obtains the processing result fed back by the second memory in response to the service processing sub-request, and executes the service request.

10. The memory according to claim 9, characterized in that, The memory further includes a peer-to-peer interconnection network module; the controller sends a service processing sub-request to the second memory according to the service type, specifically for: The service processing sub-request is sent to the second memory via the peer-to-peer internet module.

11. The memory according to claim 9 or 10, characterized in that, The controller sends a service processing sub-request to the second memory according to the service type, specifically for: Based on the business type, it is determined that the first business data is located in the first memory and the second business data is located in the second memory. A third service processing sub-request is generated and sent to the second memory to request the acquisition of the second service data from the second memory.

12. The memory according to claim 11, characterized in that, The controller executes the service request based on the processing result fed back by the second memory in response to the service processing sub-request, specifically for: Obtain the second service data fed back by the second memory; The first service data is obtained from the storage area of ​​the first memory; The first business data and the second business data are processed according to the business processing parameters to obtain the processed business data.

13. A distributed system, characterized in that, The distributed system includes multiple memories, including a first memory and a second memory. The first memory receives a service request from the processor, the service request including a service type and service processing parameters; wherein, the service request is used to request the first memory to process the requested service data according to the service processing parameters; according to the service type, a service processing sub-request is sent to the second memory; the processing result fed back by the second memory in response to the service processing sub-request is obtained, and the service request is executed.

14. The distributed system according to claim 13, characterized in that, The first storage device includes a peer-to-peer internet module. Based on the service type, the first storage device sends a service processing sub-request to the second storage device, specifically for: The service processing sub-request is sent to the second memory via the peer-to-peer internet module.

15. The distributed system according to claim 13 or 14, characterized in that, The first memory sends a service sub-request to the second memory according to the service type, specifically for: Based on the business type, it is determined that the first business data is located in the first memory and the second business data is located in the second memory. A third service processing sub-request is generated and sent to the second memory to request the acquisition of the second service data from the second memory.

16. The distributed system according to claim 15, characterized in that, The first memory executes the service request based on the processing result fed back by the second memory in response to the service processing sub-request, specifically for: Obtain the second service data fed back by the second memory; The first service data is obtained from the storage area of ​​the first memory; The first business data and the second business data are processed according to the business processing parameters to obtain the processed business data.

17. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a memory, enable the memory to perform the data processing method as described in any one of claims 1-8.

18. A computer program product containing instructions, characterized in that, When the instruction is executed by the memory, the memory performs the data processing method as described in any one of claims 1-8.

Images