A task processing method, a task scheduling method, a device, a medium and a product

Abstract

The present invention discloses a task processing method, a task scheduling method, a device, a medium and a product, and relates to the field of storage technology. This solution divides a plurality of storage areas with independent computing capabilities inside a solid-state hard disk, and is specifically based on an improved solid-state hard disk controller, and configures a dedicated micro-computing unit, a data transmission control module and a timing control module for each storage area, which can process the data computing operations of the corresponding storage area, greatly reducing the transmission of data between the storage device and the external computing unit, and avoiding the I / O bottleneck; at the same time, the solid-state hard disk is connected to multiple near-storage computing nodes through a first preset protocol adapter module, so as to flexibly configure and expand computing and storage hardware according to actual needs, effectively reducing hardware costs and enhancing system compatibility.

Description

Technical Field

[0001] The present invention relates to the field of storage technologies, and in particular, to a task processing method, a task scheduling method, a device, a medium, and a product. Background Art

[0002] Near-Storage Computing (NSC) technology aims to reduce data transmission overhead and improve system efficiency by migrating some computing tasks to be executed near storage devices.

[0003] However, in the current cooperation mode between near-storage computing and storage resources, the computing unit and the storage unit of the traditional separated architecture are independent, and data depends on the operating system Input / Output (I / O) scheduling for transmission, resulting in I / O bottlenecks and resource waste in large-scale environments. At the same time, early near-storage computing architectures tried to integrate simple computing functions within storage devices. However, limited by computing power, a large amount of data still needs to be transmitted to external computing units, and the communication protocol and interface design are sub-optimized. In addition, although modular hardware design can achieve a certain degree of expansion, affected by physical limitations and compatibility issues, the expansion flexibility and scale are limited.

[0004] In view of the above, how to solve the I / O bottlenecks, limited computing power, sub-optimized communication protocols, limited expansion and poor compatibility of modular design faced by current near-storage computing technologies is an urgent problem for those skilled in the art. Summary of the Invention

[0005] The present invention provides a task processing method, a task scheduling method, a device, a medium, and a product to at least solve the problems of I / O bottlenecks, limited computing power, sub-optimized communication protocols, limited expansion and poor compatibility of modular design faced by current near-storage computing technologies.

[0006] The present invention provides a task processing method, which is applied to a solid-state drive controller of a solid-state drive; the solid-state drive controller at least includes a command scheduling module, a micro-computing unit corresponding to each storage area of the solid-state drive, a data transmission control module, and a timing control module; the solid-state drive controller is communicatively connected to a plurality of near-storage computing nodes through a first preset protocol transfer module; the method includes:

[0007] When receiving a command of a storage operation task transmitted by a host through the first preset protocol transfer module, determining, through the command scheduling module, a target storage area corresponding to the command, and allocating the command to the data transmission control module corresponding to the target storage area; wherein, the host includes a host node and each near-storage computing node;

[0008] The command is transmitted to the corresponding timing control module through the data transmission control module, and the data transmission control module and the timing control module process the storage operation task according to the command.

[0009] The present invention also provides a task scheduling method, which is applied to a host node; the host node is communicatively connected to a plurality of distributed computing and storage cooperation units through a second preset protocol transfer module; the distributed computing and storage cooperation units include a solid-state drive and a plurality of near-storage computing nodes; the solid-state drive controller of the solid-state drive at least includes a command scheduling module, a micro-computing unit corresponding to each storage area of the solid-state drive, a data transmission control module and a timing control module; the solid-state drive controller is communicatively connected to the corresponding plurality of near-storage computing nodes through a first preset protocol transfer module; the method includes:

[0010] When a computing task is received, perform task feature analysis on the computing task to determine the computing task type;

[0011] Determine the adapted task types of each distributed computing and storage cooperation unit;

[0012] According to the computing task type and each adapted task type, determine the target distributed computing and storage cooperation unit among each distributed computing and storage cooperation unit;

[0013] Send the computing task to the target distributed computing and storage cooperation unit through the second preset protocol transfer module to execute the computing task.

[0014] The present invention also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of the above task processing method or the steps of the above task scheduling method when executing the computer program.

[0015] The present invention also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of the above task processing method or the steps of the above task scheduling method are implemented.

[0016] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above task processing method or the steps of the above task scheduling method are implemented.

[0017] The beneficial effect of the present invention is that a plurality of storage areas with independent computing capabilities are divided inside the solid-state hard disk. Specifically, based on the improved solid-state hard disk controller, a dedicated micro-computing unit, a data transmission control module and a timing control module are configured for each storage area, which can process the data computing operations of the corresponding storage area, greatly reducing the transmission of data between the storage device and the external computing unit, and avoiding the occurrence of I / O bottlenecks; at the same time, the solid-state hard disk is communicatively connected with a plurality of near-storage computing nodes through a first preset protocol adapter module, so as to flexibly configure and expand computing and storage hardware according to actual needs, effectively reducing hardware costs and enhancing system compatibility.

[0018] At the same time, the present invention also determines the adaptation task type of each computing unit by analyzing the characteristics of the computing tasks and quantitatively evaluating the relevant capabilities of each computing unit, so as to accurately send the computing tasks to the corresponding computing units, thereby realizing the reasonable scheduling and efficient allocation of computing tasks, reducing the occupancy of near storage computing nodes, reducing the amount of data transmission within the system, and thus improving the overall computing efficiency.

[0019] In addition, the present invention also provides an electronic device, a computer-readable storage medium, and a computer program product, with the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A logical architecture diagram of a distributed computing and storage collaboration unit provided in an embodiment of the present invention;

[0022] Figure 2 A flowchart of a task processing method provided by the present invention;

[0023] Figure 3 An architectural diagram of a pluggable hardware computing cluster provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of a task scheduling method provided by an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of a task processing device provided by an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of a task scheduling device provided by an embodiment of the present invention;

[0027] Figure 7 Schematic diagram of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0028] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0029] It should be noted that in the description of the present invention, the terms "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present invention are used to distinguish similar objects and not to describe a specific order or sequence.

[0030] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0031] The core concept of near-memory computing technology is to migrate part of the computing tasks to be executed near the storage device to reduce data transmission overhead and improve system efficiency. This technology shows significant advantages at the theoretical level, especially in edge computing scenarios. By performing preliminary data processing near the data acquisition end, it can effectively relieve the network bandwidth pressure and speed up the response speed. However, the current near-memory computing technology faces various challenges in practical applications.

[0032] First, in the traditional separated architecture, the computing unit and the storage unit are relatively independent and connected through a high-speed bus or network. This architecture is prone to serious I / O bottlenecks in a large-scale system environment because frequent data transmission will occupy a large amount of network bandwidth and system resources, making the computing unit wait for data for too long, thus reducing the data processing efficiency. In addition, the I / O scheduling mechanism of the operating system may not be able to adapt to the requirements of different application scenarios, further affecting the data transmission efficiency. For example, in real-time data analysis, data transmission delay may lead to the inability to provide support for decision-making in a timely manner.

[0033] Secondly, early near-storage computing architectures attempted to add simple computing functions inside storage devices. Although they could perform preliminary data filtering and aggregation operations, a large amount of data still needed to be transferred to external computing units for complex computing tasks. This did not fundamentally solve the problem of poor coordination between computing and storage resources. Moreover, the communication protocols and interface designs between storage devices and external computing units were not optimized enough, resulting in low data transfer efficiency. Especially when dealing with complex tasks such as large-scale image data, it was difficult to meet the requirements of real-time and accuracy.

[0034] Although modular hardware design provides certain expansion capabilities, it is limited by physical constraints such as hardware interface standards and the number of motherboard slots, with limited flexibility and scale of expansion. As the number of modules increases, hardware compatibility issues may gradually emerge, affecting system stability. For example, in a small server, due to slot number limitations, it may not be able to meet the rapidly growing data storage needs. In addition, as the cluster scale expands, the difficulties in aspects such as communication overhead between nodes, data consistency maintenance, and task scheduling increase exponentially, further restricting system performance and scalability.

[0035] In summary, current near-storage computing technologies face many problems such as I / O bottlenecks, coordination between computing and storage resources, communication protocol optimization, and hardware scalability. To solve the above problems, the present invention provides a task processing method applied to an improved Solid State Drive Controller (SSD Controller).

[0036] Figure 1 It is the logical architecture diagram of the distributed computing and storage cooperation unit provided by the embodiment of the present invention. As Figure 1 shown, the distributed computing and storage cooperation unit mainly consists of a Solid State Drive (SSD), a first preset protocol conversion module, and multiple near-storage computing nodes. The SSD and each near-storage computing node are respectively communicatively connected to the first preset protocol conversion module through corresponding first preset protocol interfaces, realizing a pluggable storage architecture. Therefore, the scale of the distributed computing and storage cooperation unit can be adjusted according to business changes. It should be noted that in this solution, there is no limitation on the communication protocol used by the first preset protocol conversion module. For example, it can be Compute ExpressLink (CXL), Non-Volatile Memory Express (NVMe) protocol, or Peripheral Component Interconnect Express (PCIe) protocol, depending on the specific implementation situation.

[0037] Furthermore, inside the solid-state drive, multiple storage areas with independent computing capabilities are divided based on the improved SSD controller. Each storage area is configured with a micro-computation unit, a data transfer control module, and a timing control module, which can directly read and write data in the flash memory, reduce the data transfer between the storage device and the external near-storage computing nodes, and perform computing operations on the data stored locally, such as data filtering, data aggregation, etc. The task processing method will be described in detail below:

[0038] Figure 2 It is a flowchart of a task processing method provided by the present invention. As Figure 2 shown, the method includes:

[0039] S10: When receiving a command of a storage operation task transmitted by the host through the first preset protocol transfer module, determine the target storage area corresponding to the command through the command scheduling module, and allocate the command to the data transfer control module corresponding to the target storage area.

[0040] Among them, the host includes a host node and each near-storage computing node.

[0041] Specifically, the SSD controller monitors the storage operation tasks transmitted by the host. The host includes a host node and each near-storage computing node communicatively connected to the SSD. It should be noted that the host node is the parent node of the distributed computing and storage cooperation unit where the solid-state drive is located.

[0042] When the host has a storage operation task, corresponding commands will be generated, such as write commands, read commands, and erase commands, etc., and the commands will be transmitted to the SSD controller through the first preset protocol transfer module, and specifically received by the command scheduling module in the SSD controller. The command scheduling module will further determine the target storage area corresponding to the command according to the command content and the division of the internal storage areas of the SSD, and allocate the command to the data transfer control module corresponding to the target storage area.

[0043] S11: Transmit the command to the corresponding timing control module through the data transfer control module, and the data transfer control module and the timing control module process the storage operation task according to the command.

[0044] It is worth noting that there are corresponding independent data transfer control modules in different storage areas of the SSD, which can be responsible for the efficient data transfer between the host and each independent storage area of the SSD according to the commands allocated by the command scheduling module. Control the rate, direction, and order of data transfer to ensure that the data accurately reaches the target storage area. At the same time, it is responsible for handling errors and exceptions during the data transfer process.

[0045] At the same time, after receiving the command, the data transmission control module will transmit the command to the corresponding timing control module. The timing control module is responsible for generating accurate timing signals to control the read, write, and erase operations of the SSD's Not AND Flash Memory (NAND). Ensure that the operation sequence is correct and the time interval complies with the NAND technical specifications. Therefore, the processing of storage operation tasks is actually implemented by the data transmission control module and the timing control module.

[0046] It should be noted that in this embodiment, there is no restriction on the specific process of transmitting the command to the corresponding timing control module through the data transmission control module, and the data transmission control module and the timing control module processing the storage operation task according to the command, which depends on the specific implementation situation.

[0047] In this embodiment, a plurality of storage areas with independent computing capabilities are divided inside the solid-state hard disk. Specifically based on the improved solid-state hard disk controller, a dedicated micro-computing unit, a data transmission control module and a timing control module are configured for each storage area, which can process the data computing operations of the corresponding storage area, greatly reducing the transmission of data between the storage device and the external computing unit, and avoiding I / O bottlenecks; at the same time, the solid-state hard disk is communicatively connected to a plurality of near-storage computing nodes through a first preset protocol adapter module, so as to flexibly configure and expand computing and storage hardware according to actual needs, effectively reducing hardware costs and enhancing system compatibility.

[0048] Based on the above embodiments, in some embodiments, determining the target storage area corresponding to the command through the command scheduling module and allocating the command to the data transmission control module corresponding to the target storage area includes:

[0049] S101: Performing syntax analysis and semantic analysis on a command through a command scheduling module to determine command information of the command.

[0050] Among them, the command information includes at least command type, sub-operation type, target storage area identifier, address information, partition identifier, data length, data content, data verification information, access control information, encryption flag, authentication information, timing control information, number of retries, response requirements and error handling methods.

[0051] S102: Transmitting the command and command information to the corresponding data transmission control module through the command scheduling module.

[0052] In order to accurately transmit the command to the data transmission control module corresponding to the target storage area and obtain relevant information of the storage operation task, in the specific implementation, the command scheduling module performs syntax analysis and semantic analysis on the command to determine the command information of the command. At least the following command information is included:

[0053] Command types include read operations (Read), write operations (Write), erase operations (Erase), verify operations (Verify), and other specific operations (such as Reset, Lock, etc.); sub-operation types include block erase (Block Erase) and chip erase (Chip Erase), etc.; target storage area identifiers, specifically including storage medium types; address information, including start address (Start Address), end address (End Address), or block / sector number (Block / Sector Number); partition identifiers, including boot partition, application partition, etc.; data length, including operation data length (in units such as bytes / words / pages, etc.), for the payload data length of write operations, and for read operations, the requested data length to be read; data content, including the data to be written (Payload Data) during write operations; data verification information; access control information, including security level / permission requirements; encryption flag, such as whether it is encrypted transmission; authentication information; timing control information, including timeout setting (TimeoutValue); retry count (Retry Count); response requirements, including whether an acknowledgement response (ACK / NACK) is required and whether data is required to be returned (such as in read operations); error handling, including error detection methods (such as parity check, ECC, etc.) and error handling strategies (abort / retry / ignore).

[0054] Finally, the command and command information are transmitted to the corresponding data transfer control module through the command scheduling module, so that the data transfer control module can accurately execute the command.

[0055] Next, in combination with different command types, the collaborative working process of each module of the SSD controller will be described in detail:

[0056] (1) When the command type is a write operation;

[0057] Based on the above embodiments, in some embodiments, the command is transmitted to the corresponding timing control module through the data transfer control module, and the data transfer control module and the timing control module process the storage operation tasks according to the command, including:

[0058] S103: Cache the target data and verification data corresponding to the command through the data transfer control module.

[0059] S104: The data transfer control module performs verification and error correction on the target data according to the verification data.

[0060] S105: Determine the dynamic transfer rate of the target data through the data transfer control module.

[0061] S106: Transmit the command and target data to the timing control module through the data transmission control module.

[0062] S107: The timing control module generates timing signals according to the command.

[0063] S108: The timing control module transmits the target data to the target storage area according to the timing signals, writes the target data into the target storage area according to the timing signals and the dynamic transmission rate, and feeds back the write status information to the data transmission control module in real time until the write operation is completed.

[0064] Specifically, when the type of the received command is a write operation, the command scheduling module performs syntax and semantic analysis on the command, extracts key command information such as the target storage area identifier and data length, and sends it to the data transmission control module of the corresponding storage area.

[0065] After receiving the write command, the data transmission control module receives the target data to be written and the corresponding check data from the host, and stores the target data and the check data in the cache. During this period, the data transmission control module checks and corrects the target data according to the check data. It should be noted that in this embodiment, the specific process of storing the target data and the check data in the cache is not limited, and the process of checking and correcting the target data is also not limited, which depends on the specific implementation situation. At the same time, the data transmission control module determines the dynamic transmission rate of the target data to be written this time. The determination method of the dynamic transmission rate of the target data in this embodiment is also not limited.

[0066] Furthermore, the timing control module receives the write data and related command information from the data transmission control module, and generates accurate write timing signals, including chip select signals, write enable signals, address latch signals, etc. It should be noted that in this embodiment, the generation process of the timing signals is not limited, which depends on the specific implementation situation.

[0067] Finally, the timing control module outputs the timing signals to the NAND flash memory of the corresponding target storage area, controls it to write the target data into the corresponding target storage area according to the dynamic transmission rate. At the same time, during the data writing period, the timing control module feeds back the write status information to the data transmission control module in real time until the write operation is completed. The specific content of the write status information in this embodiment is not limited, including but not limited to information related to the status register value, error detection and correction information, and timing parameter monitoring information, which depends on the specific implementation situation.

[0068] (2) When the command type is a read operation;

[0069] Based on the above embodiments, in some embodiments, the command is transmitted to the corresponding timing control module through the data transmission control module, and the data transmission control module and the timing control module process the storage operation task according to the command, including:

[0070] S109: Transmit the command to the timing control module through the data transmission control module, and determine the target data corresponding to the command.

[0071] S110: Determine the dynamic transmission rate of the target data through the data transmission control module.

[0072] S111: Generate a timing signal by the timing control module according to the command.

[0073] S112: Transmit the timing signal and the command to the target storage area by the timing control module, so as to read the target data and the corresponding check data in the target storage area according to the timing signal and the dynamic transmission rate, and feedback the read status information to the data transmission control module in real time until the read operation is completed.

[0074] S113: Transmit the target data and the corresponding check data to the data transmission control module through the timing control module.

[0075] S114: Cache the target data and the check data corresponding to the command through the data transmission control module.

[0076] S115: Check and correct the target data by the data transmission control module according to the check data.

[0077] S116: Transmit the target data to the host through the data transmission control module and the command scheduling module.

[0078] Specifically, when the type of the received command is a read operation, the command scheduling module performs syntax and semantic analysis on the command, extracts key command information such as the target storage area identifier and the data length, and sends it to the data transmission control module of the corresponding storage area.

[0079] After receiving the command, the data transmission control module transmits the command to the timing control module, and determines the target data corresponding to the command and the dynamic transmission rate of the target data. It should be noted that in this embodiment, the method for determining the dynamic transmission rate of the target data is not limited and depends on the specific implementation situation.

[0080] After receiving a read command from the data transfer control module, the timing control module generates timing signals according to the command, including chip select signals, read enable signals, address transfer signals, etc. Further, the timing signals and the command are transmitted to the NAND flash memory in the target storage area, controlling it to read the target data and the corresponding check data in the target storage area according to the timing signals and the dynamic transfer rate, and real-time feedback the read status information to the data transfer control module until the read operation is completed. Finally, the timing control module transmits the target data and the corresponding check data to the data transfer control module. It should be noted that in this embodiment, the specific process of the timing control module generating timing signals according to the command is not limited, and the specific content of the read status information is also not limited, which depends on the specific implementation situation.

[0081] The data transfer control module receives the read target data and check data from the timing control module and caches the target data and check data. In this embodiment, the specific caching method of the check data for the target data is not limited. Subsequently, the data transfer control module performs verification and error correction on the target data according to the check data. In this embodiment, the verification and error correction process of the target data is not limited. The data after verification and error correction is transmitted to the host through the data transfer control module and the command scheduling module.

[0082] (3) When the command type is an erase operation;

[0083] Based on the above embodiments, in some embodiments, the command is transmitted to the corresponding timing control module through the data transfer control module, and the data transfer control module and the timing control module process the storage operation tasks according to the command, including:

[0084] S117: Transmit the command to the timing control module through the data transfer control module.

[0085] S118: The timing control module generates timing signals according to the command.

[0086] S119: Transmit the timing signals and the command to the target storage area through the timing control module to erase the data in the target storage area according to the timing signals, and real-time feedback the erase status information to the data transfer control module until the erase operation is completed.

[0087] Specifically, when the received command type is an erase operation, the command scheduling module parses the command and sends it to the data transfer control module of the corresponding target storage area.

[0088] After receiving the erasure command, the data transmission control module forwards it to the timing control module. Meanwhile, the data transmission operation of this storage area is paused to ensure that the erasure process is not interfered with. After receiving the erasure command, the timing control module generates timing signals, including chip select signals, erasure enable signals, etc. In this embodiment, there is no limitation on the process of generating timing signals, which depends on the specific implementation. Finally, the timing control module transmits the timing signals and the command to the NAND flash memory of the target storage area to erase the data in the target storage area according to the timing signals. During the erasure process, the erasure status information is fed back to the data transmission control module in real time until the erasure operation is completed. In this embodiment, there is no limitation on the specific content of the erasure status information.

[0089] In summary, the command scheduling module is responsible for command parsing and command scheduling, the data transmission control module is responsible for data caching and controlling the data transmission rate, and the timing control module generates accurate timing signals. Each module collaborates to process the storage operation tasks, ensuring the accurate and efficient execution of the storage operation tasks.

[0090] Based on the above embodiments, in some embodiments, the data transmission control module caches the target data and parity data corresponding to the command, including:

[0091] S120: When the command type is a write operation, write the corresponding target data and parity data into the first buffer.

[0092] S121: When the command type is a read operation, write the corresponding target data and parity data into the second buffer.

[0093] To better store the target data and parity data, an intelligent cache pool is set for the data transmission control module in this embodiment. The intelligent cache pool is designed based on a dual-bank structure, supports read-write separation and dynamic capacity allocation, and integrates a data compression engine. Therefore, the data is transmitted in a segmented manner. When the command type is a write operation, the corresponding target data and parity data are written into the first buffer. When the command type is a read operation, the corresponding target data and parity data are written into the second buffer.

[0094] In this embodiment, the target data and parity data are written into the first buffer and the second buffer respectively according to the command type, realizing read-write separation, ensuring that read and write data do not affect each other, and improving the stability of the data transmission control module.

[0095] Based on the above embodiments, in some embodiments, the data transmission control module determines the dynamic transmission rate of the target data, including:

[0096] S122: Obtain the temperature alarm signal, error correction code count, and system load factor of the solid-state drive.

[0097] S123: When the temperature alarm signal is valid, determine that the dynamic transmission rate is the first transmission rate.

[0098] S124: When the error correction code count is greater than the first threshold and / or the system load factor is greater than the second threshold, determine that the dynamic transmission rate is the second transmission rate.

[0099] S125: When the system load factor is greater than the third threshold and not greater than the second threshold, determine that the dynamic transmission rate is the third transmission rate.

[0100] S126: When the temperature alarm signal is invalid, the error correction code count is not greater than the first threshold, and the system load factor is not greater than the third threshold, determine that the dynamic transmission rate is the fourth transmission rate.

[0101] Wherein, the second threshold is greater than the third threshold; the first transmission rate is less than the second transmission rate, the second transmission rate is less than the third transmission rate, and the third transmission rate is less than the fourth transmission rate.

[0102] Specifically, the data transmission control module can implement dynamic transmission rate adjustment, and specifically obtain three input signals: the temperature alarm signal (temp_alert), the error correction code count (ecc_count), and the system load factor (load_factor) of the solid-state drive.

[0103] It should be noted that there are four gears for the data transmission rate in this embodiment: the first transmission rate R0, the second transmission rate R1, the third transmission rate R2, and the fourth transmission rate R3. Among them, the first transmission rate R0 is less than the second transmission rate R1, the second transmission rate R1 is less than the third transmission rate R2, and the third transmission rate R2 is less than the fourth transmission rate R3. In this embodiment, the specific transmission rates for each gear are not limited. For example, based on the highest transmission speed X, the first transmission rate R0 is 1 / 4X, the second transmission rate R1 is 1 / 2X, the third transmission rate R2 is 3 / 4X, and the fourth transmission rate R3 is X.

[0104] When the temperature alarm signal is valid, determine that the dynamic transmission rate is the first transmission rate to reduce the data transmission rate and ensure the stable operation of the system at high temperatures. When the error correction code count is greater than the first threshold, and / or the system load factor is greater than the second threshold, due to a large number of data transmission errors or a high system load, determine that the dynamic transmission rate is the second transmission rate, and reduce the transmission rate to ensure the accuracy of data transmission or avoid system overload. When the system load factor is greater than the third threshold and not greater than the second threshold, determine that the dynamic transmission rate is the third transmission rate, thereby making corresponding adjustments to the transmission rate. When the temperature alarm signal is invalid, the error correction code count is not greater than the first threshold, and the system load factor is not greater than the third threshold, determine that the dynamic transmission rate is the fourth transmission rate, that is, use the highest transmission rate for data transmission to make full use of system resources and improve transmission efficiency.

[0105] It should be noted that the second threshold is greater than the third threshold. In this embodiment, there are no restrictions on the magnitudes of the first threshold, the second threshold, and the third threshold, which are determined according to specific implementation circumstances.

[0106] In this embodiment, the data transmission control module realizes the adjustment of the dynamic transmission rate for data reading / writing, ensuring the efficient transmission of data and at the same time ensuring system load balancing.

[0107] Based on the above embodiments, in some embodiments, the data transmission control module checks and corrects the target data according to the verification data, including:

[0108] S127: Check the target data according to the cyclic redundancy check code, and determine whether the target data is complete; if not, proceed to step S128; if so, proceed to step S129.

[0109] S128: Execute the retransmission of the target data.

[0110] S129: Correct the target data according to the error correction code.

[0111] In specific implementation, the verification data of the target data includes a cyclic redundancy check code (CRC) and an error correction code (ECC). The cyclic redundancy check code and the error correction code are two important technologies for ensuring data integrity during data transmission and storage. CRC is mainly used for error detection. By adding a check code calculated based on a polynomial to the data, the receiving end can recalculate the check code and compare it with the check code sent by the sending end to detect whether there are errors. ECC can not only detect errors but also has the ability to correct errors. It adds additional error correction bits to the data, enabling the receiving end to identify and correct errors within a certain range, thereby improving the reliability and integrity of the data.

[0112] Therefore, the data transmission control module checks the target data according to the cyclic redundancy check code to determine whether the target data is complete. If it is confirmed that the target data is incomplete, retransmission of the target data is performed, including retransmission of the target data from the host to the SSD during a write operation and retransmission of the target data from the storage area to the timing control module during a read operation. If it is confirmed that the target data is complete, error correction is performed on the target data according to the error correction code, thereby obtaining the target data after verification and error correction, ensuring the correctness of the target data.

[0113] Based on the above embodiments, in some embodiments, the timing control module generates timing signals according to commands, including:

[0114] S130: Generate a clock signal according to a multi-phase clock tree.

[0115] Among them, the number of phases of the multi-phase clock tree is the same as the number of storage areas of the solid-state drive.

[0116] S131: Generate an initial timing signal according to the command and the clock signal.

[0117] S132: Sample the data bus signal to determine whether the data bus signal is stable; if so, proceed to step S133; if not, proceed to step S134.

[0118] S133: Determine the initial timing signal as the timing signal.

[0119] S134: Add a delay to the initial timing signal to obtain the timing signal.

[0120] Among them, the timing signal at least includes a chip select signal, a read / write enable signal, and an address latch signal.

[0121] In a specific implementation, the generation of the timing signals of the timing control module is based on a multi-phase clock tree. It should be noted that the number of phases of the multi-phase clock tree is the same as the number of storage areas of the solid-state drive. In this embodiment, no specific limit is imposed on the number of phases. For example, four-phase clocks of 0°, 90°, 180°, and 270° are generated to produce stable clock signals. Through physical Bank partitioning, the NAND flash memory is divided into multiple physically independent Banks (set to 4). Each Bank includes an independent data bus (DQ), exclusive control signals (CE#, WE#, RE#), and a private cache register (Cache Register). Parallel operation eliminates bus conflicts and maximizes IO utilization. Among them, the multi-Bank parallel timing control cross-phase scheduling specifically includes: Bank0 corresponds to a phase offset of 0° and the applicable operation is page read; Bank1 corresponds to a phase offset of 90° and the applicable operation is cache programming; Bank2 corresponds to a phase offset of 180° and the applicable operation is block erase; Bank3 corresponds to a phase offset of 270° and the applicable operation is status register access.

[0122] Therefore, a clock signal is generated according to the multi-phase clock tree and used as the benchmark for the entire timing control. Further, an initial timing signal is generated based on the command and the clock signal. Subsequently, the data bus DQ[7:0] signal is sampled to determine whether the data bus signal is stable.

[0123] If it is confirmed that the signal is unstable, a delay is added to the initial timing signal, which is specifically implemented based on a programmable delay line and supports delay adjustment in steps of 0.1 ns to obtain the timing signal. If it is confirmed that the signal is stable, the initial timing signal is determined as the timing signal. Among them, the timing signal at least includes a chip select signal, a read / write enable signal, and an address latch signal.

[0124] In this embodiment, a basic clock signal is generated through the multi-phase clock tree and a delay is added to the timing signal, ensuring the correct operation sequence and the time interval conforming to the NAND technical specification.

[0125] Based on the above embodiment, in some embodiments, after the data transmission control module receives the write status information / read status information, it further includes:

[0126] S135: Obtain the status register value, the actual programming time, the error correction code count, and the flash memory particle temperature according to the real-time write status information / read status information.

[0127] S136: When it is determined that the target data write / read fails according to the status register value, adjust the dynamic transmission rate to the second transmission rate to obtain the adjusted dynamic transmission rate.

[0128] S137: When the error correction code count is greater than the first threshold, adjust the dynamic transmission rate to the second transmission rate to obtain the adjusted dynamic transmission rate.

[0129] S138: When the flash memory particle temperature is greater than the temperature threshold, adjust the dynamic transmission rate to the second transmission rate to obtain the adjusted dynamic transmission rate.

[0130] S139: When the actual programming time is greater than the programming time threshold, adjust the dynamic transmission rate to the third transmission rate to obtain the adjusted dynamic transmission rate.

[0131] S140: Control the writing / reading of the target data according to the adjusted dynamic transmission rate.

[0132] During the writing / reading of the target data, the data transmission control module receives the writing status information / reading status information transmitted by the timing control module in real time. During this process, the data transmission control module will adjust the dynamic transmission rate based on the real-time data transmission status, as follows:

[0133] According to the real-time writing status information / reading status information, obtain the status register value (StatusRegister), the actual programming time (tPROG_actual), the error correction code count (ecc_count), and the flash memory particle temperature (temp). Check the least significant bit of the NAND status register value, that is, the status register value. If this bit is 1, it means that the writing / reading fails. At this time, adjust the data transmission rate to the second transmission rate to avoid data loss or errors caused by writing failures, and at the same time, it may reserve more time and resources for subsequent retransmission operations.

[0134] If the error correction code count is greater than the set first threshold, it indicates that there are more data errors. Adjust the transmission rate to the second transmission rate, that is, reduce it to half of the full speed. By reducing the rate, the accuracy of data transmission can be improved and the probability of errors can be reduced. When the flash memory particle temperature is greater than the set temperature threshold, for the protection of the flash memory, adjust the transmission rate to the second transmission rate. Reducing the rate can reduce the power consumption of the flash memory and avoid damage to the flash memory caused by high temperature. If the actual programming time is greater than the programming time threshold, for example, greater than 1.2 times the nominal programming time (tPROG_NOMINAL), it means that the flash memory particles may be worn. To protect the particles and ensure the stability of data writing, adjust the transmission rate to the third transmission rate.

[0135] The adjusted dynamic transmission rate is obtained through the above adjustment method, and then the subsequent writing / reading of the target data is controlled according to the adjusted dynamic transmission rate. In addition, if the above conditions are not all met, that is, the writing is successful, the ECC error correction count does not exceed the threshold, the flash temperature is normal, and the actual programming time does not exceed the specified range, then the full-speed transmission is maintained to ensure the high efficiency of data transmission.

[0136] In this way, by adjusting the dynamic transmission rate in real time during the data writing / reading process, the high-efficiency transmission of data is ensured, and at the same time, the system load balance is ensured.

[0137] Based on the above embodiments, in some embodiments, after the write operation / read operation / erase operation is completed, it further includes:

[0138] S141: The data transmission control module determines whether the write operation / read operation / erase operation is successful according to the write status information / read status information / erase status information; if so, go to step S142; if not, go to step S143.

[0139] S142: The data transmission control module reports the write result / read result / erase result to the command scheduling module.

[0140] S143: The data transmission control module starts the error handling process.

[0141] Specifically, after the write operation / read operation / erase operation is completed, the data transmission control module will take corresponding handling measures according to the write status information / read status information / erase status information. If the feedback information shows that the operation is successful, the data transmission control module will continue to process the next batch of data to be processed. If the feedback information shows an error, the data transmission control module will immediately start the error handling process, which may include retransmitting the data or reporting the error situation to the command scheduling module.

[0142] At the same time, the data transmission control module will also synchronously feedback the write status information / read status information / erase status information to the command scheduling module. After receiving the feedback information from the data transmission control module, if the command scheduling module confirms success, it will continue to process the next command; if an error occurs, it will decide whether to re-initiate the command or report the error to the host according to the error type and priority.

[0143] In summary, the feedback on the execution status of the storage operation task is realized, so as to better determine the execution status of the task and take corresponding handling measures.

[0144] Figure 3 This is the architecture diagram of a pluggable hardware computing cluster provided by the embodiments of the present invention. As Figure 3As shown in the figure, the computing cluster is composed of host nodes and multiple distributed computing and storage cooperation units. The host node is communicatively connected to the multiple distributed computing and storage cooperation units through a second preset protocol switching module, and can plug and unplug the distributed computing and storage cooperation units at any time according to the change of the traffic volume to adjust the system configuration. The distributed computing and storage cooperation unit includes a solid-state drive and multiple near-storage computing nodes; the structure of the solid-state drive is the same as that in the above embodiment and will not be described here. It should be noted that in this embodiment, the communication protocol of the second preset protocol switching module is not limited. For example, it can be the same as or different from the communication protocol of the first preset protocol switching module, depending on the specific implementation situation.

[0145] It should be noted that the host node, as the core control unit of the entire pluggable hardware computing cluster, is responsible for managing data read and write operations, scheduling tasks, handling errors, and communicating with external systems. It runs a firmware program to coordinate and control the pluggable hardware modules. The host node module exists in the form of an independent circuit board, including a main control unit, memory (for caching data and running firmware), a clock circuit, and a power management circuit. When it is necessary to upgrade or replace the main control module, only need to unplug and insert a new main control module.

[0146] Based on the above pluggable hardware computing cluster, a task scheduling method is proposed in this embodiment, which is applied to the host node. By analyzing the characteristics of the computing tasks and quantitatively evaluating the relevant capabilities of the computing units, it is determined to which computing unit the tasks should be sent. A tight cooperation mechanism is built between the SSD controller and the storage area. Through reasonable scheduling of the computing tasks, cooperative computing is realized, the occupation of the near-storage computing nodes is reduced, the internal data transmission volume of the system is reduced, and thus the overall computing efficiency is improved.

[0147] Figure 4 It is a schematic diagram of a task scheduling method provided by an embodiment of the present invention. As Figure 4 shown, the method includes:

[0148] S15: When receiving a computing task, perform task feature analysis on the computing task to determine the computing task type.

[0149] Specifically, when the host receives a computing task, it first needs to extract key information such as the data requirements, computing operation type, and expected execution time of the task, and perform task feature analysis on the computing task based on these key information to determine the computing task type. It should be noted that in this embodiment, the specific process of determining the computing task type is not limited.

[0150] S16: Determine the adaptation task types of the respective distributed computing and storage cooperation units.

[0151] S17: Determine a target distributed computing and storage cooperation unit among the distributed computing and storage cooperation units according to the computing task type and each adaptation task type.

[0152] It can be understood that since there may be differences in the hardware devices among the distributed computing and storage cooperation units, and the computing capabilities of different near-storage computing nodes and micro-computing nodes are different, in order to better execute computing tasks, it is necessary to further determine the adaptation task types of the distributed computing and storage cooperation units, so as to determine the target distributed computing and storage cooperation unit among the distributed computing and storage cooperation units according to the computing task type and each adaptation task type, and thus use the most suitable target distributed computing and storage cooperation unit to process the computing task.

[0153] It should be noted that in this embodiment, there is no limitation on the specific process of determining the adaptation task types of the distributed computing and storage cooperation units, nor on the specific process of determining the target distributed computing and storage cooperation unit, which depends on the specific implementation situation.

[0154] S18: Transmit the computing task to the target distributed computing and storage cooperation unit through the second preset protocol transfer module to execute the computing task.

[0155] Finally, after determining the target distributed computing and storage cooperation unit, transmit the computing task to the target distributed computing and storage cooperation unit through the second preset protocol transfer module to execute the computing task through the target distributed computing and storage cooperation unit. It should be noted that there is no limitation on the specific manner in which the target distributed computing and storage cooperation unit executes the computing task in this embodiment.

[0156] In this embodiment, by analyzing the characteristics of the computing task and quantitatively evaluating the relevant capabilities of each computing unit, the adaptation task types of each computing unit are determined, so as to accurately transmit the computing task to the corresponding computing unit, realizing the reasonable scheduling and efficient allocation of the computing task, reducing the occupation of the near-storage computing node, reducing the internal data transmission volume of the system, and thus improving the overall computing efficiency.

[0157] Based on the above embodiments, in some embodiments, task feature analysis is performed on the computing task to determine the computing task type, including:

[0158] S151: Determine the degree of association between the data required by the computing task and the local data of each distributed computing and storage cooperation unit.

[0159] S152: Determine the computing complexity of the computing task according to the number of operations, operation types, and operation dependency relationships of the computing task.

[0160] S153: Determine the type of computing task according to the degree of association and the computational complexity;

[0161] Among them, the types of computing tasks include the first type, the second type, the third type, and the fourth type; for the computing tasks of the first type, the degree of association is greater than the first preset value, and the computational complexity is not greater than the second preset value; for the computing tasks of the second type, the degree of association is greater than the first preset value, and the computational complexity is greater than the second preset value; for the computing tasks of the third type, the degree of association is not greater than the first preset value, and the computational complexity is not greater than the second preset value; for the computing tasks of the fourth type, the degree of association is not greater than the first preset value, and the computational complexity is greater than the second preset value.

[0162] To determine the type of computing task, three task characteristics are specifically considered in this embodiment:

[0163] One is data correlation, which evaluates the degree of association between the data required for the computing task and the local data of each distributed computing and storage cooperation unit. If the computing task mainly operates on a large amount of data in a specific area within the distributed computing and storage cooperation unit and the data transmission overhead is large, then such a task has high data correlation. For example, a task of performing statistical analysis on a certain continuous data block in the storage space. For a task that needs to frequently obtain data from different storage areas or external data sources, the data correlation is considered to be low.

[0164] The second is computational complexity. Decompose the computational operations of the computing task into basic operations, and estimate the computational complexity according to the number of operations, the type of operations, and the running dependency relationship. Tasks composed of simple logical judgments, arithmetic operations on a small amount of data, etc. have low computational complexity. Tasks involving complex mathematical models and large-scale data matrix operations have high computational complexity, such as convolutional operation tasks in deep learning;

[0165] Therefore, in this embodiment, the type of computing task is determined according to the degree of association and the computational complexity. Specifically, the computing tasks are subdivided into four types: the first type, the second type, the third type, and the fourth type; for the computing tasks of the first type, the degree of association is greater than the first preset value, and the computational complexity is not greater than the second preset value; for the computing tasks of the second type, the degree of association is greater than the first preset value, and the computational complexity is greater than the second preset value; for the computing tasks of the third type, the degree of association is not greater than the first preset value, and the computational complexity is not greater than the second preset value; for the computing tasks of the fourth type, the degree of association is not greater than the first preset value, and the computational complexity is greater than the second preset value. Generally speaking, the computing tasks of the first type, the second type, the third type, and the fourth type are tasks with high data correlation and low computational complexity, tasks with high data correlation and high computational complexity, tasks with low data correlation and low computational complexity, and tasks with low data correlation and high computational complexity, respectively.

[0166] It should be noted that in this embodiment, there is no limitation on the magnitudes of the first preset value and the second preset value, which are determined according to specific implementation circumstances.

[0167] In this embodiment, the type of a computing task is determined according to the data correlation and computing complexity of the computing task, so as to facilitate determining a suitable distributed computing and storage cooperation unit in the subsequent process for efficiently and completely executing the computing task.

[0168] Based on the above embodiment, when multiple computing tasks are received, before performing task feature analysis on the computing tasks, it further includes:

[0169] S154: Determine the first priority score of each computing task according to whether each computing task needs to be processed in real time.

[0170] Among them, the first priority score of a computing task that needs to be processed in real time is higher than that of a computing task that does not need to be processed in real time.

[0171] S155: Determine the deadline of each computing task and determine the maximum task execution time.

[0172] S156: Determine the difference between the current time and the deadline of each computing task.

[0173] S157: Determine the second priority score of each computing task according to each difference and the maximum task execution time.

[0174] S158: Determine the third priority score of each computing task according to the importance level of each computing task.

[0175] S159: Determine the total priority score of the corresponding computing task according to each first priority score, each second priority score, and each third priority score, so as to process each computing task in sequence according to the corresponding total priority score.

[0176] Considering that the host node may receive multiple computing tasks simultaneously, in order to ensure the sequential execution of each computing task, in this embodiment, specifically determine the first priority score P of each computing task according to whether each computing task needs to be processed in real time t . For example, the first priority of a computing task that needs to be processed in real time can be set to P t1 , and the first priority score of a computing task that does not need to be processed in real time is set to P t2 . Among them, the first priority score of a computing task that needs to be processed in real time is higher than that of a computing task that does not need to be processed in real time, that is, P t1 >P t2 .

[0177] Further determine the deadline of each computing task and determine the maximum task execution time Dmax Determine the difference d between the current time and the deadline of each computing task, and based on each difference and the maximum task execution time D max , determine the second priority score P of each computing task d = 1 - d / D max .

[0178] According to the importance level of each computing task, determine the third priority score P of each computing task i . In this embodiment, the specific type of the importance level is not limited. For example, it can be set that the high importance level corresponds to the priority score P i1 , the medium importance level corresponds to the score P i2 , the low importance level corresponds to the score P i3 , and P i1 > P i2 > P i3 .

[0179] Finally, based on each first priority score P t1 , each second priority score P d and each third priority P i score, determine the total priority score of the corresponding computing task, so as to process each computing task in descending order according to the corresponding total priority score. The specific calculation formula of the total priority score is as follows:

[0180] P = w t P t + w d P d + w i P i ;

[0181] where P is the total priority score of the corresponding computing task, w t , w d and w i are weight coefficients respectively, and w t + w d + w i = 1.

[0182] In this embodiment, by determining the priority degrees of the task types, deadlines, and importance levels of the computing tasks respectively, the priorities of the computing tasks are determined, so as to process each computing task in order of priority and ensure the rationality of the task execution order.

[0183] On the basis of the above embodiment, in some embodiments, determine the adapted task types of each distributed computing and storage collaboration unit, including:

[0184] S160: Determine the computing units corresponding to each distributed computing and storage collaboration unit.

[0185] Among them, the computing unit includes a near-memory computing node and a micro-computing unit in a solid-state drive controller.

[0186] S161: Perform data correlation quantization evaluation and computing complexity quantization evaluation on each computing unit respectively to determine the data correlation quantization evaluation result and computing complexity quantization evaluation result of each computing unit.

[0187] S162: Determine the task adaptability index of the corresponding computing unit according to each computing complexity quantization evaluation result and each data correlation quantization evaluation result.

[0188] S163: Determine the adapted task type of each computing unit according to the task adaptability index.

[0189] In a specific implementation, in order to determine the adapted task type of each distributed computing and storage cooperation unit, it is first necessary to determine the computing unit corresponding to each distributed computing and storage cooperation unit.

[0190] It can be understood that computing tasks need to be executed by units with computing capabilities. In the distributed computing and storage cooperation unit, the units with computing capabilities include the near-memory computing node and the micro-computing unit in the solid-state drive controller. Therefore, the computing unit in this embodiment should include the near-memory computing node and the micro-computing unit in the solid-state drive controller, and subsequent computing tasks should also be executed by the near-memory computing node or the micro-computing unit in the distributed computing and storage cooperation unit.

[0191] Furthermore, perform data correlation quantization evaluation and computing complexity quantization evaluation on each computing unit respectively to determine the data correlation quantization evaluation result and computing complexity quantization evaluation result of each computing unit. It should be noted that the data correlation quantization evaluation aims to measure the locality of the data access pattern when the computing unit executes a task. The computing complexity quantization evaluation focuses on the computing resources and time required for the computing unit to execute a specific task. It is usually measured by the time complexity and space complexity of the algorithm, reflecting the computing difficulty and resource requirements of the task. In this embodiment, the specific process of the data correlation quantization evaluation and the computing complexity quantization evaluation is not limited and depends on the specific implementation situation.

[0192] Finally, determine the task adaptability index of the corresponding computing unit according to each computing complexity quantization evaluation result and each data correlation quantization evaluation result, and determine the adapted task type of each computing unit according to the task adaptability index.

[0193] In this embodiment, by performing data correlation quantization evaluation and computing complexity quantization evaluation on the computing units of the distributed computing and storage cooperation unit, the adaptation task type of the computing unit can be finally determined according to the results of each computing complexity quantization evaluation and each data correlation quantization evaluation, so as to accurately determine the computing unit to which the computing task should be sent, and realize the reasonable scheduling and efficient allocation of the computing task.

[0194] On the basis of the above embodiment, in some embodiments, performing data correlation quantization evaluation on each computing unit includes:

[0195] S164: Determine the data access times and the total data access times of each computing unit for obtaining data from the local storage space when executing a single task.

[0196] Among them, the local storage space includes the local memory of the near-storage computing node and the storage area corresponding to the micro-computing unit.

[0197] S165: Determine the data local access rate index of the corresponding computing unit according to each data access time and each total data access time.

[0198] S166: Determine the number of computing operations involved in each computing unit when executing a single task and the total data transmission volume transmitted with external components.

[0199] S167: Determine the transmission-computation ratio index of the corresponding computing unit according to each total data transmission volume and each number of computing operations.

[0200] Specifically, during the process of a computing unit executing a single task, count the data access times N of obtaining data from the local storage space (such as the local memory of the near-storage computing node, the storage space corresponding to the micro-computing unit in the SSD, etc.) local , and the total data access times N total . According to each data access time N local and each total data access time N total , determine the data local access rate index R local = N local / N total . The closer this index is to 1, the higher the degree of dependence of the computing unit on local data when processing tasks, and the more suitable it is for processing tasks with high data correlation. For example, when executing a certain task, the total data access times is 1000 times, and the number of times N local of obtaining data from the local storage is 800 times, then R local = 0.8.

[0201] Furthermore, measure the total data transmission volume D between the computing unit and external storage or other components during the execution of a single task by the computing unit transfer, and record the number of computing operations N involved in completing the task operations , estimated by instruction counting, based on the total data transfer volume D for each transfer and the number of computing operations N for each operations , determine the transfer-computation ratio index R of the corresponding computing unit dt =D transfer / N operations . The lower this ratio, the smaller the data transfer volume associated with each unit computing operation, the less the computing unit is affected by data transfer when processing this task, and it is more suitable for processing tasks with high data correlation. For example, if a computing unit transfers 500 MB of data to complete a task, and the estimated number of computing operations involved is 10 million times, then R dt =(500 × 1024 × 1024) / (1000 × 10000)=52.4288 Byte / instruction operation.

[0202] In summary, by determining the data local access rate index and transfer-computation ratio index of the computing node, a quantitative evaluation of the data correlation of the computing node is achieved.

[0203] Based on the above embodiments, in some embodiments, a quantitative evaluation of the computing complexity of each computing unit is performed, including:

[0204] S168: Determine the number of non-scalar instruction executions and the total number of instruction executions of each computing unit when executing a single task.

[0205] Among them, non-scalar instructions include at least arithmetic operation instructions and multi-cycle instructions.

[0206] S169: Determine the non-scalar instruction execution ratio index of the corresponding computing unit according to the number of non-scalar instruction executions and the total number of instruction executions of each.

[0207] S170: Determine the first execution time of each computing unit when executing a first reference task with a corresponding computing complexity lower than a third preset value, and the second execution time when executing a second reference task with a corresponding computing complexity not lower than the third preset value.

[0208] S171: Determine the execution complexity index of the corresponding computing unit according to each first execution time and each second execution time.

[0209] Specifically, during the process of a computing unit executing a single task, count the number of non-scalar instruction executions M complex , that is, count the number of arithmetic operation instructions and multi-cycle instruction executions; at the same time, count the total number of instruction executions M total . According to the number of non-scalar instruction executions M for each complexand the total number of executions M of each instruction total , determine the non-scalar instruction execution ratio index P of the corresponding computing unit complex = M complex / M total . The higher this ratio, the greater the proportion of complex calculations executed by the computing unit when processing tasks, and the more suitable it is for processing tasks with high computational complexity. For example, during the execution of a task, the total number of instruction executions is 50 million times, and the number of complex instruction executions is 10 million times, then P complex = 0.2

[0210] Furthermore, select a first reference task with a known computational complexity lower than the third preset value (such as a simple addition operation sequence), and record its first execution time T on the computing unit simple ; at the same time, select a second reference task with a known computational complexity not lower than the third preset value, and record its second execution time T on the computing unit target . According to each first execution time T simple and each second execution time T target , determine the execution complexity index R of the corresponding computing unit time = T target / T simple . The larger this ratio, the greater the increase in the execution time of the second reference task relative to the first reference task, reflecting the higher computational complexity of the second reference task. If the computing unit can efficiently complete such tasks, it is more suitable for processing tasks with high computational complexity. For example, if the execution time of the first reference task is 1 second and the execution time of the second task is 10 seconds, then R time = 10. In this embodiment, there is no limit on the size of the third preset value

[0211] In summary, by determining the non-scalar instruction execution ratio index and the execution complexity index of the computing node, a quantitative evaluation of the computational complexity of the computing node is achieved

[0212] Based on the above embodiments, in some embodiments, according to each computational complexity quantitative evaluation result and each data correlation quantitative evaluation result, determine the task suitability index of the corresponding computing unit, including:

[0213] S172: According to each data local access rate index, each transfer-computation ratio index, each non-scalar instruction execution ratio index, and each execution complexity index, determine the task suitability index of the corresponding computing unit

[0214] Among them, the task suitability index includes a first task suitability index corresponding to the first type, a second task suitability index corresponding to the second type, a third task suitability index corresponding to the third type, and a fourth task suitability index corresponding to the fourth type

[0215] Specifically, after determining the quantitative evaluation results of the computational complexity and data correlation of each computing node, a task adaptability index for the computing unit is constructed based on the quantitative evaluation results of the computational complexity and data correlation, as follows:

[0216] (1) Construct a first task adaptability index that represents high data correlation and low computational complexity. The first task adaptability index corresponds to the first type. The specific formula is as follows:

[0217] S1 = w1R local + w2(1 - R dt ) + w3(1 - P complex ) + w4(1 - R time );

[0218] Among them, S1 is the first task adaptability index, and w1, w2, w3, and w4 are all weights, which are determined according to the degree of emphasis on each index in the actual application scenario, and w1 + w2 + w3 + w4 = 1. The higher the S1 value of the computing unit, the stronger its ability to handle tasks with high data correlation and low computational complexity.

[0219] (2) Construct a second task adaptability index that represents high data correlation and high computational complexity. The second task adaptability index corresponds to the second type. The specific formula is as follows:

[0220] S2 = w5R local + w6(1 - R dt ) + w7P complex + w8R time ;

[0221] Among them, S2 is the second task adaptability index, and w5, w6, w7, and w8 are all weights, which are determined according to the degree of emphasis on each index in the actual application scenario, and w5 + w6 + w7 + w8 = 1. The higher the S2 value of the computing unit, the more prominent its ability to handle such tasks.

[0222] (3) Construct a third task adaptability index that represents low data correlation and low computational complexity. The third task adaptability index corresponds to the third type. The specific formula is as follows:

[0223] S3 = w9(1 - R local ) + w 10 R dt + w 11 (1 - P complex ) + w 12 (1 - R time );

[0224] Among them, S3 is the third task adaptability index, and w9, w10 , w 11 and w 12 are both weights, determined according to the degree of emphasis on each indicator in the actual application scenario, and w9 + w 10 + w 11 + w 12 = 1. The higher the S3 value of the computing unit, the better the computing unit performs in processing such tasks.

[0225] (4) Construct a fourth task adaptability indicator that characterizes low data correlation and high computational complexity. The fourth task adaptability indicator corresponds to the fourth type. The specific formula is as follows:

[0226] S4 = w 13 (1 - R local ) + w 14 R dt + w 15 P complex + w 16 R time ;

[0227] where S4 is the fourth task adaptability indicator, w 13 , w 14 , w 15 and w 16 are both weights, determined according to the degree of emphasis on each indicator in the actual application scenario, and w 13 + w 14 + w 15 + w 16 = 1. The higher the S4 value of the computing unit, the stronger the computing unit's ability to process such tasks.

[0228] Correspondingly, determine the adapted task types of each computing unit according to the task adaptability indicators, including:

[0229] S173: Determine the maximum value of each task adaptability indicator corresponding to each computing unit.

[0230] S174: Determine the task type corresponding to the maximum value as the adapted task type of the computing unit.

[0231] Finally, after obtaining the task adaptability indicators of each computing node, determine the maximum value of the corresponding task adaptability indicators, and determine the task type corresponding to the maximum value as the adapted task type of the computing unit. For example, among the task adaptability indicators corresponding to the computing unit. If S3 is significantly higher than other indicators, then the adapted task type of the computing node is a computing task with low data correlation and high computational complexity.

[0232] Based on the above embodiments, in some embodiments, determining a target distributed computing and storage cooperation unit in each distributed computing and storage cooperation unit according to the computing task type and each adaptation task type includes:

[0233] S175: Determine candidate computing units corresponding to the computing task type according to the adaptation task types of each computing unit.

[0234] S176: Obtain the current performance status of each candidate computing unit, and determine whether the corresponding candidate computing unit is allowed to execute the computing task according to each current performance status; if so, proceed to step S177.

[0235] S177: Determine the corresponding candidate computing unit as the target computing unit.

[0236] Specifically, after determining the computing task type and the adaptation task types of each computing node, compare the computing task type with the adaptation task types of each computing node to find the adaptation task type that is the same as the computing task type, so as to determine the corresponding multiple candidate computing units.

[0237] To ensure that the computing unit can execute smoothly, it is also necessary to obtain the current performance status of each candidate computing unit, including but not limited to computing power, memory bandwidth, cache hit rate, latency, and energy efficiency, etc., and determine whether the corresponding candidate computing unit is allowed to execute the computing task according to each current performance status. If it is confirmed that the corresponding candidate computing unit is allowed to execute the computing task based on the current performance status, then determine the corresponding candidate computing unit as the target computing unit, so that the target computing unit can execute the computing task. If it is confirmed that none of the candidate computing units are allowed to execute the computing task, then after waiting for a period of time, enter again the step of obtaining the current performance status of each candidate computing unit and determining whether the corresponding candidate computing unit is allowed to execute the computing task according to each current performance status.

[0238] In addition, since the target computing unit can be a near-storage computing node in the target distributed computing and storage cooperation unit or a micro-control unit in the SSD controller, when forwarding the computing task to the target distributed computing and storage cooperation unit through the second preset protocol transfer module, it is necessary to specifically forward the computing task to the target computing unit through the second preset protocol transfer module, or forward the computing task to the target computing unit through the second preset protocol transfer module and the first preset protocol transfer module to execute the computing task.

[0239] In summary, the selection of the target computing unit for executing the computing task is realized according to the adaptation task types of each computing unit. During the process, through in-depth analysis of the characteristics of the computing task and quantitative evaluation of the relevant capabilities of the computing unit, the accurate determination of the target computing unit is achieved. The collaborative computing and efficient task allocation are realized, the occupation of the near-memory computing unit is reduced, the internal data transmission volume of the system is decreased, and thus the overall computing efficiency is improved.

[0240] In addition, in order to further improve the processing efficiency of the computing task, before the computing cluster runs, the total amount of computable resources that can be allocated in the system can be clarified, including the near-memory computing unit and the computing unit corresponding to the storage area; at the same time, relevant weight parameters for quantitative evaluation of resource capabilities are set; the initial information of the storage device and the computing task is collected, and the initialization settings of load monitoring and priority evaluation are completed.

[0241] When each computing unit executes the computing task for the first time and conducts task adaptability evaluation, the following strategy can be adopted:

[0242] For computing tasks with high data correlation and low computational complexity, they are preferentially sent to the micro-computing unit corresponding to the storage space for execution. Such tasks can make full use of the local data in the storage space, reduce the data transmission overhead, and the computing unit of the storage chip can quickly complete simple calculations.

[0243] For computing tasks with high data correlation and high computational complexity, it is necessary to comprehensively evaluate the task timeliness and the communication link status. If the timeliness is strong and the communication between the near-memory computing unit and the data source is good, it can be sent to the near-memory computing unit for execution to quickly complete the task with its powerful computing ability. If the timeliness requirement is relatively low or the timeliness is strong, but the communication load between the near-memory computing unit and the data source is high, it is sent to the micro-computing unit in the storage space for execution.

[0244] For computing tasks with low data correlation and low computational complexity, it is necessary to comprehensively evaluate the task timeliness and the communication link status. If the timeliness is strong, it is sent to the near-memory computing unit for execution to quickly complete the task with its powerful computing ability. If the timeliness requirement is relatively low, it is sent to the micro-computing unit in the storage space for execution.

[0245] For computing tasks with low data correlation and high computational complexity, the near-memory computing unit is preferentially considered. Because the task has little dependence on local data, the general computing ability of the near-memory computing unit can efficiently process such tasks, and at the same time, the good communication ability with other components can be used to obtain the required data.

[0246] In summary, through the first task evaluation of the computing unit, the determination efficiency of the target computing unit can be greatly shortened, the processing speed of the computing task can be improved, and the resources of the computing cluster are saved.

[0247] Based on the above embodiments, in some embodiments, the method further includes:

[0248] S178: During the execution of the computing task, according to a preset period / triggered by an event, re-determine the adaptation task types of each distributed computing and storage cooperation unit, and return to the step of determining the target distributed computing and storage cooperation unit in each distributed computing and storage cooperation unit according to the computing task type and each adaptation task type.

[0249] Specifically, during the execution of the computing task by the target computing node, according to a preset period or triggered by an event, update the load information of the computing unit and the storage space and the status information of the computing task; re-calculate the quantization score of the computing unit and the priority score of the computing task; according to the latest score results, determine the task types suitable for the computing unit to execute, and select the computing unit with the highest score to adjust and allocate the computing task; monitor the system operation situation after resource allocation, and make necessary resource allocation adjustments according to the feedback, so as to realize the dynamic adjustment and allocation of resources and achieve the precise control of computing resources.

[0250] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0251] Figure 5 It is a schematic diagram of a task processing device provided by an embodiment of the present invention. The task processing device is applied to a solid-state drive controller of a solid-state drive; the solid-state drive controller at least includes a command scheduling module, micro-computing units corresponding to each storage area of the solid-state drive, a data transmission control module, and a timing control module; the solid-state drive controller is communicatively connected to a plurality of near-storage computing nodes through a first preset protocol transfer module; as Figure 5 shown, the task processing device includes:

[0252] An allocation module 10, configured to, when receiving a command of a storage operation task transmitted by a host through the first preset protocol transfer module, determine a target storage area corresponding to the command through the command scheduling module, and allocate the command to the data transmission control module corresponding to the target storage area; wherein, the host includes a host node and each near-storage computing node;

[0253] A processing module 11, configured to transmit the command to the corresponding timing control module through the data transmission control module, and the data transmission control module and the timing control module process the storage operation task according to the command.

[0254] For the description of the features in the corresponding embodiment of the task processing device, reference can be made to the relevant description in the corresponding embodiment of the task processing method, which will not be elaborated here one by one.

[0255] Figure 6 The figure is a schematic diagram of a task scheduling device provided by an embodiment of the present invention. The task scheduling device is applied to a host node; the host node is communicatively connected to a plurality of distributed computing and storage cooperation units through a second preset protocol transfer module; the distributed computing and storage cooperation units include a solid-state drive and a plurality of near-storage computing nodes; the solid-state drive controller of the solid-state drive at least includes a command scheduling module, a micro-computing unit, a data transmission control module, and a timing control module corresponding to each storage area of the solid-state drive; the solid-state drive controller is communicatively connected to the corresponding plurality of near-storage computing nodes through a first preset protocol transfer module, as Figure 6 shown, the task scheduling device includes:

[0256] An analysis module 12, configured to perform task feature analysis on a computing task when receiving the computing task to determine the computing task type;

[0257] A first determination module 13, configured to determine the adapted task types of each distributed computing and storage cooperation unit;

[0258] A second determination module 14, configured to determine a target distributed computing and storage cooperation unit among each distributed computing and storage cooperation unit according to the computing task type and each adapted task type;

[0259] A transmission module 15, configured to send the computing task to the target distributed computing and storage cooperation unit through the second preset protocol transfer module to execute the computing task.

[0260] For the description of the features in the corresponding embodiment of the task scheduling device, reference can be made to the relevant description in the corresponding embodiment of the task scheduling method, which will not be elaborated here one by one.

[0261] Figure 7 The figure is a schematic diagram of an electronic device provided by an embodiment of the present invention. As Figure 7 shown, the electronic device includes a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in the embodiment of the above-mentioned task processing method, or execute the steps in the embodiment of the above-mentioned task scheduling method.

[0262] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in the embodiment of the above-mentioned task processing method, or execute the steps in the embodiment of the above-mentioned task scheduling method when running.

[0263] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), external hard drives, magnetic disks, or optical discs that can store computer programs.

[0264] An embodiment of the present invention also provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in the embodiment of the above task processing method or the steps in the embodiment of the above task scheduling method.

[0265] An embodiment of the present invention also provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in the embodiment of the above task processing method or the steps in the embodiment of the above task scheduling method.

[0266] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0267] The above has introduced in detail a task processing method, a task scheduling method, a device, a medium, and a product provided by the present invention. Specific examples are used herein to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A task processing method, characterized in that: A solid-state hard disk controller applied to a solid-state hard disk; the solid-state hard disk controller at least includes a command scheduling module, a micro-computing unit corresponding to each storage area of ​​the solid-state hard disk, a data transmission control module and a timing control module; The solid state drive controller is communicatively connected with a plurality of near storage computing nodes via a first preset protocol adapter module; the method comprises: When receiving a storage operation task command transmitted by the host through the first preset protocol adapter module, the command scheduling module determines the target storage area corresponding to the command, and distributes the command to the data transmission control module corresponding to the target storage area; wherein the host includes a host node and each of the near storage computing nodes; The command is transmitted to the corresponding timing control module through the data transmission control module, and the data transmission control module and the timing control module process the storage operation task according to the command.

2. The task processing method according to claim 1, characterized in that: Determining the target storage area corresponding to the command through the command scheduling module, and allocating the command to the data transmission control module corresponding to the target storage area, including: Performing syntax analysis and semantic analysis on the command by the command scheduling module to determine command information of the command; The command information includes at least the command type, sub-operation type, target storage area identifier, address information, partition identifier, data length, data content, data verification information, access control information, encryption flag, authentication information, timing control information, number of retries, response requirements and error handling method; The command and the command information are transmitted to the corresponding data transmission control module through the command scheduling module.

3. The task processing method according to claim 2, characterized in that: When the command type is a write operation, the command is transmitted to the corresponding timing control module through the data transmission control module, and the data transmission control module and the timing control module process the storage operation task according to the command, including: Cache the target data and verification data corresponding to the command by the data transmission control module; The data transmission control module verifies and corrects the target data according to the verification data; Determining the dynamic transmission rate of the target data by the data transmission control module; transmitting the command and the target data to the timing control module through the data transmission control module; The timing control module generates a timing signal according to the command; The timing control module transfers the target data to the target storage area according to the timing signal, writes the target data to the target storage area according to the timing signal and the dynamic transmission rate, and feeds back the write status information to the data transmission control module in real time until the write operation is completed.

4. The task processing method according to claim 2, characterized in that: When the command type is a read operation, the command is transmitted to the corresponding timing control module through the data transmission control module, and the data transmission control module and the timing control module process the storage operation task according to the command, including: The command is transmitted to the timing control module through the data transmission control module, and the target data corresponding to the command is determined; Determining the dynamic transmission rate of the target data by the data transmission control module; The timing control module generates a timing signal according to the command; The timing control module transmits the timing signal and the command to the target storage area, so as to read the target data and the corresponding verification data in the target storage area according to the timing signal and the dynamic transmission rate, and feeds back the reading status information to the data transmission control module in real time until the reading operation is completed; Transmitting the target data and the corresponding verification data to the data transmission control module through the timing control module; caching the target data and the verification data corresponding to the command by the data transmission control module; The data transmission control module verifies and corrects the target data according to the verification data; The target data is transmitted to the host through the data transmission control module and the command scheduling module.

5. The task processing method according to claim 2, characterized in that: When the command type is an erase operation, the command is transmitted to the corresponding timing control module through the data transmission control module, and the data transmission control module and the timing control module process the storage operation task according to the command, including: transmitting the command to the timing control module through the data transmission control module; The timing control module generates a timing signal according to the command; The timing signal and the command are transmitted to the target storage area through the timing control module to erase the data in the target storage area according to the timing signal, and the erasing status information is fed back to the data transmission control module in real time until the erasing operation is completed.

6. The task processing method according to claim 3 or 4, characterized in that: caching the target data and the verification data corresponding to the command by the data transmission control module, including: When the command type is a write operation, writing the corresponding target data and the verification data into a first buffer; When the command type is a read operation, the corresponding target data and the verification data are written into the second buffer.

7. The task processing method according to claim 6, characterized in that: Determining the dynamic transmission rate of the target data by the data transmission control module includes: Obtaining a temperature alarm signal, an error correction code count, and a system load factor of the solid state drive; When the temperature alarm signal is valid, determining the dynamic transmission rate to be a first transmission rate; When the error correction code count is greater than a first threshold, and / or the system load factor is greater than a second threshold, determining the dynamic transmission rate to be a second transmission rate; When the system load factor is greater than a third threshold value and not greater than the second threshold value, determining the dynamic transmission rate to be a third transmission rate; When the temperature alarm signal is invalid, the error correction code count is not greater than the first threshold, and the system load factor is not greater than the third threshold, determining the dynamic transmission rate to be a fourth transmission rate; The second threshold is greater than the third threshold; the first transmission rate is less than the second transmission rate, the second transmission rate is less than the third transmission rate, and the third transmission rate is less than the fourth transmission rate.

8. The task processing method according to claim 6, characterized in that: The data transmission control module verifies and corrects the target data according to the verification data, including: Checking the target data according to a cyclic redundancy check code to determine whether the target data is complete; If not, retransmitting the target data; If so, error correction is performed on the target data according to the error correction code.

9. The task processing method according to claim 6, characterized in that: The timing control module generates a timing signal according to the command, including: Generating a clock signal according to a multi-phase clock tree; wherein the number of phases of the multi-phase clock tree is the same as the number of storage areas of the solid state drive; generating an initial timing signal according to the command and the clock signal; Sampling the data bus signal to determine whether the data bus signal is stable; If yes, determining the initial timing signal as the timing signal; If not, adding a delay to the initial timing signal to obtain the timing signal; Wherein, the timing signal at least includes a chip select signal, a read / write enable signal and an address latch signal.

10. The task processing method according to claim 7, characterized in that: After receiving the write status information / read status information, the data transmission control module further includes: According to the real-time write status information / read status information, obtain the status register value, actual programming time, error correction code count and flash memory particle temperature; When it is determined according to the status register value that the target data writing / reading fails, adjusting the dynamic transmission rate to the second transmission rate to obtain the adjusted dynamic transmission rate; When the error correction code count is greater than a first threshold, adjusting the dynamic transmission rate to the second transmission rate to obtain the adjusted dynamic transmission rate; When the temperature of the flash memory particle is greater than a temperature threshold, adjusting the dynamic transmission rate to the second transmission rate to obtain the adjusted dynamic transmission rate; When the actual programming time is greater than the programming time threshold, adjusting the dynamic transmission rate to the third transmission rate to obtain the adjusted dynamic transmission rate; The target data is written / read according to the adjusted dynamic transmission rate control.

11. The task processing method according to any one of claims 3 to 5, characterized in that: After completing the write operation / read operation / erase operation, it also includes: The data transmission control module determines whether the write operation / read operation / erase operation is successful according to the write status information / read status information / erase status information; If yes, then report the write result / read result / erase result to the command scheduling module through the data transmission control module; If not, the error handling process is started by the data transmission control module.

12. A task scheduling method, characterized in that: Applied to a host node; the host node is connected to a plurality of distributed computing and storage coordination units through a second preset protocol switching module; the distributed computing and storage coordination unit includes a solid state drive and a plurality of near storage computing nodes; the solid state drive controller of the solid state drive includes at least a command scheduling module, a micro-computing unit corresponding to each storage area of ​​the solid state drive, a data transmission control module and a timing control module; The solid state drive controller is communicatively connected with the corresponding plurality of near storage computing nodes via a first preset protocol adapter module; the method comprises: When receiving a computing task, performing task feature analysis on the computing task to determine the computing task type; Determining the adaptation task type of each of the distributed computing and storage collaboration units; Determining a target distributed computing and storage collaboration unit in each of the distributed computing and storage collaboration units according to the computing task type and each of the adaptation task types; The computing task is sent to the target distributed computing and storage collaboration unit through the second preset protocol switching module to execute the computing task.

13. The task scheduling method according to claim 12, characterized in that: Performing task feature analysis on the computing task to determine the computing task type includes: Determine the degree of association between the data required for the computing task and the local data of each of the distributed computing and storage collaboration units; Determining the computational complexity of the computing task according to the number of operations, the type of operations, and the dependency of operations of the computing task; Determining the computing task type according to the association degree and the computing complexity; Among them, the computing task types include a first type, a second type, a third type and a fourth type; the correlation degree of the computing tasks of the first type is greater than a first preset value, and the computing complexity is not greater than a second preset value; the correlation degree of the computing tasks of the second type is greater than the first preset value, and the computing complexity is greater than the second preset value; the correlation degree of the computing tasks of the third type is not greater than the first preset value, and the computing complexity is not greater than the second preset value; the correlation degree of the computing tasks of the fourth type is not greater than the first preset value, and the computing complexity is greater than the second preset value.

14. The task scheduling method according to claim 13, characterized in that: When a plurality of computing tasks are received, before performing task feature analysis on the computing tasks, the method further includes: Determining the first priority score of each computing task according to whether each computing task requires real-time processing; wherein the first priority score of the computing task that requires real-time processing is higher than the first priority score of the computing task that does not require real-time processing; Determining a deadline for each of the computing tasks and determining a maximum task execution time; Determine the difference between the current time and the deadline of each of the computing tasks; Determining a second priority score for each of the computing tasks according to each of the differences and the maximum task execution time; Determining a third priority score for each of the computing tasks according to the importance level of each of the computing tasks; According to each of the first priority scores, each of the second priority scores and each of the third priority scores, a total priority score corresponding to the computing task is determined, so as to process each of the computing tasks in sequence according to the corresponding total priority score.

15. The task scheduling method according to claim 13, characterized in that: Determining the adaptation task type of each of the distributed computing and storage collaboration units includes: Determine the computing unit corresponding to each of the distributed computing and storage coordination units; wherein the computing unit includes the near storage computing node and the micro computing unit in the solid state drive controller; Performing data correlation quantitative evaluation and computational complexity quantitative evaluation on each of the computing units respectively to determine a data correlation quantitative evaluation result and a computational complexity quantitative evaluation result of each of the computing units; Determining the task adaptability index corresponding to the computing unit according to the quantitative evaluation results of the computational complexity and the quantitative evaluation results of the data relevance; The adaptation task type of each of the computing units is determined according to the task adaptability index.

16. The task scheduling method according to claim 15, characterized in that: Performing a quantitative evaluation of data relevance on each of the computing units, including: Determine the number of data accesses and the total number of data accesses of each computing unit to obtain data from the local storage space when executing a single task; wherein the local storage space includes the local memory of the near storage computing node and the storage area corresponding to the micro computing unit; Determine a data local access rate indicator corresponding to the computing unit according to the number of accesses to each data and the total number of accesses to each data; Determining the number of computing operations involved in each computing unit performing a single task and the total amount of data transmission transmitted with external components; According to the total amount of each data transmission and the number of each computing operation, a transfer-computation ratio indicator corresponding to the computing unit is determined.

17. The task scheduling method according to claim 16, characterized in that: Performing a computational complexity quantification evaluation on each of the computing units, including: Determining the number of executions of non-scalar instructions and the total number of instruction executions of each computing unit when executing a single task; wherein the non-scalar instructions include at least arithmetic operation instructions and multi-cycle instructions; Determine a non-scalar instruction execution ratio index corresponding to the computing unit according to the number of executions of each non-scalar instruction and the total number of executions of each instruction; Determine a first execution time of each of the computing units when executing a first benchmark task whose corresponding computational complexity is lower than a third preset value, and a second execution time of each of the computing units when executing a second benchmark task whose corresponding computational complexity is not lower than the third preset value; An execution complexity index corresponding to the computing unit is determined according to each of the first execution times and each of the second execution times.

18. The task scheduling method according to claim 17, characterized in that: According to each of the quantitative evaluation results of the computational complexity and each of the quantitative evaluation results of the data relevance, determining the task adaptability index corresponding to the computing unit includes: Determine the task adaptability index corresponding to the computing unit according to each of the data local access rate indicators, each of the transfer-to-computation ratio indicators, each of the non-scalar instruction execution ratio indicators, and each of the execution complexity indicators; The task adaptability index includes a first task adaptability index corresponding to the first type, a second task adaptability index corresponding to the second type, a third task adaptability index corresponding to the third type, and a fourth task adaptability index corresponding to the fourth type; Correspondingly, determining the adaptation task type of each computing unit according to the task adaptability index includes: Determining the maximum value of each of the task adaptability indicators corresponding to each of the computing units; The task type corresponding to the maximum value is determined as the adaptation task type of the computing unit.

19. The task scheduling method according to claim 15, characterized in that: Determining a target distributed computing and storage collaboration unit in each of the distributed computing and storage collaboration units according to the computing task type and each of the adaptation task types includes: Determine, according to the adaptation task type of each computing unit, a candidate computing unit corresponding to the computing task type; Acquire the current performance status of each of the candidate computing units, and determine whether the corresponding candidate computing unit is allowed to execute the computing task according to each of the current performance statuses; If yes, the corresponding candidate computing unit is determined as the target computing unit; Correspondingly, sending the computing task to the target distributed computing and storage collaboration unit through the second preset protocol switching module includes: The computing task is sent to the target computing unit through the second preset protocol switching module, or the computing task is sent to the target computing unit through the second preset protocol switching module and the first preset protocol switching module to execute the computing task.

20. The task scheduling method according to any one of claims 12 to 19, characterized in that: Also includes: During the execution of the computing task, the adaptation task type of each of the distributed computing and storage collaborative units is re-determined according to a preset cycle / according to an event trigger, and the process returns to the step of determining the target distributed computing and storage collaborative unit in each of the distributed computing and storage collaborative units according to the computing task type and each adaptation task type.

21. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the task processing method according to any one of claims 1 to 11, or the steps of the task scheduling method according to any one of claims 12 to 20 when executing the computer program.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the task processing method according to any one of claims 1 to 11, or the steps of the task scheduling method according to any one of claims 12 to 20.

23. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the task processing method according to any one of claims 1 to 11 or the steps of the task scheduling method according to any one of claims 12 to 20 are implemented.

Images