Message processing method, computer equipment, storage medium and program product

Abstract

The invention relates to the technical field of data storage, and discloses a message processing method, computer equipment, a storage medium and a program product, and the method comprises the steps: obtaining a data processing message to be sent to the rear end of a storage system and the current traffic feature information of each logic unit in the storage system; determining a message load index of each logic unit according to a target message type of the data processing message and the current traffic feature information of each logic unit; screening a target logic unit according to the message transmission priority represented by the message load index of each logic unit; and sending the data processing message to the rear end of the storage system based on the target logic unit. According to the method, the target logic unit is screened, and the data processing message is sent to the rear end of the storage system based on the target logic unit, so that the load of each logic unit is balanced, and the bandwidth utilization rate of the logic units is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a message processing method, a computer device, a storage medium and a program product. Background Art

[0002] With the advent of the big data era, servers have increasingly higher requirements for the performance and reliability of storage systems. NAND flash-based solid-state drives (SSDs) are widely used as server storage systems because they have higher reliability, better performance, and lower energy consumption.

[0003] In the related art, the SSD front end usually sends a message to the SSD back end through the Flash Translation Layer (FTL). The Flash Translation Layer receives the message sent by the SSD front end, uses polling to determine the logical unit that sends the message to the SSD back end, and uses the logical unit to send the message to the SSD back end.

[0004] However, since the messages sent by the SSD front end include multiple message types, the logical units consume different traffic when transmitting messages of different message types, resulting in unbalanced loads on each logical unit and reducing the bandwidth utilization of the logical units. Summary of the invention

[0005] In view of this, the present invention provides a message processing method, a computer device, a storage medium and a program product to solve the problem that the messages sent by the SSD front end include multiple message types, and the logical units consume different traffic when transmitting messages of different message types, resulting in unbalanced loads on each logical unit and reduced bandwidth utilization of the logical units.

[0006] In a first aspect, the present invention provides a message processing method, the method comprising:

[0007] Obtaining data processing messages to be sent to the back end of the storage system and current traffic characteristic information of each logical unit in the storage system;

[0008] Determining a message load index of each of the logical units according to a target message type of the data processing message and current traffic characteristic information of each of the logical units;

[0009] Filtering the target logical unit according to the message transmission priority represented by the message load indicator of each of the logical units;

[0010] Based on the target logical unit, the data processing message is sent to the storage system backend.

[0011] The message processing method provided in this embodiment obtains the data processing message to be sent to the back end of the storage system and the current traffic characteristic information of each logical unit in the storage system, determines the message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit, screens the target logical unit according to the message transmission priority represented by the message load index of each logical unit, and sends the data processing message to the back end of the storage system based on the target logical unit. By determining the message load index of each logical unit according to the current traffic characteristic information of each logical unit and the target message type of the data processing message, and then screening the target logical unit according to the message transmission priority represented by the message load index of each logical unit, and using the target logical unit to send the data processing message to the back end of the storage system, the message load of each logical unit is balanced, the bandwidth utilization of the logical unit is improved, and the efficiency of message processing is improved.

[0012] In an optional implementation, determining the message load index of each of the logical units according to the target message type of the data processing message and the current traffic characteristic information of each of the logical units includes:

[0013] Determining the pheromone concentration of each of the logical units according to the target message type of the data processing message and the current traffic characteristic information of each of the logical units;

[0014] According to the pheromone concentration of each of the logic units, a message load index of each of the logic units is determined.

[0015] The message processing method provided in this embodiment determines the pheromone concentration of the logical unit according to the target message type of the data processing message and the current traffic characteristic information of the logical unit, and then determines the message load index according to the pheromone concentration, which helps to ensure the reliability of the obtained message load index, thereby making the screened target logical units more accurate.

[0016] In an optional implementation, determining the pheromone concentration of each of the logical units according to the target message type of the data processing message and the current traffic characteristic information of each of the logical units includes:

[0017] The pheromone concentration of each of the logical units is determined based on the following formula:

[0018]

[0019] Among them, ρ represents the logical unit lun idx The pheromone concentration, idx represents the identification of the logical unit, Flow (lun idx ) represents the logical unit lun idxThe current traffic value of the logic unit, φ represents the rated transmission bandwidth of the logic unit, and loss(type) represents the logic unit lun idx Processing target message type data processing message data transmission overhead, type represents the target message type, α is the impact factor;

[0020] The current traffic characteristic information includes the current traffic value, the rated transmission bandwidth, and the data transmission overhead of processing data processing messages of the target message type.

[0021] The message processing method provided in this embodiment determines the pheromone concentration of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit, thereby ensuring the accuracy and reliability of the determined pheromone concentration of each logical unit.

[0022] In an optional embodiment, the method further includes:

[0023] The rated transmission bandwidth of the logic unit is determined based on the following formula:

[0024]

[0025] Among them, BW represents the theoretical total bandwidth value of the storage system, Sum(lun) represents the total number of logical units in the storage system, and Sum(bad_lun) represents the number of bad block logical units in the storage system.

[0026] The message processing method provided in this embodiment determines the rated transmission bandwidth of the logic unit by using the above formula, thereby ensuring the accuracy and reliability of the determined pheromone concentration of each logic unit.

[0027] In an optional embodiment, the method further includes:

[0028] Determine the logical unit lun based on the following formula idx Processing data of the target message type Data transmission overhead of processing messages:

[0029] loss(type)=Cnt(type)*t type *A BW

[0030] Among them, Cnt(type) represents the logical unit lun idx The number of data processed messages of the target message type processed within 1 second, t type Represents the logical unit lun idx The time it takes to actually process a data processing message of the target message type, A BW Represents the measured logic unit lun idx bandwidth per second.

[0031] The message processing method provided in this embodiment determines the data transmission overhead of the data processing message of the target message type processed by each logic unit by using the above formula, thereby ensuring the accuracy and reliability of the determined pheromone concentration of each logic unit.

[0032] In an optional implementation, determining the message load index of each of the logic units according to the pheromone concentration of each of the logic units includes:

[0033] According to the pheromone concentration of each of the logic units, selecting the candidate logic unit with the highest pheromone concentration;

[0034] In the case that the to-be-selected logical unit is unique, the pheromone concentration is used as the message load indicator;

[0035] In the case that the candidate logical unit is not unique, determining the heuristic factor of each candidate logical unit according to the pheromone concentration and influencing characteristic information of each candidate logical unit;

[0036] The heuristic factor is used as the message load indicator.

[0037] The message processing method provided in this embodiment ensures the accuracy and reliability of the message load indicators of each logical unit by using the information density as the message load indicator when the candidate logical unit with the highest pheromone concentration is unique, and using the heuristic factor as the message load indicator when the candidate logical unit is not unique, thereby ensuring the accuracy and reliability of the message load indicators of each logical unit, thereby ensuring the accuracy and reliability of the screened target logical unit.

[0038] In an optional implementation, determining the heuristic factor of each of the candidate logical units according to the pheromone concentration and the influencing characteristic information of each of the candidate logical units includes:

[0039] The heuristic factor of each of the candidate logic units is determined based on the following formula:

[0040]

[0041] Among them, h represents the selected logic unit lun idx The heuristic factor, c is the preset pheromone dependency factor, ρ is the candidate logic unit lun idx The pheromone concentration, M_cnt(lun idx ) is the selected logic unit lun idx The number of data processing messages successfully sent to the back end of the storage system, Flow_avg is the first flow average of each logical unit; the impact feature information includes the selected logical unit lun idxThe number of data processing messages successfully sent to the storage system backend and the first flow average of each of the logical units.

[0042] The message processing method provided in this embodiment uses the above formula to determine the heuristic factor of each logical unit to be selected when the logical unit to be selected is not unique, and uses the heuristic factor as the message load indicator, thereby ensuring the accuracy and reliability of the message load indicator of each logical unit, and further ensuring the accuracy and reliability of the screened target logical unit.

[0043] In an optional embodiment, the method further includes:

[0044] The first flow average value of each of the logic units is determined based on the following formula:

[0045]

[0046] Where k = Sum (lun) represents the total number of logical units in the storage system, Flow (i) = Flow (lun idx ) is the current flow value of the i-th logical unit.

[0047] The message processing method provided in this embodiment ensures the accuracy and reliability of the message load index of each logical unit, thereby ensuring the accuracy and reliability of the screened target logical unit.

[0048] In an optional implementation, screening the target logical unit according to the message transmission priority represented by the message load indicator of each logical unit includes:

[0049] In the case where the message load indicator is the pheromone concentration, the candidate logical unit with the highest pheromone concentration is selected as the target logical unit;

[0050] In the case where the message load indicator is an inspiration factor, the candidate logical unit with the highest inspiration factor is used as the target logical unit.

[0051] The message processing method provided in this embodiment selects the target logical unit according to the message transmission priority represented by the message load indicator, thereby ensuring the accuracy of the selected target logical unit.

[0052] In an optional implementation, before sending the data processing message to the storage system backend based on the target logical unit, the method further includes:

[0053] Obtaining the latest flow value of each of the logical units;

[0054] Determine a second flow average value of each of the logical units according to the latest flow value of each of the logical units;

[0055] Determining whether the target logical unit passes verification according to the first traffic average and the second traffic average;

[0056] In the case where the first traffic average is the same as the second traffic average, determining that the target logical unit passes the verification, and executing the step of sending the data processing message to the storage system backend based on the target logical unit;

[0057] When the first traffic mean is different from the second traffic mean, it is determined that the target logical unit has failed the verification, and the process returns to the step of determining the message load index of each logical unit based on the target message type of the data processing message and the current traffic characteristic information of each logical unit, so as to re-screen the target logical unit until the first traffic mean is the same as the second traffic mean.

[0058] The message processing method provided in this embodiment verifies the target logical unit according to whether the first traffic mean and the second traffic mean are the same, ensuring that the screened target logical unit passes the verification, thereby ensuring the accuracy of the screened target logical unit, thereby balancing the load on each logical unit and improving the bandwidth utilization of each logical unit.

[0059] In an optional embodiment, the method further includes:

[0060] When it is determined that the target logical unit fails the verification, flow round robin processing is performed on each of the logical units according to the screening time of the target logical unit and the target message type, so as to decrement the current flow value of each of the logical units.

[0061] The message processing method provided in this embodiment performs traffic round-robin processing on each logical unit according to the screening time of the target logical unit and the target message type, so as to reduce the current traffic value of each logical unit, thereby ensuring the accuracy of the current traffic value of each logical unit.

[0062] In an optional implementation manner, the current traffic characteristic information includes a current traffic value, and obtaining the current traffic characteristic information of each logical unit in the storage system includes:

[0063] For any of the logical units, the current flow value of the logical unit is determined according to the message type and message sending time of the data processing message currently sent by the logical unit to the back end of the storage system.

[0064] The message processing method provided in this embodiment determines the current flow value of the logical unit through the message type and message sending time of the data processing message currently sent by the logical unit to the back end of the storage system, thereby ensuring the accuracy of the current flow value of the logical unit.

[0065] In an optional implementation, the sending the data processing message to the storage system backend based on the target logical unit includes:

[0066] In the case where the target message type is a write message or an erase message, based on the target logical unit, sending the data processing message as a to-be-processed message to a preset write message interaction queue or a preset erase message interaction queue;

[0067] According to preset time constraints, the pending messages in the preset write message interaction queue or the preset erase message interaction queue are sent to the storage system backend, so that the storage system backend processes the pending messages.

[0068] The message processing method provided in this embodiment sends the pending messages in the preset write message interaction queue or the preset erase message interaction queue to the storage system back end according to the preset time constraint, so that the storage system back end processes the pending messages, so that the pending messages of different message types can be processed evenly, so that the storage back end resources can achieve the fastest circulation.

[0069] In an optional implementation, sending the to-be-processed messages in the preset write message interaction queue or the preset erase message interaction queue to the storage system backend according to the preset time constraint condition includes:

[0070] Sending the pending messages in the preset write message interaction queue to the storage system backend in order according to a first preset time interval;

[0071] According to a second preset time interval, the to-be-processed messages in the preset message interaction queue are sent to the storage system backend in order.

[0072] The message processing method provided in this embodiment sends the pending messages in the interaction queue to the storage system backend according to different preset time intervals for different interaction queues. The storage system backend processes the pending messages in the order in which the pending messages are received, so that the pending messages of different message types can be processed evenly.

[0073] In an optional embodiment, the method further includes:

[0074] The first preset time interval is determined based on the following formula:

[0075] t wr =T ftl / wr_cnt-t nfc_wr

[0076] Among them, t wrrepresents the first preset time interval, wr_cnt represents the remaining write data amount of the current data block, T ftl Indicates the time it takes for the flash translation layer to write a data block, t nfc_wr Indicates the time it takes for the storage system backend to process a write message.

[0077] The message processing method provided in this embodiment sends the pending messages in the preset write message interaction queue to the storage system backend in sequence according to the first preset time interval. The storage system backend processes the pending messages in the order in which the pending messages are received, so that the pending messages of different message types can be processed evenly.

[0078] In an optional embodiment, the method comprises:

[0079] The second preset time interval is determined based on the following formula:

[0080] t ers =T ftl / RAID_SIZE-t nfc_ers

[0081] Among them, t ers represents the second preset time interval, RAID_SIZE represents the number of parallel logic units, T ftl Indicates the time it takes for the flash translation layer to write a data block, t nfc_ers Indicates the time it takes for the storage system backend to process a delete message.

[0082] The message processing method provided in this embodiment sends the pending messages in the preset message interaction queue to the storage system back end in sequence according to the second preset time interval. The storage system back end processes the pending messages in the order in which the pending messages are received, so that the pending messages of different message types can be processed evenly.

[0083] In a second aspect, the present invention provides a message processing device, the device comprising:

[0084] An acquisition module is used to acquire data processing messages to be sent to the back end of the storage system and current traffic characteristic information of each logical unit in the storage system;

[0085] A determination module, used to determine the message load index of each of the logical units according to the target message type of the data processing message and the current traffic characteristic information of each of the logical units;

[0086] A screening module, used for screening target logical units according to the message transmission priority represented by the message load index of each of the logical units;

[0087] A sending module is used to send the data processing message to the storage system backend based on the target logical unit.

[0088] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the message processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0089] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the message processing method of the first aspect or any corresponding embodiment thereof.

[0090] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the message processing method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0092] Figure 1 is a flow chart of a message processing method according to an embodiment of the present invention;

[0093] Figure 2 is a flow chart of another message processing method according to an embodiment of the present invention;

[0094] Figure 3 is a schematic diagram of the process of the ant colony optimization algorithm according to an embodiment of the present invention;

[0095] Figure 4 is a flow chart of a message processing method according to an embodiment of the present invention;

[0096] Figure 5 is a schematic diagram of a storage system according to an embodiment of the present invention;

[0097] Figure 6 is a schematic diagram of message processing by a storage system according to an embodiment of the present invention;

[0098] Figure 7 is a structural block diagram of a message processing device according to an embodiment of the present invention;

[0099] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0100] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0101] With the rapid development of Internet technology, human society has entered an era of big data where the amount of data is growing exponentially. The advent of the big data era requires more support from storage technology, and various big data applications have posed greater challenges to the performance and reliability of storage systems. For example, servers have increasingly higher requirements for the performance and reliability of storage systems.

[0102] With the emergence and widespread use of NAND flash memory, solid-state drives based on NAND flash memory have the characteristics of higher reliability, better performance, lower energy consumption, etc. Therefore, solid-state drives based on NAND flash memory are widely used as storage systems for servers.

[0103] To ensure the input / output (I / O) performance of the SSD, FTL usually processes messages in parallel by logical unit number (LUN). Taking the RAID 5 level of the Redundant Array of Independent Disks (RAID) as an example, the message processing in parallel by logical unit is explained: the data blocks with the same number on 32 LUNs are collected as a super block to process the message.

[0104] When the server needs to read, write, or erase data, it sends the corresponding message to the SSD front end, which then sends the message to the SSD back end through the Flash Translation Layer (FTL). The FTL receives the message from the SSD front end, uses polling to determine the LUN that sends the message to the SSD back end from among the 32 LUNs, and uses the LUN to send the message to the SSD back end for processing.

[0105] However, since the messages sent by the SSD front end include multiple message types, the traffic consumed when LUN transmits messages of different message types is different, resulting in unbalanced loads on each LUN and reducing the bandwidth utilization of the logical unit.

[0106] An embodiment of the present invention provides a message processing method, which determines the message load index of each logical unit through the target message type of the data processing message to be sent to the back end of the storage system and the current traffic characteristic information of each logical unit, and selects the target logical unit according to the message transmission priority represented by the message load index of each logical unit. The target logical unit is used to send the data processing message to the back end of the storage system to achieve the effect of balancing the load of each logical unit and improving the bandwidth utilization of the logical unit.

[0107] According to an embodiment of the present invention, a message processing method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0108] In this embodiment, a message processing method is provided, which can be used in a mobile terminal, such as a server, etc. Figure 1 is a flow chart of a message processing method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0109] Step S101, obtaining a data processing message to be sent to a storage system backend and current traffic characteristic information of each logical unit in the storage system.

[0110] The server sends a data processing message to the storage system front end of the server. The storage system front end receives the data processing message and sends the data processing message to the FTL for processing. The FTL sends the data processing message to the storage system back end.

[0111] The FTL obtains the data processing message to be sent to the backend of the storage system and obtains the current traffic characteristic information of each logical unit in the storage system. Among them, the logical unit is the smallest independent unit of the flash memory that executes commands and reports its own status.

[0112] Among them, the storage system in this embodiment is a solid-state hard disk based on NAND flash memory. Due to the characteristic that NAND flash memory must be "erase before writing", when the message type of the data processing message is a write message, a data block erase message will be generated.

[0113] Therefore, the message type of the data processing message to be sent to the storage system backend may be a write message, a read message, or an erase message.

[0114] Step S102: determining a message load index of each logical unit according to a target message type of the data processing message and current traffic characteristic information of each logical unit.

[0115] Among them, after obtaining the data processing message to be sent to the back end of the storage system and the current traffic characteristic information of each logical unit in the storage system, FTL determines the message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit.

[0116] Specifically, the FTL includes a load balancer and a flow controller. The flow controller records the message processing status of each logical unit, and uses the flow controller to obtain the current flow characteristic information of each logical unit in the storage system. The load balancer is used to control the message sending on each logical unit, and uses the load balancer to determine the message load index of each logical unit according to the target message type of the data processing message and the current flow characteristic information of each logical unit.

[0117] Step S103 , screening target logical units according to the message transmission priority represented by the message load index of each logical unit.

[0118] After determining the message load index of each logical unit, the load balancer selects the target logical unit according to the message transmission priority represented by the message load index of each logical unit.

[0119] It can be understood that the target logical unit is the logical unit with the highest message transmission priority represented by the message load indicator.

[0120] Step S104: sending the data processing message to the storage system backend based on the target logical unit.

[0121] After the target logical unit is screened out, the FTL sends the data processing message to be sent to the storage system backend to the storage system backend based on the target logical unit.

[0122] The message processing method provided in this embodiment obtains the data processing message to be sent to the back end of the storage system and the current traffic characteristic information of each logical unit in the storage system, determines the message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit, screens the target logical unit according to the message transmission priority represented by the message load index of each logical unit, and sends the data processing message to the back end of the storage system based on the target logical unit. By determining the message load index of each logical unit according to the current traffic characteristic information of each logical unit and the target message type of the data processing message, and then screening the target logical unit according to the message transmission priority represented by the message load index of each logical unit, and using the target logical unit to send the data processing message to the back end of the storage system, the message load of each logical unit is balanced, the bandwidth utilization of the logical unit is improved, and the efficiency of message processing is improved.

[0123] In this embodiment, a message processing method is provided, which can be used in a mobile terminal, such as a server, etc. Figure 2 is a flow chart of a message processing method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0124] Step S201, obtain the data processing message to be sent to the storage system backend and the current traffic characteristic information of each logical unit in the storage system. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0125] Step S202: determining a message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit.

[0126] Specifically, the above step S202 includes:

[0127] Step S2021, determining the pheromone concentration of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit.

[0128] Among them, the load balancer determines the message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit. Specifically, the pheromone concentration of each logical unit is obtained through the ant colony optimization algorithm, and then the message load index of each logical unit is determined according to the pheromone concentration.

[0129] It should be noted that the pheromone concentration of the logic unit is the bandwidth idle rate on the logic unit.

[0130] Figure 3 is a flow chart of the ant colony optimization algorithm according to an embodiment of the present invention. Figure 3 As shown, firstly, the population and parameters are initialized, which may specifically include pheromone initialization and population size initialization.

[0131] Then, the selection probability of each logical unit is calculated, wherein the selection probability of each logical unit is characterized by the pheromone concentration of each logical unit. Specifically, the pheromone concentration of each logical unit is determined according to the target message type of the data processing message and the current traffic characteristic information of each logical unit, and then the pheromone concentration of each logical unit is determined as the selection probability of each logical unit.

[0132] Step S2022: determining the message load index of each logical unit according to the pheromone concentration of each logical unit.

[0133] After the pheromone concentration of each logical unit is determined, the message load index of each logical unit is determined according to the pheromone concentration of each logical unit.

[0134] Step S203: Filter the target logical unit according to the message transmission priority represented by the message load index of each logical unit. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0135] Step S204: Based on the target logical unit, the data processing message is sent to the storage system backend. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0136] The message processing method provided in this embodiment determines the message load index of each logical unit according to the current traffic characteristic information of each logical unit and the target message type of the data processing message, and then screens the target logical unit according to the message transmission priority represented by the message load index of each logical unit, and uses the target logical unit to send the data processing message to the back end of the storage system, so that the message load of each logical unit is balanced, the bandwidth utilization of the logical unit is improved, and the efficiency of message processing is improved.

[0137] In some optional implementations, the above step S2021 includes:

[0138] Step a1, determining the pheromone concentration of each logical unit based on the following formula:

[0139]

[0140] Where φ represents the logic unit lun idx The pheromone concentration, idx represents the identification of the logical unit, Flow (lun idx ) represents the logical unit lun idxThe current traffic value of the logic unit, φ represents the rated transmission bandwidth of the logic unit, and loss(type) represents the logic unit lun idx Processing target message type data processing message data transmission overhead, type represents the target message type, α is the impact factor.

[0141] The current traffic characteristic information includes the current traffic value, the rated transmission bandwidth, and the data transmission overhead of the data processing message of the processing target message type.

[0142] It should be noted that the logical unit lun can be directly obtained from the traffic controller idx The impact factor is an adjustable parameter, which specifically indicates the impact factor of pheromones on ants and is adjusted by technicians based on the NAND flash memory.

[0143] Type indicates the target message type, which may include write message, read message and erase message. Table 1 shows the type definitions corresponding to different target message types.

[0144] Table 1

[0145] Message Type type definition Write a message M_WR Read Message M_RD Wipe message M_ERS

[0146] In some optional implementations, the message processing method further includes:

[0147] Step b1, determining the rated transmission bandwidth of the logic unit based on the following formula:

[0148]

[0149] Among them, BW represents the theoretical total bandwidth value of the storage system, Sum(lun) represents the total number of logical units in the storage system, and Sum(bad_lun) represents the number of bad block logical units in the storage system.

[0150] It should be noted that the rated transmission bandwidth of each logical unit in the storage system is the same.

[0151] A logic unit can have multiple planes. A bad block logic unit refers to a logic unit with a preset number of bad blocks on any plane. For example, if there is one bad block on any plane of the logic unit, the logic unit is a bad block logic unit. Alternatively, if there are three bad blocks on any plane of the logic unit, the logic unit is a bad block logic unit. The specific value of the preset number is determined by the technician and is not specifically limited here.

[0152] In some optional implementations, the message processing method further includes:

[0153] Step c1, determine the logical unit lun based on the following formulaidx Processing data of the target message type Data transmission overhead of processing messages:

[0154] loss(type)=Cnt(type)*t type *A BW

[0155] Among them, Cnt(type) represents the logical unit lun idx The number of data processed messages of the target message type processed within 1 second, t type Represents logical unit lun idx The time it takes to actually process a data processing message of the target message type, A BW Represents the measured logic unit lun idx bandwidth per second.

[0156] In some optional implementations, the above step S2022 includes:

[0157] Step d1, based on the pheromone concentration of each logic unit, select the candidate logic unit with the highest pheromone concentration.

[0158] After the pheromone concentration of each logic unit is determined, the candidate logic unit with the highest pheromone concentration is screened out from all the logic units according to the pheromone concentration of each logic unit.

[0159] Step d2: When there is only one logical unit to be selected, the pheromone concentration is used as the message load indicator.

[0160] The uniqueness of the selected logical unit indicates that there is only one logical unit with the highest pheromone concentration. When the selected logical unit is unique, the pheromone concentration of the selected logical unit is used as the message load indicator of the selected logical unit.

[0161] Step d3, when the candidate logical unit is not unique, the heuristic factor of each candidate logical unit is determined according to the pheromone concentration and the influencing characteristic information of each candidate logical unit.

[0162] Among them, the non-uniqueness of the selected logical unit indicates that there are multiple logical units with the highest pheromone concentration. In the case that the selected logical unit is not unique, in order to avoid the abnormal situation that the ants rely on the pheromone concentration when the pheromone concentration is the same, this embodiment uses the heuristic factor to reflect the relative importance of the pheromone on the path to the ants when selecting the path. Specifically, the heuristic factor of each selected logical unit is determined according to the pheromone concentration and the influencing characteristic information of each selected logical unit.

[0163] Step d4, using the heuristic factor as a message load indicator.

[0164] When the candidate logical unit is not unique, the heuristic factor of each candidate logical unit is used as the message load index of each candidate logical unit.

[0165] In some optional implementations, the above step d3 includes:

[0166] Step d31, determining the heuristic factor of each candidate logic unit based on the following formula:

[0167]

[0168] Among them, h represents the selected logic unit lun idx The heuristic factor, c is the preset pheromone dependency factor, ρ is the candidate logic unit lun idx The pheromone concentration, M_cnt(lun idx ) is the selected logic unit lun idx The number of data processing messages successfully sent to the backend of the storage system. Flow_avg is the average value of the first flow of each logical unit.

[0169] The impact feature information includes the candidate logical unit lun idx The number of data processing messages successfully sent to the storage system backend and the first traffic mean of each logical unit.

[0170] It should be noted that the heuristic factor represents the candidate logic unit lun idx The influence of the above pheromone concentration on the ants. The preset pheromone dependence factor is the degree of ant dependence on pheromones, which is an adjustable parameter and is set by the technician.

[0171] In some optional implementations, the message processing method further includes:

[0172] Step e1, determining a first flow average value of each logical unit based on the following formula:

[0173]

[0174] Where k = Sum (lun) represents the total number of logical units in the storage system, Flow (i) = Flow (lun idx ) is the current flow value of the i-th logical unit.

[0175] In some optional implementations, the above step S203 includes:

[0176] Step f1, when the message load indicator is the pheromone concentration, the candidate logic unit with the highest pheromone concentration is used as the target logic unit.

[0177] The message load indicator is the pheromone concentration, which indicates that there is only one candidate logical unit with the highest pheromone concentration, and the only candidate logical unit is used as the target logical unit.

[0178] Step f2: when the message load index is the heuristic factor, the candidate logical unit with the highest heuristic factor is used as the target logical unit.

[0179] The message load index is a heuristic factor indicating that there are multiple candidate logical units with the highest pheromone concentration, and the candidate logical unit with the highest heuristic factor among the multiple candidate logical units is used as the target logical unit.

[0180] In some optional implementations, before sending the data processing message to the storage system backend based on the target logical unit, the message processing method further includes:

[0181] Step g1, obtaining the latest flow value of each logical unit.

[0182] Among them, since SSD operation requests often have concurrency, such as read and write operations after the garbage collection mechanism (GC) is started, mixed read and write test conditions, etc., it may cause the pheromone to be updated during the optimization process. Therefore, after screening the target logical unit, the target logical unit needs to be verified.

[0183] After the target logical units are screened out, the latest flow value of each logical unit is obtained.

[0184] Step g2: determining a second flow average value of each logical unit according to the latest flow value of each logical unit.

[0185] After obtaining the latest flow value of each logical unit, the second flow average of each logical unit is determined according to the latest flow value of each logical unit. It can be understood that the method of determining the second flow average is similar to the method of determining the first flow average, except that the current flow value on each logical unit needs to be modified to the latest flow value of each logical unit to determine the second flow average.

[0186] It should be noted that Figure 3 The updated pheromone in is to obtain the second flow mean of each logical unit.

[0187] Step g3: judging whether the target logical unit passes the verification according to the first traffic average and the second traffic average.

[0188] like Figure 3As shown, the target logical unit is the optimal LUN. It is determined whether the optimal LUN is found. If the optimal LUN is found, it is also necessary to determine whether the optimal LUN passes verification. Specifically, it is determined whether the target logical unit passes verification based on the first flow average and the second flow average.

[0189] Step g4, when the first traffic average value is the same as the second traffic average value, it is determined that the target logical unit has passed the verification, and a step of sending a data processing message to the storage system backend based on the target logical unit is executed.

[0190] Among them, the first traffic mean is the same as the second traffic mean, indicating that concurrent requests do not affect the criticality of the link. The target logical unit passes the verification and continues to execute the step of sending the data processing message to the back end of the storage system based on the target logical unit. The criticality of the link lies in the stability of the storage system and the efficiency of message processing.

[0191] It should be noted that the optimal LUN passes the verification, the optimal LUN is output, and the step of sending the data processing message to the storage system backend based on the optimal LUN is continued.

[0192] Step g5, when the first traffic mean is different from the second traffic mean, it is determined that the target logical unit has not passed the verification, and the process returns to the step of determining the message load index of each logical unit based on the target message type of the data processing message and the current traffic characteristic information of each logical unit, so as to re-screen the target logical unit until the first traffic mean is the same as the second traffic mean.

[0193] Among them, the first traffic mean is different from the second traffic mean, indicating that the concurrent request has affected the criticality of the link, and the target logical unit has failed the verification. The process returns to the step of determining the message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit, so as to re-screen the target logical unit until the first traffic mean is the same as the second traffic mean.

[0194] like Figure 3 As shown, the optimal LUN fails to pass the verification, the current population is all dead, the optimization fails, the parameter information of the ant colony optimization algorithm is updated, and a new round of optimization is started until the first traffic mean is the same as the second traffic mean, ensuring that the storage can achieve load balancing. Among them, updating the parameter information of the ant colony optimization algorithm can include updating the current traffic feature information of each logical unit to the latest traffic feature information.

[0195] In some optional implementations, the message processing method further includes:

[0196] Step h1, when it is determined that the target logical unit has not passed the verification, traffic round robin processing is performed on each logical unit according to the screening time of the target logical unit and the target message type to reduce the current traffic value of each logical unit.

[0197] When it is determined that the target logical unit has not passed the verification, the traffic controller performs traffic round-robin processing on each logical unit according to the screening time of the target logical unit and the target message type. The traffic round-robin processing is a blank LUN processing, and the blank LUN processing is achieved by subtracting the blank traffic unit from the current traffic value of each logical unit.

[0198] It should be noted that the value of the by-pass flow unit is determined by the target logic unit screening time and the target message type. The target logic unit screening time is the time consumed to screen out the target logic unit.

[0199] Further, when it is determined that the target logical unit has passed the verification, flow round robin processing is performed on each logical unit according to the screening time of the target logical unit and the target message type, so as to reduce the current flow value of each logical unit.

[0200] In some optional implementations, the current flow characteristic information includes a current flow value, and the above step S201 includes:

[0201] Step i1, for any logical unit, according to the message type and message sending time of the data processing message currently sent by the logical unit to the back end of the storage system, determine the current flow value of the logical unit.

[0202] For any logical unit, the flow controller determines the current flow value of the logical unit according to the message type and message sending time of the data processing message currently successfully sent by the logical unit to the back end of the storage system.

[0203] Different message types correspond to different traffic units, and Table 2 shows the definitions of traffic units corresponding to different message types. It should be noted that the type corresponding to the bye LUN processing is bye.

[0204] Table 2

[0205] type Flow Unit Definition Read Message RD_FLOW_UNIT Write a message WR_FLOW_UNIT Wipe message ERS_FLOW_UNIT Byes EMPTY_FLOW_UNIT

[0206] The specific values ​​of the traffic units corresponding to different message types are determined according to the time taken from the FTL successfully sending the data processing message of the message type to the storage system backend to the storage system backend processing and returning the message to the FTL.

[0207] It is understandable that the message sending time of different message types is generally different. The traffic controller determines the current traffic value of the logical unit based on the message type and message sending time of the data processing message currently successfully sent by the logical unit to the storage system backend.

[0208] It should be noted that the flow unit corresponding to the bye is the time it takes for the central processor to poll all the main functions of the FTL, that is, the time it takes for the FTL to screen the target logic unit.

[0209] It should be noted that the flow controller accumulates different flow units according to the message type and message sending time of the data processing message currently successfully sent by the logical unit to the back end of the storage system to obtain the current flow value of the logical unit. The flow controller includes the parameters shown in Table 3.

[0210] Table 3

[0211] Parameter Definition Parameter Description Flow Current LUN traffic value M_cnt Total number of messages processed on the current LUN φ Rated transmission bandwidth value on the current LUN W_cnt Number of write messages being processed on the current LUN R_cnt Number of read messages being processed on the current LUN ERS_cnt The number of erase messages being processed on the current LUN

[0212] It should be noted that, after the logic unit successfully sends the data processing message to the storage system backend, the flow controller accumulates and updates the current flow value of the logic unit according to the flow unit corresponding to the message type of the data processing message.

[0213] When the storage system backend completes processing of the data processing message and returns a processing completion message, the flow controller decrements and updates the current flow value of the logical unit according to the flow unit corresponding to the message type of the data processing message.

[0214] It should be noted that for any logical unit, if all the data blocks on the plane of the logical unit are bad blocks, the flow value setting process of the logical unit needs to be performed. The flow value setting process includes setting the flow value of the logical unit to -1, and the logical unit does not participate in the operation of the by-LUN processing.

[0215] In some optional implementations, the above step S204 includes:

[0216] Step j1, when the target message type is a write message or an erase message, based on the target logical unit, the data processing message is sent as a to-be-processed message to a preset write message interaction queue or a preset erase message interaction queue.

[0217] Wherein, when the target logical unit passes the verification, a step of sending the data processing message as a to-be-processed message to a preset write message interaction queue or a preset erase message interaction queue is executed based on the target logical unit.

[0218] Figure 4 is a flow chart of a message processing method according to an embodiment of the present invention. Figure 4 As shown, it is determined whether the ant colony optimization algorithm has reached the end standard, wherein the end standard may be that the first traffic mean and the second traffic mean are the same. It can be understood that the ant colony optimization algorithm reaches the end standard and continues to execute the step of sending the data processing message as a to-be-processed message to a preset write message interaction queue or a preset erase message interaction queue based on the target logical unit.

[0219] When the ant colony optimization algorithm fails to reach the end criterion, the target logical unit fails to pass the verification, and the algorithm returns to the step of determining the message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit, so as to re-screen the target logical unit until the first traffic mean is the same as the second traffic mean, that is, the end criterion of the ant colony optimization algorithm is reached.

[0220] When it is determined that the target logical unit fails the verification, a blank LUN process is performed on each logical unit according to the screening time of the target logical unit and the target message type, so as to decrease the current flow value of each logical unit.

[0221] Wherein, when the target message type is a write message or an erase message, based on the target logical unit, the data processing message is sent as a pending message to a preset write message interaction queue or a preset erase message interaction queue, specifically including: when the target message type is a write message, determine whether there is a suspension, that is, determine whether there is a suspended message on the target logical unit, if there is no suspended message on the target logical unit, then use the target logical unit to send the data processing message as a pending message to the preset write message interaction queue, corresponding to the current information sending. Wherein, the pending message is also called the current message, and the suspended message is the message being processed.

[0222] It should be noted that, when it is determined that the target logical unit has passed the verification, the flow round-robin processing is performed on each logical unit according to the screening time of the target logical unit and the target message type to reduce the current flow value of each logical unit, including:

[0223] When it is determined that the target logical unit passes the verification, flow round robin processing is performed on other logical units except the target logical unit according to the screening time of the target logical unit and the target message type, so as to reduce the current flow values ​​of other logical units.

[0224] When it is determined that the target logical unit has passed the verification and there is no pending message on the target logical unit, after sending the message to be processed to the corresponding interaction queue, the target logical unit is processed by blanking LUN according to the screening time and target message type of the target logical unit to reduce the current traffic value of the target logical unit.

[0225] If there is a pending message on the target logical unit, the pending message on the target logical unit is sent to the corresponding interaction queue according to the message type of the pending message. Exemplarily, if the message type of the pending message is a write message, the pending message is sent to the preset write message interaction queue. It can be understood that the pending message is also a pending message.

[0226] After the suspended message on the target logical unit is sent to the corresponding interaction queue according to the message type of the suspended message, the data processing message is suspended as a suspended message on the target logical unit, that is, the current message is suspended, waiting for the target logical unit to process the suspended message, that is, sending the suspended message to the corresponding interaction queue.

[0227] It should be noted that when it is determined that the target logical unit has passed the verification and there is a suspended message on the target logical unit, after the current message is suspended, the target logical unit is processed with LUN blanking according to the screening time of the target logical unit and the target message type to reduce the current traffic value of the target logical unit.

[0228] When the target message type is a wipe message, determine whether there is a suspension on the target logical unit, that is, determine whether there is a suspended message on the target logical unit. If there is no suspended message on the target logical unit, use the target logical unit to send the data processing message as a to-be-processed message to the preset wipe message interaction queue.

[0229] If there is a pending message on the target logical unit, the pending message on the target logical unit is sent to the corresponding interaction queue according to the message type of the pending message. Exemplarily, if the message type of the pending message is a wipe message, the pending message is sent to a preset wipe message interaction queue.

[0230] After the suspended message on the target logical unit is sent to the corresponding interaction queue according to the message type of the suspended message, the data processing message is suspended as a suspended message on the target logical unit, that is, the current message is suspended, waiting for the target logical unit to process the suspended message, that is, sending the suspended message to the corresponding interaction queue.

[0231] It is understandable that there may be two situations when a suspend message is sent to a corresponding interaction queue, namely, failure to send the suspend message and success to send the suspend message. The failure to send the suspend message means that the suspend message fails to be sent to the corresponding interaction queue, and the success to send the suspend message means that the suspend message is successfully sent to the corresponding interaction queue.

[0232] Step j2: according to preset time constraints, the pending messages in the preset write message interaction queue or the preset erase message interaction queue are sent to the storage system backend, so that the storage system backend processes the pending messages.

[0233] Among them, the pending messages in the preset write message interaction queue or the preset erase message interaction queue are sent to the storage system back end according to the preset time constraint conditions, so that the storage system back end can process the pending messages in the order of the received pending messages, ensuring that the pending messages of each message type can be processed evenly.

[0234] It should be noted that in the related art, due to the characteristics of NAND flash memory, the processing granularity of read messages, write messages and erase messages is performed in units of page, wl and block respectively, which makes the processing time of a read message, write message and erase message processed by the storage system backend different. In addition, the storage system backend usually processes the data processing messages on each LUN in a polling manner, and the order of data processing messages cannot be guaranteed, which leads to an unbalanced message load on each LUN, resulting in the blockage and slowdown of the flow of some scarce resources of FTL.

[0235] Due to the lack of consideration of the actual load on each LUN, there are obvious differences in the load on each LUN, resulting in low bandwidth utilization. For example, in order to speed up GC processing, faster static random access memory (SRAM) resources will be used as buffers for data migration. Uneven load will lead to untimely release of SRAM buffer resources, seriously affecting the I / O performance of the SSD.

[0236] This embodiment sends the pending messages in the preset write message interaction queue or the preset erase message interaction queue to the storage system backend according to the preset time constraint condition, so that the storage system backend processes the pending messages. This achieves balanced processing of various messages, and the resources such as SRAM buffer achieve the fastest flow, ensuring that the I / O performance of the SSD reaches saturation.

[0237] In some optional embodiments, when the number of times the step of determining the message load index of each logical unit based on the target message type of the data processing message and the current traffic characteristic information of each logical unit reaches a preset number and the first traffic mean is still different from the second traffic mean, it is determined that the optimal LUN has not been found and all populations have died, and the step of sending the data processing message to the storage system back end based on the latest target logical unit is executed.

[0238] The latest target logic unit is the target logic unit selected when the number of optimization times of the ant optimization algorithm reaches a preset number.

[0239] That is to say, when the ant search algorithm has searched for a preset number of times and the selected target logical unit still fails to pass the verification, the search will no longer continue, and the latest selected target logical unit will be directly used to send the data processing message to the storage system backend.

[0240] It can be understood that the detailed description of sending the data processing message to the storage system backend based on the latest target logical unit is similar to the description of sending the data processing message to the storage system backend based on the target logical unit, and will not be repeated here.

[0241] In some optional implementations, the above step j2 includes:

[0242] Step j21, sending the to-be-processed messages in the preset write message interaction queue to the storage system backend in order according to the first preset time interval.

[0243] The FTL sends each to-be-processed message to the storage system backend at a first preset time interval and in the order of the to-be-processed messages in the preset write message interaction queue.

[0244] It is understandable that the storage system backend processes the pending messages in the order in which they are received, and the pending messages that have not reached the sending time are placed in a preset write message interaction queue.

[0245] Step j22: sending the pending messages in the preset message interaction queue to the storage system backend in order according to the second preset time interval.

[0246] The FTL sends each to-be-processed message to the storage system backend at a second preset time interval and in the order of the to-be-processed messages in the preset message interaction queue.

[0247] It is understandable that the storage system backend processes the pending messages in the order in which they are received, and the pending messages that have not reached the sending time are placed in a preset message interaction queue.

[0248] It should be noted that, when the target message type is a read message, based on the target logical unit, the data processing message is sent to a preset read message interaction queue as a to-be-processed message.

[0249] As long as the storage system backend is not processing pending messages in the preset write message interaction queue or the preset erase message interaction queue, it will process pending messages in the preset read message interaction queue.

[0250] In some optional implementations, the message processing method further includes:

[0251] Step k1, determining a first preset time interval based on the following formula:

[0252] t wr =T ftl / wr_cnt-t nfc_wr

[0253] Among them, t wr represents the first preset time interval, wr_cnt represents the remaining write data amount of the current data block, T ftl Indicates the time it takes for the flash translation layer to write a data block, t nfc_wr Indicates the time it takes for the storage system backend to process a write message.

[0254] It should be noted that the remaining write data amount of the current data block is the write data amount required to fill the current data block, that is, how many WLS are left to write data after the bad blocks on the current data block are removed.

[0255] In some optional implementations, the message processing method further includes:

[0256] Step L1, determining a second preset time interval based on the following formula:

[0257] t ers =T ftl / RAID_SIZE-t nfc_ers

[0258] Among them, t ers represents the second preset time interval, RAID_SIZE represents the number of parallel logic units, T ft1 Indicates the time it takes for the flash translation layer to write a data block, t nfc_ers Indicates the time it takes for the storage system backend to process a delete message.

[0259] It should be noted that, when the level of the Redundant Array of Independent Disks (RAID) is RAID5, the number of parallel logical units is 32.

[0260] The message processing method provided in this embodiment sends the pending messages in the preset write message interaction queue or the preset erase message interaction queue to the storage system backend according to the preset time constraint condition, so that the storage system backend processes the pending messages. It realizes balanced processing of various messages, and the resources such as SRAM buffer achieve the fastest flow, ensuring that the I / O performance of the SSD reaches saturation.

[0261] The storage system used in the above embodiments is as follows: Figure 5As shown, the storage system includes a storage system front end, an FTL and a storage system back end, wherein the storage system front end is used for HOST (host side) command parsing and completing message processing.

[0262] HOST command parsing includes parsing the command issued by the server host to determine the data processing message to be sent to the storage system backend and the target message type of the data processing message.

[0263] Completing the message processing includes obtaining in real time the processing status of the data processing message by the storage system backend to determine whether the storage system backend has completed the message processing.

[0264] FTL includes a load balancer and a flow controller, wherein the flow controller records the message processing status of each logical unit, and uses the flow controller to obtain the current flow characteristic information of each logical unit in the storage system. The load balancer is used to control the message sending on each logical unit, and the load balancer is used to determine the message load index of each logical unit according to the target message type of the data processing message and the current flow characteristic information of each logical unit, and the target logical unit is screened according to the message transmission priority represented by the message load index of each logical unit.

[0265] In other words, the load balancer is used to process the front-end read and write commands as well as the erase commands generated during block consumption.

[0266] The storage system backend includes a message balancing processor and NAND flash memory.

[0267] The message balancing processor is used to control the number and order of processing each type of message according to the different processing times of each type of message, so that each type of message is processed evenly. Specifically used to implement the above step j2 and the steps described in step j2.

[0268] Figure 6 FIG. 1 is a schematic diagram of a storage system performing message processing according to an embodiment of the present invention. Figure 6 As shown, the storage system uses a message processor, a load balancer, a flow controller and a message balance processor to process messages. Among them, the message processor is used to store and execute a computer program that implements the above message processing method, the load balancer is used to control the message sending on each logical unit, and the flow controller is used to record the status of the data processing message being processed on each logical unit. The message balance processor is used to control the number and order of processing each type of message according to the different processing time of each type of message, so that various types of messages are processed evenly.

[0269] The message processing method provided in this embodiment screens the target logical unit and sends the data processing message to the back end of the storage system based on the target logical unit, so that the load of each logical unit is balanced and the bandwidth utilization of the logical unit is improved. According to the preset time constraint, the pending messages in the preset write message interaction queue or the preset erase message interaction queue are sent to the back end of the storage system so that the back end of the storage system processes the pending messages. Balanced processing of various messages is achieved, and resources such as SRAM buffer are transferred at the fastest speed, ensuring that the I / O performance of the SSD reaches saturation.

[0270] In the present embodiment, a message processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0271] This embodiment provides a message processing device, such as Figure 7 As shown, including:

[0272] The acquisition module 701 is used to acquire the data processing message to be sent to the back end of the storage system and the current traffic characteristic information of each logical unit in the storage system.

[0273] The determination module 702 is used to determine the message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit.

[0274] The screening module 703 is used to screen the target logical unit according to the message transmission priority represented by the message load index of each logical unit.

[0275] The sending module 704 is used to send the data processing message to the storage system backend based on the target logical unit.

[0276] In some optional implementations, the determining module 702 includes:

[0277] The first determination unit is used to determine the pheromone concentration of each logical unit according to the target message type of the data processing message and the current flow characteristic information of each logical unit.

[0278] The second determining unit is used to determine the message load index of each logical unit according to the pheromone concentration of each logical unit.

[0279] In some optional implementations, the first determining unit includes:

[0280] The first determination subunit is used to determine the pheromone concentration of each logic unit based on the following formula:

[0281]

[0282] Among them, ρ represents the logical unit lun idx The pheromone concentration, idx represents the identification of the logical unit, Flow (lun idx ) represents the logical unit lun idx The current traffic value of the logic unit, φ represents the rated transmission bandwidth of the logic unit, and loss(type) represents the logic unit lun idx Processing target message type data processing message data transmission overhead, type represents the target message type, α is the impact factor.

[0283] The current traffic characteristic information includes the current traffic value, the rated transmission bandwidth, and the data transmission overhead of the data processing message of the processing target message type.

[0284] In some optional implementations, the message processing device further includes:

[0285] The rated transmission bandwidth determination unit is used to determine the rated transmission bandwidth of the logic unit based on the following formula:

[0286]

[0287] Among them, BW represents the theoretical total bandwidth value of the storage system, Sum(lun) represents the total number of logical units in the storage system, and Sum(bad_lun) represents the number of bad block logical units in the storage system.

[0288] In some optional implementations, the message processing device further includes:

[0289] The data transmission overhead determination unit is used to determine the logical unit lun based on the following formula idx Processing data of the target message type Data transmission overhead of processing messages:

[0290] loss(type)=Cnt(type)*t type *A BW

[0291] Among them, Cnt(type) represents the logical unit lun idx The number of data processed messages of the target message type processed within 1 second, t type Represents logical unit lun idx The time it takes to actually process a data processing message of the target message type, A BW Represents the measured logic unit lun idx bandwidth per second.

[0292] In some optional implementations, the second determining unit includes:

[0293] The first screening unit is used to screen the candidate logic unit with the highest pheromone concentration according to the pheromone concentration of each logic unit.

[0294] The first message load index determining unit is used to use the pheromone concentration as the message load index when there is only one logical unit to be selected.

[0295] The heuristic factor determination unit is used to determine the heuristic factor of each candidate logical unit according to the pheromone concentration and influencing characteristic information of each candidate logical unit when the candidate logical unit is not unique.

[0296] The second message load index determining unit is used to use the heuristic factor as the message load index.

[0297] In some optional implementations, the heuristic factor determination unit includes:

[0298] The heuristic factor determination subunit is used to determine the heuristic factor of each candidate logic unit based on the following formula:

[0299]

[0300] Among them, h represents the selected logic unit lun idx The heuristic factor, c is the preset pheromone dependency factor, ρ is the candidate logic unit lun idx The pheromone concentration, M_cnt(lun idx ) is the selected logic unit lun idx The number of data processing messages successfully sent to the backend of the storage system. Flow_avg is the average of the first flow of each logical unit. The influencing characteristic information includes the candidate logical unit lun idx The number of data processing messages successfully sent to the storage system backend and the first traffic mean of each logical unit.

[0301] In some optional implementations, the message processing device further includes:

[0302] The first flow average value determining unit is configured to determine the first flow average value of each logical unit based on the following formula:

[0303]

[0304] Where k = Sum (lun) represents the total number of logical units in the storage system, Flow (i) = Flow (lun idx ) is the current flow value of the i-th logical unit.

[0305] In some optional implementations, the screening module 703 includes:

[0306] The first target logic unit determination unit is used to select the candidate logic unit with the highest pheromone concentration as the target logic unit when the message load indicator is the pheromone concentration.

[0307] The second target logic unit determination unit is used to select the candidate logic unit with the highest heuristic factor as the target logic unit when the message load indicator is the heuristic factor.

[0308] In some optional implementations, before sending the data processing message to the storage system backend based on the target logical unit, the message processing device includes:

[0309] The latest flow value acquisition unit is used to acquire the latest flow value of each logical unit.

[0310] The second flow mean value determining unit is used to determine the second flow mean value of each logical unit according to the latest flow value of each logical unit.

[0311] The judging unit is used to judge whether the target logic unit passes the verification according to the first flow average value and the second flow average value.

[0312] The verification passing determination unit is used to determine that the target logical unit has passed the verification when the first traffic average value is the same as the second traffic average value, and execute the step of sending the data processing message to the storage system backend based on the target logical unit.

[0313] The verification failure determination unit is used to determine that the target logical unit has failed the verification when the first traffic mean is different from the second traffic mean, and return to the step of determining the message load index of each logical unit according to the target message type of the data processing message and the current traffic characteristic information of each logical unit, so as to re-screen the target logical unit until the first traffic mean is the same as the second traffic mean.

[0314] In some optional implementations, the message processing device further includes:

[0315] The traffic round-robin processing unit is used to perform traffic round-robin processing on each logical unit according to the screening time and target message type of the target logical unit when it is determined that the target logical unit has not passed the verification, so as to reduce the current traffic value of each logical unit.

[0316] In some optional implementations, the current flow characteristic information includes a current flow value, and the acquisition module 701 includes:

[0317] The current flow value determination unit is used to determine the current flow value of any logical unit according to the message type and message sending time of the data processing message currently sent by the logical unit to the back end of the storage system.

[0318] In some optional implementations, the sending module 704 includes:

[0319] The first sending unit is used to send the data processing message as a to-be-processed message to a preset write message interaction queue or a preset erase message interaction queue based on a target logical unit when the target message type is a write message or an erase message.

[0320] The second sending unit is used to send the pending messages in the preset write message interaction queue or the preset erase message interaction queue to the storage system backend according to the preset time constraint condition, so that the storage system backend processes the pending messages.

[0321] In some optional implementations, the second sending unit includes:

[0322] The first to-be-processed message sending unit is used to send the to-be-processed messages in the preset write message interaction queue to the storage system backend in sequence according to a first preset time interval.

[0323] The second to-be-processed message sending unit is used to send the to-be-processed messages in the preset message interaction queue to the storage system backend in sequence according to a second preset time interval.

[0324] In some optional implementations, the message processing device includes:

[0325] The first preset time interval determining unit is configured to determine the first preset time interval based on the following formula:

[0326] t wr =T ftl / wr_cnt-t nfc_wr

[0327] Among them, t wr represents the first preset time interval, wr_cnt represents the remaining write data amount of the current data block, T ftl Indicates the time it takes for the flash translation layer to write a data block, t nfc_wr Indicates the time it takes for the storage system backend to process a write message.

[0328] In some optional implementations, the message processing device includes:

[0329] The second preset time interval determining unit is configured to determine the second preset time interval based on the following formula:

[0330] t ers =Tftl / RAID_SIZE-t nfc_ers

[0331] Among them, t ers represents the second preset time interval, RAID_SIZE represents the number of parallel logic units, T ftl Indicates the time it takes for the flash translation layer to write a data block, t nfc_ers Indicates the time it takes for the storage system backend to process a delete message.

[0332] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0333] The message processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0334] The embodiment of the present invention also provides a computer device having the above Figure 7 The message processing device shown.

[0335] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 801, memory 802, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Each component utilizes different buses to communicate with each other, and can be installed on a common mainboard or installed in other ways as required. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 801 is taken as an example.

[0336] The processor 801 may be a central processing unit, a network processor or a combination thereof. The processor 801 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0337] The memory 802 stores instructions executable by at least one processor 801, so that the at least one processor 801 executes the method shown in the above embodiment.

[0338] The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 802 may optionally include a memory remotely arranged relative to the processor 801, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0339] The memory 802 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 802 may also include a combination of the above types of memory.

[0340] The computer device also includes a communication interface 803, which is used for the computer device to communicate with other devices or a communication network.

[0341] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0342] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0343] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A message processing method, characterized in that: The method comprises: Obtaining data processing messages to be sent to the back end of the storage system and current traffic characteristic information of each logical unit in the storage system; Determining a message load index of each of the logical units according to a target message type of the data processing message and current traffic characteristic information of each of the logical units; Filtering the target logical unit according to the message transmission priority represented by the message load indicator of each of the logical units; Based on the target logical unit, the data processing message is sent to the storage system backend.

2. The method according to claim 1, characterized in that The determining the message load index of each of the logical units according to the target message type of the data processing message and the current traffic characteristic information of each of the logical units includes: Determining the pheromone concentration of each of the logical units according to the target message type of the data processing message and the current traffic characteristic information of each of the logical units; According to the pheromone concentration of each of the logic units, a message load index of each of the logic units is determined.

3. The method according to claim 2, characterized in that The step of determining the pheromone concentration of each of the logic units according to the target message type of the data processing message and the current traffic characteristic information of each of the logic units includes: The pheromone concentration of each of the logical units is determined based on the following formula: Where ρ represents the logical unit lun idx The pheromone concentration, idx represents the identification of the logical unit, Flow (lun idx ) represents the logical unit lun idx The current traffic value of the logic unit, φ represents the rated transmission bandwidth of the logic unit, and loss(type) represents the logic unit lun idx Processing target message type data processing message data transmission overhead, type represents the target message type, α is the impact factor; The current traffic characteristic information includes the current traffic value, the rated transmission bandwidth, and the data transmission overhead of processing data processing messages of the target message type.

4. The method according to claim 3, characterized in that The method further comprises: The rated transmission bandwidth of the logic unit is determined based on the following formula: Where BW represents the theoretical total bandwidth of the storage system, Sum(lun) represents the total number of logical units in the storage system, and Sum(bad_lun) represents the number of bad block logical units in the storage system.

5. The method according to claim 4, characterized in that The method further comprises: Determine the logical unit lun based on the following formula idx Processing data of the target message type Data transmission overhead of processing messages: loss(type)=Cnt(type)*t type *A Bw Among them, Cnt(type) represents the logical unit lun idx The number of data processed messages of the target message type processed within 1 second, t type Represents the logical unit lun idx The time it takes to actually process a data processing message of the target message type, A BW Represents the measured logic unit lun idx bandwidth per second.

6. The method according to claim 2, characterized in that Determining the message load index of each of the logic units according to the pheromone concentration of each of the logic units includes: According to the pheromone concentration of each of the logic units, selecting the candidate logic unit with the highest pheromone concentration; In the case that the to-be-selected logical unit is unique, the pheromone concentration is used as the message load indicator; In the case that the candidate logical unit is not unique, determining the heuristic factor of each candidate logical unit according to the pheromone concentration and influencing characteristic information of each candidate logical unit; The heuristic factor is used as the message load indicator.

7. The method according to claim 6, characterized in that The step of determining the heuristic factor of each of the candidate logical units according to the pheromone concentration and the influencing characteristic information of each of the candidate logical units comprises: The heuristic factor of each of the candidate logic units is determined based on the following formula: Among them, h represents the selected logic unit lun idx The heuristic factor, c is the preset pheromone dependency factor, ρ is the candidate logic unit lun idx The pheromone concentration, M_cnt(lun idx ) is the selected logic unit lun idx The number of data processing messages successfully sent to the back end of the storage system, Flow_avg is the first flow average of each logical unit; the impact feature information includes the selected logical unit lun idx The number of data processing messages successfully sent to the storage system backend and the first flow average of each of the logical units.

8. The method according to claim 7, characterized in that The method further comprises: The first flow average value of each of the logic units is determined based on the following formula: Where k = Sum (lun) represents the total number of logical units in the storage system, Flow (i) = Flow (lun idx ) is the current flow value of the i-th logical unit.

9. The method according to claim 6, characterized in that The step of screening the target logical unit according to the message transmission priority represented by the message load indicator of each logical unit includes: In the case where the message load indicator is the pheromone concentration, the candidate logical unit with the highest pheromone concentration is selected as the target logical unit; In the case where the message load indicator is an inspiration factor, the candidate logical unit with the highest inspiration factor is used as the target logical unit.

10. The method according to claim 8, characterized in that Before sending the data processing message to the storage system backend based on the target logical unit, the method further includes: Obtaining the latest flow value of each of the logical units; Determine a second flow average value of each of the logical units according to the latest flow value of each of the logical units; Determining whether the target logical unit passes verification according to the first traffic average and the second traffic average; In the case where the first traffic average is the same as the second traffic average, determining that the target logical unit passes the verification, and executing the step of sending the data processing message to the storage system backend based on the target logical unit; When the first traffic mean is different from the second traffic mean, it is determined that the target logical unit has failed the verification, and the process returns to the step of determining the message load index of each logical unit based on the target message type of the data processing message and the current traffic characteristic information of each logical unit, so as to re-screen the target logical unit until the first traffic mean is the same as the second traffic mean.

11. The method according to claim 10, characterized in that The method further comprises: When it is determined that the target logical unit fails the verification, flow round robin processing is performed on each of the logical units according to the screening time of the target logical unit and the target message type, so as to decrement the current flow value of each of the logical units.

12. The method according to claim 1, characterized in that The current traffic characteristic information includes a current traffic value. Acquiring the current traffic characteristic information of each logical unit in the storage system includes: For any of the logical units, the current flow value of the logical unit is determined according to the message type and message sending time of the data processing message currently sent by the logical unit to the back end of the storage system.

13. The method according to claim 1, characterized in that The step of sending the data processing message to the storage system backend based on the target logical unit includes: In the case where the target message type is a write message or an erase message, based on the target logical unit, sending the data processing message as a to-be-processed message to a preset write message interaction queue or a preset erase message interaction queue; According to preset time constraints, the pending messages in the preset write message interaction queue or the preset erase message interaction queue are sent to the storage system backend, so that the storage system backend processes the pending messages.

14. The method according to claim 13, characterized in that The sending, according to the preset time constraint, the to-be-processed messages in the preset write message interaction queue or the preset erase message interaction queue to the storage system backend includes: Sending the pending messages in the preset write message interaction queue to the storage system backend in order according to a first preset time interval; According to a second preset time interval, the to-be-processed messages in the preset message interaction queue are sent to the storage system backend in order.

15. The method according to claim 14, characterized in that The method further comprises: The first preset time interval is determined based on the following formula: t wr =T ftl / wr_cnt-t nfc_wr Among them, t wr represents the first preset time interval, wwr_cnt represents the remaining write data amount of the current data block, T ftl Indicates the time it takes for the flash translation layer to write a data block, t nfc_wr Indicates the time it takes for the storage system backend to process a write message.

16. The method according to claim 14, characterized in that The method comprises: The second preset time interval is determined based on the following formula: t ers =T ftl / RAID_SIZE-t nfc_ers Among them, t ers represents the second preset time interval, RAID_SIZE represents the number of parallel logic units, T ftl Indicates the time it takes for the flash translation layer to write a data block, t nfc_ers Indicates the time it takes for the storage system backend to process a delete message.

17. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 16 by executing the computer instructions.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 16.

19. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 16.

Images