Method and system for optimizing cache flush of dirty data by a file storage gateway

Abstract

The application provides a method and system for optimizing cache dirty data flushing of a file storage gateway. The method comprises: dynamically monitoring dirty data IO operations in the cache, identifying the model and size of the IO; according to the model and size of the IO, applying a balanced binary tree data to convert the dirty data IO generated by the disordered random IO model into ordered dirty data IO; according to the size order and classification threshold of the dirty data IO, the dirty data IO is divided into two categories: small IO category and large IO category; according to the categories, the dirty data IO is aggregated or split by applying the aggregation threshold and the split threshold, and is distributed to the flush thread. According to the usage rate of the query memory block and the disk, the classification threshold, the aggregation threshold and the split threshold are adjusted. The application has the advantages of improving the disk flushing efficiency and improving the processing speed of the IO operation.

Description

TECHNICAL FIELD

[0001] The application relates to the computer technology field, in particular to the file storage technology field, and particularly relates to a file storage gateway optimized cache dirty data flushing method and system, an electronic device and a storage medium. BACKGROUND

[0002] Cache is a user state memory-based IO operation acceleration technology. In a storage system, cache can effectively alleviate the access pressure of the backend storage in large-scale data processing, low latency and high concurrency access, but the flushing mechanism of the cache exposes performance bottlenecks when dealing with complex I / O models.

[0003] Traditional flushing strategies, such as timing flushing and full flushing, often cannot effectively deal with complex and variable IO modes, and the flushing efficiency is reduced and dirty data is accumulated. The so-called dirty data refers to data that is modified in memory but has not been written back to disk. Small IO models are prone to frequent system resource calls due to their small data volume and high frequency, thereby reducing system performance. Large IO models have large data volume, but improper handling can also cause long-time occupation of system resources, causing performance bottlenecks.

[0004] Therefore, how to provide a file storage gateway optimized cache dirty data flushing method, system, electronic device and storage medium has become a technical problem to be solved in the field. SUMMARY

[0005] The purpose of the application is to provide a file storage gateway optimized cache dirty data flushing method, system, electronic device and storage medium.

[0006] According to the first aspect of the application, a file storage gateway optimized cache dirty data flushing method is provided, comprising,

[0007] According to the model and size of the dirty data IO, balanced binary tree data is applied to convert the dirty data IO generated by the disordered random dirty data IO model into ordered dirty data IO;

[0008] The dirty data IO is copied into a memory block, and an index value is used as a sorted key value, and is sequentially inserted into a self-balancing binary tree search, so as to convert the dirty data IO generated by the disordered random IO model into ordered dirty data IO;

[0009] According to the size order of the obtained ordered dirty data IO, the dirty data IO is divided into two categories, a small IO category and a large IO category, according to a preset classification threshold; according to the categories, an aggregation threshold and a splitting threshold are applied to aggregate or split the dirty data IO, and the dirty data IO is distributed to flush threads.

[0010] The dirty data IO is classified into two categories, small IO category and large IO category, according to the preset classification threshold, and the specific steps are as follows:

[0011] When the size of the dirty data IO is between 1B and 256K, the dirty data IO is identified as small IO; and when the size of the dirty data IO is greater than 256K, the dirty data IO is identified as large IO.

[0012] The aggregation threshold is between 2M and 16M.

[0013] The split threshold is between 1M and 8M.

[0014] According to the usage of the memory block and the disk, the classification threshold, the aggregation threshold and the split threshold are adjusted.

[0015] According to the method of the first aspect of the application, the method further comprises:

[0016] According to the offset of the dirty data IO, the starting index of the memory block is calculated, and according to the size of the dirty data IO, the number of required memory blocks and the ending index are calculated.

[0017] According to the method of the first aspect of the application, in the step S2, the application of the balanced binary tree data to convert the dirty data IO generated by the unordered random IO model into ordered dirty data IO according to the model and size of the IO comprises:

[0018] The dirty data IO is copied into the memory block, the index value is used as the key value for sorting, and the dirty data IO generated by the unordered random IO model is converted into ordered dirty data IO by sequentially inserting the self-balancing binary tree search.

[0019] According to the method of the first aspect of the application, the application of the aggregation threshold and the split threshold to aggregate or split the dirty data IO according to the category comprises:

[0020] If the dirty data IO is small IO, the dirty data IO is aggregated using the aggregation threshold and is distributed to the flush thread; and if the dirty data IO is large IO, the dirty data IO is split using the split threshold and is distributed to the flush thread.

[0021] The second aspect of the application discloses a file storage gateway optimized cache flush dirty data system, and the system comprises:

[0022] The first processing module is configured to dynamically monitor the dirty data IO operation in the cache, identify the model and size of the dirty data IO, and identify the model and size of the dirty data IO.

[0023] The second processing module is configured to apply the balanced binary tree data to convert the dirty data IO generated by the unordered random dirty data IO model into ordered dirty data IO according to the model and size of the dirty data IO.

[0024] copying the dirty data IO into the memory block, using the index value as a key value for sorting, sequentially inserting into a self-balancing binary tree search, and converting the dirty data IO generated by the unordered random IO model into ordered dirty data IO;

[0025] The third processing module is configured to divide the dirty data IO into two categories, a small IO category and a large IO category, according to a preset classification threshold, according to the size order of the obtained ordered dirty data IO; and apply an aggregation threshold and a split threshold to aggregate or split the dirty data IO according to the categories and distribute the dirty data IO to flush threads.

[0026] The dirty data IO is divided into two categories, a small IO category and a large IO category, according to a preset classification threshold, and specifically:

[0027] When the size of the dirty data IO is between 1B and 256K, the dirty data IO is identified as a small IO; and when the size of the dirty data IO is greater than 256K, the dirty data IO is identified as a large IO.

[0028] The aggregation threshold is between 2M and 16M.

[0029] The split threshold is between 1M and 8M.

[0030] The fourth processing module is configured to adjust the classification threshold, the aggregation threshold, and the split threshold according to the usage rate of the memory block and the disk.

[0031] According to the system of the second aspect of the present application, the first processing module is specifically configured to:

[0032] The starting index of the memory block is calculated according to the offset of the dirty data IO, and the number of memory blocks and the ending index required are calculated according to the size of the dirty data IO.

[0033] According to the system of the second aspect of the present application, the second processing module is specifically configured to convert the dirty data IO generated by the unordered random IO model into ordered dirty data IO by applying a balanced binary tree data according to the model and size of the IO, which includes:

[0034] copying the dirty data IO into the memory block, using the index value as a key value for sorting, sequentially inserting into a self-balancing binary tree search, and converting the dirty data IO generated by the unordered random IO model into ordered dirty data IO.

[0035] According to the system of the second aspect of the present application, the third processing module is specifically configured to aggregate or split the dirty data IO according to the categories by applying an aggregation threshold and a split threshold, which includes:

[0036] If the dirty data IO is a small IO type, the dirty data IO is aggregated using an aggregation threshold and distributed to a flush thread; if the dirty data IO is a large IO type, the dirty data IO is split using a split threshold and distributed to a flush thread.

[0037] The third aspect of the present application discloses an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method for optimizing cache flush dirty data of a file storage gateway according to any one of the first aspect of the present application when executing the computer program.

[0038] In the fifth aspect, the present application shows a computer program product, when the instructions in the computer program product are executed by the processor of the electronic device, the electronic device can execute the method according to any one of the above aspects.

[0039] The technical scheme provided by the present application can include the following beneficial effects:

[0040] In the file storage gateway optimization cache flush dirty data scheme provided by the present application, when the IO of the dirty data is a small IO operation, the aggregation flush disk strategy is adopted, and a certain number of small IO operations are collected through the buffer area before flush disk, which can reduce the number of disk I / O, thereby improving the flush disk efficiency. When the IO of the dirty data is a large IO operation, the segmentation and concurrent flush disk strategy is adopted, which can fully utilize the concurrent read-write capability of the disk and improve the processing speed of the large IO operation. The biggest advantage of the present application is that the flush disk threshold can be adjusted adaptively: by combining the cache resource usage and the disk usage, the threshold of small IO merging flush disk and the concurrency degree of large IO concurrent flush disk can be dynamically adjusted, thereby realizing the adaptive optimization of the flush disk strategy. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A flowchart of a file storage gateway optimization cache flush dirty data method according to an embodiment is provided;

[0042] Figure 2 A whole architecture diagram according to an embodiment is provided;

[0043] Figure 3 A front-end write IO processing flow according to an embodiment is provided;

[0044] Figure 4 A back-end flush dirty data processing flow according to an embodiment is provided;

[0045] Figure 5 A structure diagram of a file storage gateway optimization cache flush dirty data system according to an embodiment of the present application is provided;

[0046] Figure 6 A structure diagram of an electronic device according to an embodiment of the present application is provided;

[0047] Figure 7 Figure 1 is a structural diagram of a computer readable storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0049] The application discloses a file storage gateway optimized cache dirty data flushing method, aiming at the problem of system performance degradation caused by low efficiency of multi-model IO dirty data disk flushing and untimely dirty data processing in the cache system. The method comprises the following steps: dynamically monitoring IO operations in the cache, identifying the model and size of the IO, dynamically adjusting the IO classification standard, ensuring accurate classification of the IO, and avoiding performance loss caused by misclassification; collecting IO through a buffer area, then sorting and processing the collected IO requests by means of a balanced binary tree data organization form, so as to convert the dirty data generated by unordered random IO models into ordered IO; according to the monitoring result, the dirty data IO operations are divided into small IO and large IO, and different disk flushing modes are adopted respectively. The small IO operation adopts an aggregation disk flushing strategy through the buffer area; at the same time, a certain number of small IO operations are collected and then flushed, so as to reduce the number of disk I / O and improve the disk flushing efficiency; the large IO operation adopts a segmentation and concurrent disk flushing strategy, fully utilizes the concurrent read-write capability of the disk, and improves the processing speed of the large IO operation; and the threshold value of small IO merging disk flushing and the concurrency degree of large IO concurrent disk flushing are dynamically adjusted in combination with the cache resource usage and the disk usage, so as to realize adaptive optimization of the disk flushing strategy.

[0050] The file storage gateway optimized cache dirty data flushing method provided by the present application will be described below through specific embodiments.

[0051] Embodiment 1

[0052] According to a first aspect of the present application, the present application discloses a file storage gateway optimized cache dirty data flushing method. Figure 1 Figure 2 is a flowchart of a file storage gateway optimized cache dirty data flushing method according to an embodiment of the present application, as shown in Figure 1 and Figure 2 The method comprises the following steps:

[0053] Step S1, dynamically monitoring dirty data IO operations in the cache, identifying the model and size of the IO;

[0054] IO (Input / Output) is a basic concept in computer science, referring to the process of exchanging data between a computer system and external devices. IO operations include data input and output, specifically, input operations are the transfer of data from external devices (such as keyboard, mouse, etc.) to the computer's memory, while output operations are the transfer of data from memory to external devices.

[0055] In step S1, the dirty data IO operation in the cache is dynamically monitored, and the model and size of the IO are identified. The IO classification criteria are dynamically adjusted to ensure accurate classification of IO and avoid performance loss caused by misclassification.

[0056] In some embodiments, in the step S1, the method further comprises: calculating the starting index of the memory block according to the offset of the dirty data IO, and calculating the number of memory blocks required and the ending index according to the size of the dirty data IO.

[0057] Specifically, as shown in Figure 3 When data is written, it first enters the IO model identification module, and the IO model and size are identified by counting the IO data within a certain period. When data is written, the starting index of the memory block is calculated according to the offset of the IO, and the number of memory blocks required and the ending index are calculated according to the size.

[0058] Step S2, according to the model and size of the IO, apply balanced binary tree data to convert the dirty data IO generated by the unordered random IO model into ordered dirty data IO;

[0059] In some embodiments, in the step S2, as shown in Figure 3 The method of converting the dirty data IO generated by the unordered random IO model into ordered dirty data IO according to the model and size of the IO includes: copying the dirty data IO into the memory block, using the index value as the sorting key value, and sequentially inserting it into the self-balancing binary tree search, to convert the dirty data IO generated by the unordered random IO model into ordered dirty data IO.

[0060] The method of applying balanced binary tree data to convert the dirty data IO generated by the unordered random IO model into ordered dirty data IO is:

[0061] When the dirty data IO generated by the unordered random IO model is written, the starting index of the memory block is calculated according to the offset of the IO, the number of memory blocks required and the ending index are calculated according to the size, and the data is copied into the memory block, using the index value as the sorting key value, and sequentially inserted into the self-balancing binary tree search, to obtain the ordered dirty data IO;

[0062] Step S3, according to the size order of the dirty data IO and the classification threshold, the dirty data IO is divided into two categories of small IO category and large IO category;

[0063] In the step S3, the dirty data IO is divided into two categories of small IO category and large IO category according to the preset classification threshold, specifically:

[0064] When the size of the dirty data IO is between 1B and 256K (including 1B and 256K), it is identified as small IO; the size of the dirty data IO is greater than 256K, which is identified as large IO;

[0065] According to the category, the aggregation threshold and the split threshold are applied to aggregate or split the dirty data IO, and are distributed to the flush thread.

[0066] flush: refers to emptying the cache or flushing the cache, so that the fixed data is read or written more quickly.

[0067] flush thread: refers to the thread responsible for writing dirty pages in the cache system. These threads are named with "flush-" plus the device name. Their main role is to periodically write dirty pages in the system to the disk, ensuring data persistence and consistency.

[0068] Application scenario of flush thread: flush thread is mainly used to ensure data persistence and consistency. In file system operations, when data is modified, it may be temporarily stored in the buffer area of the memory instead of being directly written to the disk. Flush thread ensures data integrity and consistency by periodically writing these dirty pages to the disk. In addition, flush thread can also help optimize system performance by reducing the number of disk accesses, improving the rate of flushing dirty data and improving system response speed.

[0069] It should be noted that the numerical range of the aggregation threshold mentioned here is only a preferred way provided by the present application, and similarly, the numerical range of the split threshold is also only a preferred way provided by the present application, and should not be understood as a limitation of the present application. In actual application, reasonable selection can be made according to specific circumstances.

[0070] In step S3, as shown in Figure 4 According to the size order of the dirty data IO and the classification threshold, the dirty data IO is divided into two categories of small IO category and large IO category; according to the category, the aggregation threshold and the split threshold are applied to aggregate or split the dirty data IO, and are distributed to the flush thread.

[0071] In some embodiments, in the step S3, according to the category, the aggregation threshold and the split threshold are applied to aggregate or split the dirty data IO, including:

[0072] If the dirty data IO is a small IO type, the dirty data IO is aggregated using an aggregation threshold, that is, a certain number of small IO operations are collected through a buffer before being flushed to the disk, the number of disk I / O is reduced, the efficiency of disk flushing is improved, and the flush thread is distributed; if the dirty data IO is a large IO type, the dirty data IO is split using a split threshold, the flush thread is distributed, the concurrent read-write capability of the disk is fully utilized, and the processing speed of the large IO operation is improved.

[0073] Among them, the general aggregation threshold is between 2M-16M (including 2M and 16M); the split threshold is between 1M-8M (including 1M and 8M).

[0074] Specifically, after the dirty data is written to the disk, the memory block resource is released, and the statistical information is updated.

[0075] Step S4, adjusting the classification threshold, the aggregation threshold and the split threshold according to the usage of the memory block and the disk;

[0076] In step S4, the classification threshold, the aggregation threshold and the split threshold are adjusted according to the usage of the memory block and the disk. Combining the cache resource usage and the disk usage, the threshold of small IO merging and flushing is dynamically adjusted, and the concurrency of large IO concurrent flushing is dynamically adjusted, so that the adaptive optimization of the flushing strategy is realized.

[0077] In summary, the scheme proposed by the application can make small IO operations adopt an aggregation flushing strategy, collect a certain number of small IO operations through a buffer before flushing to the disk, reduce the number of disk I / O, and improve the efficiency of disk flushing; large IO operations adopt a split and slice concurrent flushing strategy, fully utilize the concurrent read-write capability of the disk, and improve the processing speed of large IO operations; combining the cache resource usage and the disk usage, the threshold of small IO merging and flushing is dynamically adjusted, and the concurrency of large IO concurrent flushing is dynamically adjusted, so that the adaptive optimization of the flushing strategy is realized.

[0078] It should be noted that, for the method embodiment, in order to simply describe, it is expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the order of the described actions, because according to the application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by the application.

[0079] Corresponding to the method embodiment in embodiment 1, the application discloses a file storage gateway optimized cache dirty data flushing system, which will be described below.

[0080] Embodiment 2:

[0081] Referring to Figure 5 FIG. 1 is a structural diagram of a file storage gateway optimized cache dirty data system according to an embodiment of the present application; as Figure 5 shown, the system 100 includes a first processing module 101, a second processing module 102, a third processing module 103, and a fourth processing module 104.

[0082] The first processing module 101 is configured to dynamically monitor dirty data IO operations in the cache, identify the model and size of the IO.

[0083] The second processing module 102 is configured to apply balanced binary tree data to convert the dirty data IO generated by the disordered random IO model into ordered dirty data IO according to the model and size of the IO.

[0084] The third processing module 103 is configured to divide the dirty data IO into two categories, small IO category and large IO category, according to the size order and classification threshold of the dirty data IO; and apply aggregation threshold and split threshold to aggregate or split the dirty data IO according to the category, and distribute it to the flush thread.

[0085] The fourth processing module 104 is configured to adjust the classification threshold, aggregation threshold, and split threshold according to the usage rate of the memory block and the disk.

[0086] In an implementation, the first processing module 101 is specifically configured to calculate the starting index of the memory block according to the offset of the dirty data IO, and calculate the number of memory blocks needed and the ending index according to the size of the dirty data IO.

[0087] Specifically, as Figure 3 shown, when data is written in the foreground, the IO model identification module can be entered first, and the IO model and size are identified by counting the IO data within a certain period; and when data is written, the starting index of the memory block is calculated according to the offset of the IO, and the number of memory blocks needed and the ending index are calculated according to the size.

[0088] In an implementation, the second processing module 102 is specifically configured to, as Figure 3 shown, apply balanced binary tree data to convert the dirty data IO generated by the disordered random IO model into ordered dirty data IO according to the model and size of the IO, which includes copying the dirty data IO into a memory block, using an index value as a sorted key value, sequentially inserting it into a self-balancing binary tree search, and converting the dirty data IO generated by the disordered random IO model into ordered dirty data IO.

[0089] In one implementation, the third processing module 103 is specifically configured to aggregate or split the dirty data IO according to the category by applying an aggregation threshold and a splitting threshold, including: if the dirty data IO is a small IO category, aggregating the dirty data IO using the aggregation threshold, that is, collecting a certain number of small IO operations through a buffer before performing the disk flushing, reducing the number of disk I / Os, improving the disk flushing efficiency, and distributing to a flush thread; if the dirty data IO is a large IO category, splitting the dirty data IO using the splitting threshold, distributing to the flush thread, fully utilizing the concurrent read-write capability of the disk, and improving the processing speed of the large IO operation.

[0090] Further, after the dirty data is written to the disk, the memory block resource can be released, and the statistical information is updated.

[0091] In the file storage gateway optimized cache dirty data flushing scheme provided by the application, when the IO of the dirty data is a small IO operation, an aggregation disk flushing strategy is adopted, and a certain number of small IO operations are collected through a buffer before disk flushing, which can reduce the number of disk I / Os, thereby improving the disk flushing efficiency. When the IO of the dirty data is a large IO operation, a segmentation and concurrent disk flushing strategy is adopted, which can fully utilize the concurrent read-write capability of the disk and improve the processing speed of the large IO operation. The biggest advantage of the application is that the disk flushing threshold can be adaptively adjusted: by combining the cache resource usage and the disk usage, the threshold of small IO merging disk flushing and the concurrency degree of large IO concurrent disk flushing can be dynamically adjusted, thereby realizing adaptive optimization of the disk flushing strategy.

[0092] For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts are described in the method embodiment.

[0093] The electronic device and the storage medium provided by the application will be described below.

[0094] Embodiment 3:

[0095] The application discloses an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps in the file storage gateway optimized cache dirty data flushing method of any one of the embodiments disclosed in the application when executing the computer program.

[0096] The electronic device provided by the application embodiment also comprises a processor, a memory, a computer program stored on the memory and executable on the processor, and the computer program is executed by the processor to implement the processes of the above method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0097] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to realize each process of the method embodiment and achieve the same technical effects. To avoid repetition, details are not described herein. The computer readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0098] Figure 6 A block diagram of an electronic device 800 is shown. The electronic device 800 can be, for example, a mobile phone, a computer, a digital broadcast terminal, a message transmitting / receiving device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0099] Referring to Figure 6 The electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0100] The processing component 802 usually controls overall operations of the electronic device 800, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the methods described above. In addition, the processing component 802 can include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0101] The memory 804 is configured to store various types of data to support operations of the electronic device 800. Examples of the data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, images, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0102] The power component 806 provides power to the various components of the electronic device 800. The power component 806 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0103] The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0104] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting an audio signal.

[0105] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0106] The sensor component 814 includes one or more sensors for providing status assessments of various aspects of the electronic device 800. For example, the sensor component 814 can detect an open / closed position of the device 800, relative positioning of components, such as a display and a keypad of the electronic device 800, a change in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, orientation or acceleration / deceleration / g-force and temperature of the electronic device 800. The sensor component 814 can include an optical sensor for detecting ambient light, a proximity sensor configured to detect proximity of an object without any physical contact, or a CMOS or CCD image sensor for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0107] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a cellular network (e.g., 2G, 3G, 4G or 5G), or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast operation information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0108] In an example embodiment, the electronic device 800 can be implemented using one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements to perform the above-described methods.

[0109] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 804 including instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to implement the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0110] Embodiment 4:

[0111] Figure 7is a block diagram of a computer-readable storage medium 1900 shown in the present application. For example, the computer-readable storage medium 1900 can be provided as a server.

[0112] Referring to Figure 7 The computer-readable storage medium 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-mentioned method.

[0113] The computer-readable storage medium 1900 can also include a power supply component 1926 configured to perform power management of the computer-readable storage medium 1900, a wired or wireless network interface 1950 configured to connect the computer-readable storage medium 1900 to a network, and an input / output (I / O) interface 1958. The computer-readable storage medium 1900 can operate based on an operating system stored in the memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0114] It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0115] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the methods described in various embodiments of the present application.

[0116] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection of the present application.

[0117] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the embodiments of the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0119] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented by other ways. For example, the device embodiments described above are only illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0120] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected to achieve the purpose of the embodiment of the present application according to actual needs.

[0121] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0122] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, etc.

[0123] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for optimizing cache flush data of a file storage gateway, characterized in that, The method comprises the following steps: dynamically monitoring dirty data IO operations in the cache, identifying the model and size of the dirty data IO; applying a balanced binary tree data to convert the dirty data IO generated by the unordered random dirty data IO model into ordered dirty data IO according to the model and size of the dirty data IO; copying the dirty data IO into a memory block, using the index value as the sorted key value, sequentially inserting the self-balancing binary tree search, and converting the dirty data IO generated by the unordered random IO model into ordered dirty data IO; according to the size order of the obtained ordered dirty data IO, the dirty data IO is divided into two categories, small IO category and large IO category, according to the classification threshold; according to the category, the dirty data IO is aggregated or split by applying the aggregation threshold and the split threshold, and is distributed to the flush thread; adjusting the classification threshold, the aggregation threshold and the split threshold according to the usage of the memory block and the disk; the method further comprises: calculating the starting index of the memory block according to the offset of the dirty data IO, and calculating the number of memory blocks and the ending index required according to the size of the dirty data IO. The method further comprises:

2. The method of claim 1, wherein, calculating the starting index of the memory block according to the offset of the dirty data IO, and calculating the number of memory blocks and the ending index required according to the size of the dirty data IO. The system comprises:

3. The method of claim 2, wherein, a first processing module configured to dynamically monitor dirty data IO operations in the cache, identify the model and size of the dirty data IO; a second processing module configured to apply a balanced binary tree data to convert the dirty data IO generated by the unordered random dirty data IO model into ordered dirty data IO according to the model and size of the dirty data IO; 4. A file storage gateway optimized cache flush dirty data system, comprising: copying the dirty data IO into a memory block, using the index value as the sorted key value, sequentially inserting the self-balancing binary tree search, and converting the dirty data IO generated by the unordered random IO model into ordered dirty data IO; a third processing module configured to divide the dirty data IO into two categories, small IO category and large IO category, according to the classification threshold according to the size order of the obtained ordered dirty data IO; according to the category, the dirty data IO is aggregated or split by applying the aggregation threshold and the split threshold, and is distributed to the flush thread; a fourth processing module configured to adjust the classification threshold, the aggregation threshold and the split threshold according to the usage of the memory block and the disk; the method further comprises: calculating the starting index of the memory block according to the offset of the dirty data IO, and calculating the number of memory blocks and the ending index required according to the size of the dirty data IO. ​ ​ If the dirty data IO is a small IO type, the dirty data IO is aggregated using an aggregation threshold and distributed to a flush thread; if the dirty data IO is a large IO type, the dirty data IO is split using a split threshold and distributed to the flush thread.

5. The file storage gateway optimized cache flush data system of claim 4, wherein, The first processing module is specifically configured to: The starting index of the memory block is calculated according to the offset of the dirty data IO, and the number of required memory blocks and the ending index are calculated according to the size of the dirty data IO.

6. The file storage gateway optimized cache flush data system of claim 5, wherein, The second processing module is specifically configured to: when the size of the dirty data IO is between 1B and 256K, the dirty data IO is identified as a small IO; and when the size of the dirty data IO is greater than 256K, the dirty data IO is identified as a large IO.

7. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the file storage gateway optimization cache dirty data flushing method in any one of claims 1 to 3 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the file storage gateway optimization cache dirty data flushing method in any one of claims 1 to 3.

Images