Performance optimization method, apparatus, and storage medium

Abstract

The application discloses a performance optimization method and device and a storage medium, relates to the field of communication technologies, and aims to solve the problem that different types of disks cannot be adaptively optimized in performance. The method comprises the following steps: acquiring first data, a first selection rate of a first engine for processing the first data, and a second selection rate of a second engine for processing the first data; calling the first engine to select data from the first data for processing based on the first selection rate, determining a first generation value, and calling the second engine to select data from the first data for processing based on the second selection rate, and determining a second generation value; and updating the first selection rate of the first engine and the second selection rate of the second engine based on the first generation value and the second generation value.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a performance optimization method, apparatus and storage medium. Background Technology

[0002] Because different manufacturers and models of non-volatile memory express solid state drives (NVMe SSDs) use different internal design mechanisms, the performance of NVMe SSDs varies greatly, and the performance of clusters composed of different NVMe SSDs can differ significantly.

[0003] Currently, common performance optimization methods are usually designed for fixed disks with specific hardware and software configurations. These methods are generally ineffective when switching to a different NVMe SSD model. However, cloud providers need to ensure high performance across different storage server and disk models. Therefore, how to adaptively optimize performance for different disk models is a pressing issue that needs to be addressed. Summary of the Invention

[0004] This application provides a performance optimization method, apparatus, and storage medium to solve the problem that general-purpose technologies cannot achieve adaptive performance optimization for different types of disks.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, a performance optimization method is provided, comprising: acquiring first data; a first selection rate of a first engine processing the first data; and a second selection rate of a second engine processing the first data; wherein the selection rate is used to characterize the probability that the engine selects data from the data for processing; the first engine and the second engine process the data in different ways; invoking the first engine to select data from the first data for processing based on the first selection rate to determine a first-generation value; and invoking the second engine to select data from the first data for processing based on the second selection rate to determine a second-generation value; wherein the generation value is used to characterize the time and / or resources required to process the data; and updating the first selection rate of the first engine and the second selection rate of the second engine based on the first-generation value and the second-generation value.

[0007] Optionally, the second engine is invoked to select data from the first data based on a second selectivity to process and determine the second-generation value. This includes: invoking the second engine to select second data from the first data based on the second selectivity; identifying third data in the second data whose size is greater than a first threshold; invoking the second engine to split the third data into multiple sub-data; processing the multiple sub-data of the third data respectively; obtaining the first time required for the second engine to process the multiple sub-data of the third data; and determining the second-generation value based on the first time and the size of the third data.

[0008] Optionally, the first engine is invoked to select data from the first data based on a first selection rate for processing to determine the first generation value, including: invoking the first engine to select fourth data from the first data based on the first selection rate; performing processing operations on the fourth data; obtaining the second time required for the first engine to perform processing operations on the fourth data; and determining the first generation value based on the second time and the data size of the fourth data.

[0009] Optionally, based on the first-generation value and the second-generation value, update the first selection rate of the first engine and the second selection rate of the second engine, including: increasing the value of the second selection rate of the second engine when the first-generation value is greater than the second-generation value; and increasing the value of the first selection rate of the first engine when the first-generation value is less than the second-generation value.

[0010] Optionally, it also includes: correcting the updated first selection rate, determining the third selection rate, and correcting the updated second selection rate to determine the fourth selection rate; obtaining the fifth data, calling the first engine to select data from the fifth data based on the third selection rate for processing to determine the third generation value, and calling the second engine to select data from the fifth data based on the fourth selection rate for processing to determine the fourth generation value; and updating the first selection rate of the first engine and the second selection rate of the second engine based on the third generation value and the fourth generation value.

[0011] Optionally, the second engine is invoked to select data from the fifth data based on the fourth selectivity for processing to determine the fourth-generation value. This includes: invoking the second engine to select the sixth data from the fifth data based on the fourth selectivity; identifying the seventh data in the sixth data whose size is greater than the second threshold; invoking the second engine to split the seventh data into multiple sub-data; processing the multiple sub-data of the seventh data respectively; obtaining the third time required for the second engine to process the multiple sub-data of the seventh data; and determining the fourth-generation value based on the third time and the size of the seventh data.

[0012] Optionally, the engine's data processing includes: read processing and / or write processing.

[0013] Secondly, a performance optimization device is provided, comprising: a communication unit and a processing unit; the communication unit is configured to acquire first data, a first selection rate of a first engine processing the first data, and a second selection rate of a second engine processing the first data; wherein the selection rate is used to characterize the probability that the engine selects data from the data for processing; the first engine and the second engine process the data in different ways; the processing unit is configured to invoke the first engine to select data from the first data for processing based on the first selection rate to determine a first generation value, and to invoke the second engine to select data from the first data for processing based on the second selection rate to determine a second generation value; wherein the generation value is used to characterize the time and / or resources required to process the data; the processing unit is further configured to update the first selection rate of the first engine and the second selection rate of the second engine based on the first generation value and the second generation value.

[0014] Optionally, the processing unit is specifically used for: calling the second engine to select second data from the first data based on a second selectivity; the processing unit is specifically used for: determining third data in the second data whose data size is greater than a first threshold; the processing unit is specifically used for: calling the second engine to split the third data to obtain multiple sub-data of the third data; the processing unit is specifically used for: performing processing operations on the multiple sub-data of the third data respectively; the acquisition unit is specifically used for: acquiring the first time required for the second engine to perform processing operations on the multiple sub-data of the third data; and the processing unit is specifically used for: determining the second-generation value based on the first time and the data size of the third data.

[0015] Optionally, the processing unit is specifically used for: calling the first engine to select the fourth data from the first data based on the first selection rate; the processing unit is specifically used for: performing processing operations on the fourth data; the acquisition unit is specifically used for: acquiring the second time required for the first engine to perform processing operations on the fourth data; and the processing unit is specifically used for: determining the first generation value based on the second time and the data size of the fourth data.

[0016] Optionally, the processing unit is specifically used to: increase the value of the second selection rate of the second engine when the value of the first generation is greater than the value of the second generation; and increase the value of the first selection rate of the first engine when the value of the first generation is less than the value of the second generation.

[0017] Optionally, the processing unit is further configured to: correct the updated first selection rate, determine the third selection rate, and correct the updated second selection rate to determine the fourth selection rate; the acquisition unit is further configured to: acquire fifth data; the processing unit is further configured to: call the first engine to select data from the fifth data based on the third selection rate for processing to determine the third generation value, and call the second engine to select data from the fifth data based on the fourth selection rate for processing to determine the fourth generation value; the processing unit is further configured to: update the first selection rate of the first engine and the second selection rate of the second engine based on the third generation value and the fourth generation value.

[0018] Optionally, the processing unit is specifically used for: calling the second engine to select the sixth data from the fifth data based on the fourth selectivity; the processing unit is specifically used for: determining the seventh data in the sixth data whose data size is greater than the second threshold; the processing unit is specifically used for: calling the second engine to split the seventh data to obtain multiple sub-data of the seventh data; the processing unit is specifically used for: performing processing operations on the multiple sub-data of the seventh data respectively; the communication unit is specifically used for: obtaining the third time required for the second engine to perform processing operations on the multiple sub-data of the seventh data; and the processing unit is specifically used for: determining the fourth generation value based on the third time and the data size of the seventh data.

[0019] Optionally, the engine's data processing includes: read processing and / or write processing.

[0020] Thirdly, a performance optimization apparatus is provided, including a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory via a bus; when the performance optimization apparatus is running, the processor executes the computer execution instructions stored in the memory, so that the performance optimization apparatus performs the performance optimization method of the first aspect.

[0021] The performance optimization device may be a network device or a component of a network device, such as a chip system within the network device. This chip system supports the network device in implementing the functions involved in the first aspect and any of its possible implementations, such as acquiring, determining, and transmitting the data and / or information involved in the aforementioned performance optimization method. The chip system includes a chip, but may also include other discrete devices or circuit structures.

[0022] Fourthly, a computer-readable storage medium is provided, comprising computer-executable instructions that, when executed on a computer, cause the computer to perform the performance optimization method of the first aspect.

[0023] Fifthly, a computer program product is also provided, the computer program product including computer instructions that, when executed on a performance optimization device, cause the performance optimization device to perform the performance optimization method as described in the first aspect above.

[0024] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the performance optimization device, or it may be packaged separately from the processor of the performance optimization device; this application does not impose any limitations on this.

[0025] The descriptions of the second, third, fourth, and fifth aspects in this application can be referenced to the detailed description of the first aspect; and the beneficial effects of the second, third, fourth, and fifth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0026] In this application, the names of the aforementioned performance optimization devices do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those in this application, they fall within the scope of the claims of this application and their equivalents.

[0027] These or other aspects of this application will become more readily apparent in the following description.

[0028] The technical solution provided in this application brings at least the following beneficial effects:

[0029] Based on any of the above, this application provides a performance optimization method. This application can dynamically select a suitable engine for processing first data, and determine a first-generation value and a second-generation value based on the time and / or resources required for processing the first data by the first engine and the second engine, continuously updating the selection rates of the first engine and the second engine. By processing the first data with a more suitable engine, this application can ensure that the data is processed faster and more efficiently. By continuously updating the selection rates of the first engine and the second engine, the engine can be adaptively adjusted for different types of disks, thus optimizing system performance. Attached Figure Description

[0030] Figure 1 A system architecture diagram of a performance optimization system provided in this application embodiment;

[0031] Figure 2 This is a schematic diagram of the hardware structure of a performance optimization device provided in an embodiment of this application;

[0032] Figure 3 A flowchart illustrating a performance optimization method provided in this application embodiment. Figure 1 ;

[0033] Figure 4 A flowchart illustrating a performance optimization method provided in this application embodiment. Figure 2 ;

[0034] Figure 5 A flowchart illustrating a performance optimization method provided in this application embodiment. Figure 3 ;

[0035] Figure 6 A flowchart illustrating a performance optimization method provided in this application embodiment. Figure 4 ;

[0036] Figure 7 This is a schematic diagram of a performance optimization device provided in an embodiment of this application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0038] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0039] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0040] The following explanations of the terms used in the embodiments of this application are provided to facilitate the reader's understanding.

[0041] (1) NVMe SSD: A solid-state drive based on the NVMe protocol, which uses the Peripheral Component Interconnect Express (PCI-E) channel for data transmission. Compared with the traditional Advanced Host Controller Interface (AHCI) protocol, the NVMe protocol has a higher transmission rate and lower latency, making NVMe SSDs superior in terms of read / write speed, response time, and transmission bandwidth.

[0042] (2) Multi-queue technology: Multi-queue technology is a resource scheduling strategy that divides the processes or tasks in the system into multiple queues with different priorities and schedules the processes or tasks in different queues to different degrees, so as to effectively improve the resource utilization of the system and enhance the system's responsiveness and stability.

[0043] (3) Nand flash: Nand flash is a non-volatile memory based on Nand technology. It uses floating gate transistors as storage cells and controls the conduction and cutoff of storage cells by controlling the gate, thereby realizing data storage.

[0044] (4) Concurrent Unit: A concurrent unit is a concept in a computer system used to describe programs or tasks that can be executed simultaneously or concurrently. A concurrent unit typically consists of multiple threads or processes that can run at the same time and share system resources.

[0045] (5) Erasure cell: An eraser cell is a concept in flash memory, referring to the smallest erasable and programmable unit. In flash memory, data is stored in the form of electric charge in floating gates, and the eraser cell is the smallest storage unit that can be independently erased and programmed.

[0046] (6) Garbage collection (GC) is an automatic memory management mechanism used to automatically reclaim memory space that is no longer used in a program, in order to prevent memory leaks and make efficient use of available memory resources. During the garbage collection process, by tracking the memory usage in the program, it automatically identifies and removes memory space that is no longer used, releasing the space it occupies so that these resources can be reused.

[0047] (7) Write amplification: Write amplification is a concept in flash memory storage, referring to the ratio of actual physical storage space to logically required storage space in flash memory. Because flash memory operates using block erasure, when updating data, the entire block must be erased and then rewritten, rather than directly modifying individual data. This results in actual storage space being more than logically required, hence the term write amplification.

[0048] (8) fio: fio is a multi-threaded input / output (I / O) generation tool that can stress test and verify the disk.

[0049] (9) Queue depth iodepth: iodepth is a parameter related to I / O performance. It is usually used to describe the depth of the I / O request queue, that is, how many I / O requests are running on average, that is, how many I / O requests the device processes at the same time.

[0050] The above is a brief introduction to some of the concepts involved in the embodiments of this application.

[0051] In large-scale distributed storage systems using at least one NVMe SSD, when benchmark performance testing tools (such as fio) are used to test single-disk or cluster performance, setting iodepth to 1 reveals that the actual performance is significantly lower than when iodepth is set to 1. This is because NVMe SSDs employ multi-queue technology, and when I / O is performed in a single queue, the performance advantage of NVMe SSDs' multi-queue concurrency cannot be fully utilized.

[0052] Because different manufacturers and models of NVMe SSDs use different internal design mechanisms, the write performance of NVMe SSDs varies greatly, and the performance of clusters composed of different NVMe SSDs also varies significantly.

[0053] Currently, common performance optimization methods are usually designed for fixed disks with specific hardware and software configurations. These methods are typically ineffective when switching to a different NVMe SSD model. However, cloud providers need to ensure high performance across different storage server and disk models. Therefore, how to achieve adaptive performance optimization at the underlying storage system level is a pressing issue that needs to be addressed.

[0054] This application provides a performance optimization method. This method dynamically selects a suitable engine for processing first data, and determines a first-generation value and a second-generation value based on the time and / or resources required for processing the first data by the first and second engines, continuously updating the selection rates of the first and second engines. By processing the first data with a more suitable engine, this application ensures faster and more efficient data processing. Furthermore, by continuously updating the selection rates of the first and second engines, it can adaptively adjust the engines for different disk models and optimize system performance.

[0055] For example, this performance optimization method is applicable to performance optimization systems. Figure 1 Taking the object storage daemon (OSD) storage engine Bluestore of the Ceph distributed storage system as an example, a system architecture diagram of a performance optimization system is shown.

[0056] exist Figure 1In this system, a software layer exists between the storage engine BlueStore and the driver layer. This software layer can handle I / O for backend NVMe SSDs. In this solution, two additional layers are added to the software layer of the performance optimization system: a dual-engine cost estimation and planning layer, and a dual-engine processing layer. Figure 1 The area outlined in the dashed line represents the dual-engine processing layer.

[0057] The dual-engine processing layer includes a first engine and a second engine. The dual-engine processing layer processes the data to be processed from the dual-engine cost estimation and planning layer, and sends the processing results to the NVMe SSD disk after processing. The NVMe SSD disk then continues processing, receives the processing results from the NVMe SSD disk, and sends the final processing results to the dual-engine cost estimation and planning layer.

[0058] The dual-engine cost estimation and planning layer is used to estimate the cost value of the first engine and the second engine, and to update the selection rates of the first engine and the second engine.

[0059] For example, this performance optimization system can be used with a performance optimization device. The performance optimization device can be a handheld device with wireless connectivity, a wireless terminal connected to another processing device via a wireless modem, or a wired terminal. Examples include smart devices such as mobile phones, personal computers (PCs), desktop computers, tablets, laptops, netbooks, and personal digital assistants (PDAs), but this application embodiment does not limit the scope of the application.

[0060] In some embodiments, the performance optimization device can be a functional module on any base station or a stand-alone physical device.

[0061] It's easy to understand that when the performance optimization device is a functional module on the base station, the interaction between the performance optimization device and the base station is the same as the interaction between modules within the base station. In this case, the interaction process is the same as when the performance optimization device is an independently configured physical device.

[0062] Optionally, when the performance optimization device is a standalone physical device, it can be a standalone server or other type of physical device. This physical device can be a server in a server cluster (composed of multiple servers), a chip within the physical device, a system-on-a-chip within the physical device, or it can be implemented through a virtual machine deployed on a physical machine. This application does not limit the scope of this embodiment.

[0063] For ease of understanding, this application uses the example of "the performance optimization device being a separately configured physical device" for illustration.

[0064] In practice, performance optimization devices can adopt Figure 2 The shown composition structure, or including Figure 2 The components shown. Figure 2 This is a schematic diagram of the hardware structure of a performance optimization device provided in an embodiment of this application. Figure 2 As shown, the performance optimization device may include a processor 201 and a communication line 202.

[0065] Furthermore, the performance optimization device may also include a communication interface 203 and a memory 204. The processor 201, memory 204, and communication interface 203 can be connected via a communication line 202.

[0066] The processor 201 can be a CPU, a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 201 can also be other devices with processing capabilities, such as circuits, devices, or software modules, without limitation.

[0067] Communication line 202 is used to transmit information between the components included in the performance optimization device.

[0068] Communication interface 203 is used to communicate with other devices or other communication networks. These other communication networks can be Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc. Communication interface 203 can be a module, circuit, communication interface, or any device capable of enabling communication.

[0069] Memory 204 is used to store instructions. These instructions can be computer programs.

[0070] The memory 204 can be a read-only memory (ROM) or other type of static storage device that can store static information and / or instructions; it can also be a random access memory (RAM) or other type of dynamic storage device that can store information and / or instructions; it can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, etc., without limitation.

[0071] It should be noted that the memory 204 can exist independently of the processor 201 or can be integrated with the processor 201. The memory 204 can be used to store instructions, program code, or some data, etc. The memory 204 can be located inside or outside the user recommendation device, without limitation. The processor 201 is used to execute the instructions stored in the memory 204 to implement the user recommendation method provided in the following embodiments of this application.

[0072] In one example, processor 201 may include one or more CPUs, such as CPU0 and CPU1.

[0073] As an alternative implementation, performance-optimized devices include multiple processors.

[0074] As an alternative implementation, the performance optimization device also includes output devices and input devices. For example, output devices are devices such as displays and speakers, while input devices are devices such as keyboards, mice, microphones, or joysticks.

[0075] It should be noted that performance optimization devices can be desktop computers, laptops, web servers, mobile phones, tablets, wireless terminals, embedded devices, chip systems, or other devices. Figure 2 Equipment with a similar structure. Furthermore... Figure 2 The composition shown does not constitute a part of the composition. Figure 2 The limitations of each device in the process, except Figure 2 In addition to the components shown, Figure 2 The various devices may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0076] In this embodiment of the application, the chip system may be composed of chips or may include chips and other discrete devices.

[0077] Furthermore, the actions, terms, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages exchanged between the various devices in the embodiments of this application are merely examples, and other names may be used in specific implementations without limitation.

[0078] The following is combined Figure 1 The performance optimization system shown describes the performance optimization method provided in the embodiments of this application. The actions, terminology, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages exchanged between various devices in the embodiments of this application are merely examples; other names may be used in specific implementations without limitation. The actions involved in the various embodiments of this application are merely examples; other names may be used in specific implementations. For example, "included in" in the embodiments of this application can be replaced with "carried on" or "carried in," etc.

[0079] The performance optimization method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings. Figure 3 As shown, the performance optimization method includes:

[0080] S301, the performance optimization device acquires first data, the first selection rate of the first engine that processes the first data, and the second selection rate of the second engine that processes the first data.

[0081] The selection rate is used to characterize the probability that the engine will choose data from the dataset for processing. The first engine and the second engine process data in different ways.

[0082] Optionally, the engine's data processing includes: read processing and / or write processing.

[0083] In one possible implementation, when initially setting the first selectability of the first engine and the second selectability of the second engine, the performance optimization device can make the first selectability equal to the second selectability, that is, set both the first selectability and the second selectability to 50%.

[0084] For example, when the first engine and the second engine are reading the first data, the performance optimization device sets the first engine's first select ratio (engine1 read select ratio) to 50%, and the performance optimization device sets the second engine's second select ratio (engine2 read select ratio) to 50%.

[0085] For example, when the first engine and the second engine are writing the first data, the performance optimization device sets the first engine's first select ratio (engine1 write select ratio) to 50%, and the performance optimization device sets the second engine's second select ratio (engine1 write select ratio) to 50%.

[0086] S302, The performance optimization device calls the first engine to select data from the first data based on the first selectivity for processing to determine the first generation value, and calls the second engine to select data from the first data based on the second selectivity for processing to determine the second generation value.

[0087] The cost value is used to characterize the time and / or resources required to process the data.

[0088] For example, the first data includes 200 first data points, with a first selection rate of 50% and a second selection rate of 50%. The performance optimization device randomly selects 100 first data points based on the first selection rate and invokes the first engine for processing. The performance optimization device determines the first-generation value of the first engine based on the time required for the first engine to process the 100 first data points and the size of the 100 first data points.

[0089] The performance optimization device randomly selects 100 first data points based on a second selectivity and calls the second engine for processing. The performance optimization device determines the second-generation value of the second engine based on the time required for the second engine to process the 100 first data points and the size of the 100 first data points. The performance optimization device sets up a lock control mechanism to prevent the first engine and the second engine from processing the same first data point.

[0090] S303, the performance optimization device updates the first engine's first selection rate and the second engine's second selection rate based on the first-generation value and the second-generation value.

[0091] In one possible implementation, after the performance optimization device determines the first-generation value and the second-generation value, the performance optimization device updates the selection rate so that the selection rate of the engine with the higher generation value is lower than the selection rate of the engine with the lower generation value.

[0092] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S301-S303, this application can dynamically select a suitable engine for data processing of the first data, and determine the first-generation value and the second-generation value based on the time and / or resources required for the first engine and the second engine to process the first data, and continuously update the selection rate of the first engine and the second engine. By processing the first data with a more suitable engine, this application can ensure that the data is processed faster and more effectively. By continuously updating the selection rate of the first engine and the second engine, the engine can be adaptively adjusted for different types of disks and the system performance can be optimized.

[0093] In one alternative embodiment, combined with Figure 3 ,like Figure 4 As shown, the process of calling the second engine in S302 to select data from the first data based on the second selectivity for processing and determining the second-generation value can be specifically implemented through the following S401-S406. The details are as follows:

[0094] S401, The performance optimization device calls the second engine to select the second data from the first data based on the second selectivity.

[0095] In one possible implementation, the performance optimization device randomly selects second data from the first data for read and / or write processing based on a second selectivity.

[0096] For example, in the reading process of the first data, when the first data includes 100 data points and the second selectivity is 50%, the performance optimization device randomly selects second data points from the first data based on the second selectivity. The second data points include 50 data points.

[0097] S402, The performance optimization device determines a third data point in the second data set whose data size is greater than the first threshold.

[0098] In one possible implementation, the performance optimization device identifies third data points in the second data set whose size is greater than a first threshold, and these are then further split by a second engine. For data points in the second data set whose size is less than or equal to the first threshold, the second engine does not perform any splitting.

[0099] Optionally, the first threshold can be 256KB, 512KB, etc., and this application does not impose any restrictions on it.

[0100] S403, the performance optimization device calls the second engine to split the third data and obtain multiple sub-data of the third data.

[0101] For example, regarding the reading of the second data, the performance optimization device determines that the data size in the second data exceeds a first threshold (read split threshold), such as a third data set of 256KB. The performance optimization device splits the third data into split units, such as 16KB, resulting in multiple sub-data sets for each third data set. These sub-data sets do not require re-allocating data memory space; they can reuse a segment of the parent data's memory space.

[0102] For example, in the write processing of the second data, the performance optimization device determines that the data size in the second data is greater than a first threshold write split threshold, such as 256KB of third data. The performance optimization device splits each third data into 16KB units, resulting in multiple sub-data for each third data.

[0103] S404 and the performance optimization device respectively process multiple sub-data of the third data.

[0104] In one possible implementation, after the second engine obtains multiple sub-data of the third data, it sends all the sub-data to the NVMe SSD drive and disk, and uses the NVMe SSD's multi-queue concurrent processing unit to process these sub-data concurrently.

[0105] S405, The performance optimization device obtains the first time required for the second engine to process multiple sub-data of the third data.

[0106] In one possible implementation, after the NVMe SSD has completed reading all the sub-data, the parent data's memory space already contains the read data. During this process, there's no need to assemble the sub-data, and the performance optimization device determines that the processing was successful. The performance optimization device captures the time from when the second engine selects data from the first data set to when it determines that the processing was successful.

[0107] In one possible implementation, after the NVMe SSD has completed writing all the child data, the data in the parent data's memory space has been completely written to the disk, and the performance optimization device determines that the processing was successful. The performance optimization device then retrieves the data selected from the first data by the second engine, up to the moment when the processing is confirmed as successful.

[0108] S406, The performance optimization device determines the second-generation value based on the data size of the first and third time data.

[0109] For example, for each piece of data in the third set of data, the second-generation value (io cost) of the second engine for each piece of data can be obtained, and the io cost satisfies the following formula 1. ：

[0110]

[0111] Where io size is the size of the data, and time is the initial processing time required to process the data. The performance optimization device linearly maps the initial processing time required to process each piece of data in the third set of data to the processing time of 256KB, avoiding time deviations caused by different data.

[0112] Optionally, for ease of calculation, the result of io cost can be an integer.

[0113] For example, for a 512KB data, if the first time required for the second engine to write the data is 0.5ms, then based on the above formula, the performance optimization device determines that the cost value for the second engine to calculate the data is 25.

[0114] In one possible implementation, the performance optimization device determines the second-generation value of each data point written by the second engine to the third data set, and averages these values ​​to obtain the final second-generation value, engine2write cost. Similarly, the performance optimization device determines the second-generation value of each data point read by the second engine to the third data set, and averages these values ​​to obtain the final second-generation value, engine2read cost.

[0115] The technical solutions provided by the above embodiments bring at least the following beneficial effects: As shown in S401-S406, this application can split the third data into multiple sub-data through the second engine, enabling the system to process these sub-data in parallel, thereby improving the speed and efficiency of data processing. Determining the second-generation value based on the first time and the data size of the third data facilitates dynamic adjustment and optimization of the data processing process by the system, ensuring the efficiency of data processing in terms of time and resources.

[0116] In one alternative embodiment, combined with Figure 3 ,like Figure 4 As shown, the process of determining the second-generation value in S302 can be specifically implemented through the following S407-S410. A detailed explanation follows:

[0117] S407, The performance optimization device calls the first engine to select the fourth data from the first data based on the first selectivity.

[0118] For example, in the write processing of the first data, when the first data consists of 100 data points and the first selection rate is 50%, the performance optimization device randomly selects a fourth data point from the first data based on the first selection rate. The fourth data point consists of 50 data points.

[0119] S408, the performance optimization device processes the fourth data.

[0120] It should be understood that some NVMe SSD models have relatively low hardware configurations or poor internal software design. For these drives, even optimizations at the driver level do not improve performance. Therefore, the first engine is not optimized in any way, and the fourth data is not processed.

[0121] In one possible implementation, the first engine does not process the fourth data and directly sends it to the NVMe SSD. The NVMe SSD receives the fourth data and processes it.

[0122] S409, The performance optimization device obtains the second time required for the first engine to process the fourth data.

[0123] In one possible implementation, after the NVMe SSD completes processing the fourth data, it returns a success message to the upper layer. The performance optimization device determines that the processing was successful and obtains the second time from when the first engine selects data from the first data to when the processing is confirmed to be successful.

[0124] S410, the performance optimization device determines the first-generation value based on the data size of the second time and the fourth data.

[0125] In one possible implementation, the performance optimization device uses Formula 1 to determine the first generation value of the first engine for each piece of data in the fourth data, based on the data size of each piece of data in the fourth data and the second time required to process each piece of data in the fourth data.

[0126] In one possible implementation, the performance optimization device determines the first-generation value of the first engine's write processing for each piece of data in the fourth data set, and takes the average value to obtain the final first-generation value, engine1 write cost. The performance optimization device also determines the first-generation value of the first engine's read processing for each piece of data in the fourth data set, and takes the average value to obtain the final first-generation value, engine1 read cost.

[0127] The technical solutions provided by the above embodiments bring at least the following beneficial effects: As can be seen from S407-S410, this application can obtain the second time required for the first engine to process the fourth data, the system can obtain the data processing performance in real time, this application can determine the first generation value based on the second time and the data size of the fourth data, and the system can dynamically adjust and optimize the data processing process to ensure the efficiency of data processing in terms of time and resources.

[0128] In one alternative embodiment, combined with Figure 3 ,like Figure 4As shown, the process of updating the first selection rate of the first engine and the second selection rate of the second engine based on the first-generation value and the second-generation value in S303 can be specifically implemented through the following S411-S412. The details are as follows:

[0129] S411. When the value of the first generation is greater than the value of the second generation, the performance optimization device increases the value of the second selection rate of the second engine.

[0130] In one possible implementation, a first engine and a second engine perform write processing on the first data. When the write cost of the first generation (engine1) is greater than the write cost of the second generation (engine2), it indicates that the first engine takes longer to process the first data. Performance optimization devices can increase the second selectivity value of the second engine.

[0131] For example, the performance optimization device can update the first selectivity (engine1 write select ratio) to 0% and the second selectivity (engine2 write select ratio) to 100%. The selectivity can also be other numbers, and this application does not limit this.

[0132] In one possible implementation, a first engine and a second engine read and process the first data. When the read cost of the first-generation value (engine1) is greater than the read cost of the second-generation value (engine2), it indicates that the first engine takes longer to process the first data. Performance optimization devices can increase the second selectivity value of the second engine.

[0133] For example, the performance optimization device can update the first selectivity engine1 read select ratio to 0% and update the second selectivity engine2 read select ratio to 100%.

[0134] S412. When the value of the first generation is less than the value of the second generation, the performance optimization device increases the first selection rate of the first engine.

[0135] In one possible implementation, a first engine and a second engine write the first data. When the write cost of the first engine (engine1) is less than the write cost of the second engine (engine2), it indicates that the first engine requires less time to process the first data. Performance optimization devices can increase the first engine's selectivity.

[0136] For example, the performance optimization device can update the first selectivity (engine1 write select ratio) to 100% and the second selectivity (engine2 write select ratio) to 0%.

[0137] In one possible implementation, a first engine and a second engine read and process the first data. When the read cost of the first-generation value (engine1) is greater than the read cost of the second-generation value (engine2), it indicates that the first engine requires less time to process the first data. Performance optimization devices can increase the first engine's selectivity.

[0138] For example, the performance optimization device can update the first selectivity (engine1 read select ratio) to 100% and the second selectivity (engine2 read select ratio) to 0%.

[0139] In one possible implementation, after the performance optimization device updates the first and second selection rates, for data smaller than a first threshold, the first engine is directly selected for processing. For data larger than the first threshold, an engine is selected for processing based on the first and second selection rates.

[0140] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S410-S411, this application can dynamically adjust the selection rates of the first engine and the second engine based on the first-generation value and the second-generation value, thereby achieving reasonable allocation and optimization of resources and further improving the effect and efficiency of data processing. By continuously updating the selection rates of the first engine and the second engine, the engines can be adaptively adjusted and the system performance optimized.

[0141] In one alternative embodiment, combined with Figure 3 ,like Figure 5 As shown, this performance optimization method also includes:

[0142] S501. The first selection rate after performance optimization equipment calibration and update is used to determine the third selection rate, and the fourth selection rate is determined by the second selection rate after calibration and update.

[0143] In one possible implementation, the performance optimization device can set both timed correction costs and manual correction costs. The performance optimization device sets timed correction cost parameters and manual correction cost parameters to automatically and / or manually update the selection rates of the first and second engines.

[0144] For example, for read processing, the performance optimization device sets a timed correction cost parameter, `calc read costhour`. The performance optimization device can set at least one integer between 0 and 24; for example, setting it to 10 will execute the correction cost at 10:00 AM. The performance optimization device also sets a manual correction cost parameter, `do read cost hour`. This parameter defaults to 0. When manual correction is required, the performance optimization device sets this parameter to 1, triggering the execution of the correction cost. After triggering, this parameter is reset to 0.

[0145] For example, for write processing, the performance optimization device sets a timed correction cost parameter, `calc write costhour`. The performance optimization device can set at least one integer between 0 and 24, such as 8, 12, and 15, which means the correction cost will be executed at 8:00, 12:00, and 15:00. The performance optimization device also sets a manual correction cost parameter, `do write cost hour`. This parameter defaults to 0. When manual correction is required, the performance optimization device sets this parameter to 1, triggering the execution of the correction cost. After triggering, this parameter is reset to 0.

[0146] In one possible implementation, for read processing, after the performance optimization device triggers the execution of cost correction, the performance optimization device queries the system's current first selectivity (engine1 read select ratio), second selectivity (engine2 read select ratio), first-generation value (engine1 read cost), and second-generation value (engine2 read cost). The performance optimization device can determine a third selectivity by correcting and updating the first selectivity, and determine a fourth selectivity by correcting and updating the second selectivity.

[0147] In one possible implementation, for write processing, after the performance optimization device triggers the execution of cost correction, the performance optimization device queries the system's current first selectivity (engine1 write select ratio), second selectivity (engine2 write select ratio), first-generation value (engine1 write cost), and second-generation value (engine2 write cost). The performance optimization device can determine a third selectivity by correcting and updating the first selectivity, and determine a fourth selectivity by correcting and updating the second selectivity.

[0148] S502, the performance optimization device obtains the fifth data, calls the first engine to select data from the fifth data based on the third selectivity for processing, determines the third generation value, and calls the second engine to select data from the fifth data based on the fourth selectivity for processing, determines the fourth generation value.

[0149] In one possible implementation, the performance optimization device acquires the fifth set of data and invokes the first engine to select data from the fifth set of data based on a third selectivity, then processes the selected data. The performance optimization device obtains the time required for the first engine to process the data, and then determines the third-generation value based on the time and data size.

[0150] S503, the performance optimization device, is based on the third-generation value and the fourth-generation value, updating the first choice rate of the first engine and the second choice rate of the second engine.

[0151] In one possible implementation, the performance optimization device updates the first and second selectivity rates based on third-generation and fourth-generation values, so that the system can perform performance tuning for specific business scenarios, specific system hardware and software configurations, and NVMESSD performance without software upgrades.

[0152] The technical solutions provided by the above embodiments bring at least the following beneficial effects: As can be seen from S501-S503, this application can process the fifth data based on the corrected and determined third and fourth selectivity rates, so that the system can adjust and optimize the data selection and processing process in real time, ensuring that the data is processed faster and more effectively, and adaptively adjusting the engine and optimizing the system performance.

[0153] In one alternative embodiment, combined with Figure 5 ,like Figure 6 As shown, S502 includes S5021 and S5022. S5021 obtains the fifth data by calling the first engine to select data from the fifth data based on the third selectivity for processing and determining the third-generation value. S5022 calls the second engine to select data from the fifth data based on the fourth selectivity for processing and determining the fourth-generation value. This process can be specifically implemented through S601-S606. A detailed explanation follows:

[0154] S601, The performance optimization device calls the second engine to select the sixth data from the fifth data based on the fourth selectivity.

[0155] For example, when the fifth data includes 200 data points and the fourth selection rate is 60%, the performance optimization device randomly selects a sixth data point from the fifth data point based on the fourth selection rate. The sixth data point includes 120 data points.

[0156] S602, The performance optimization device determines the seventh data in the sixth data that has a data size greater than the second threshold.

[0157] In one possible implementation, the performance optimization device can obtain the system's current first selectivity, second selectivity, first-generation value, and second-generation value, and adjust the first threshold of the second engine based on these parameters to determine the second threshold. The performance optimization device can also adjust the splitting unit when the second engine splits data. For example, the performance optimization device determines the second threshold to be 512KB, and identifies a seventh data point in the sixth data set whose size exceeds the second threshold; subsequently, the second engine performs splitting processing on this seventh data point.

[0158] S603, the performance optimization device calls the second engine to split the seventh data and obtain multiple sub-data of the seventh data.

[0159] For example, when the second threshold is 512KB and the splitting unit is 32KB, the performance optimization device splits each piece of data in the seventh data that is larger than 512KB into multiple sub-data pieces with a splitting unit of 32KB, resulting in multiple sub-data pieces for each seventh data piece.

[0160] S604 and the performance optimization device respectively process multiple sub-data of the seventh data.

[0161] In one possible implementation, after the second engine splits the seventh data into multiple sub-data, the second engine sends all the sub-data of the seventh data to the NVMe SSD disk and uses the multi-queue concurrent processing unit of the NVMe SSD to process these sub-data concurrently, thereby speeding up the data processing.

[0162] S605, The performance optimization device obtains the third time required for the second engine to process multiple sub-data of the seventh data.

[0163] In one possible implementation, after the NVMe SSD has finished processing all sub-data, the performance optimization device determines that the data processing was successful. The performance optimization device then obtains a third time required for the second engine to select the sixth data from the fifth data and determine that the processing was successful.

[0164] S606, the performance optimization device determines the fourth-generation value based on the data size of the third time and the seventh data.

[0165] In one possible implementation, the performance optimization device determines the cost of processing each seventh data point by the second engine using Formula 1, based on the time required to process each seventh data point and the size of that seventh data point. The performance optimization device then takes the average of the cost values ​​of all seventh data points as the fourth generation value of the second engine.

[0166] The technical solutions provided by the above embodiments bring at least the following beneficial effects: As can be seen from S601-S606, this application can adjust the second threshold of the second engine to specially process data larger than the second threshold, ensuring that the data is processed faster and more effectively. By dynamically adjusting the second threshold and the selection rate of the second engine, the system can better adapt to different data processing needs and scenarios, enhancing the system's adaptability.

[0167] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] This application embodiment can divide the performance optimization device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0169] like Figure 7 The diagram shown is a structural schematic of a performance optimization device provided in an embodiment of this application. This performance optimization device can be used to perform... Figures 3 to 6 The performance optimization method shown is as follows. Figure 7 The performance optimization device 70 shown includes a communication unit 701 and a processing unit 702.

[0170] The communication unit 701 is used to acquire first data, a first selection rate of a first engine that processes the first data, and a second selection rate of a second engine that processes the first data; wherein the selection rate is used to characterize the probability that the engine selects data from the data for processing; the first engine and the second engine process the data in different ways.

[0171] The processing unit 702 is used to call the first engine to select data from the first data based on a first selectivity for processing to determine a first generation value, and to call the second engine to select data from the first data based on a second selectivity for processing to determine a second generation value; wherein the generation value is used to characterize the time and / or resources required to process the data.

[0172] The processing unit 702 is also used to update the first selection rate of the first engine and the second selection rate of the second engine based on the first generation value and the second generation value.

[0173] Optionally, processing unit 702 is specifically used for: calling the second engine to select second data from the first data based on the second selectivity; processing unit 702 is specifically used for: determining third data in the second data whose data size is greater than the first threshold; processing unit 702 is specifically used for: calling the second engine to split the third data to obtain multiple sub-data of the third data; processing unit 702 is specifically used for: performing processing operations on the multiple sub-data of the third data respectively; communication unit 701 is specifically used for: obtaining the first time required for the second engine to perform processing operations on the multiple sub-data of the third data; processing unit 702 is specifically used for: determining the second generation value based on the first time and the data size of the third data.

[0174] Optionally, the processing unit 702 is specifically used for: calling the first engine to select the fourth data from the first data based on the first selection rate; the processing unit 702 is specifically used for: performing processing operations on the fourth data; the communication unit 701 is specifically used for: obtaining the second time required by the first engine to perform processing operations on the fourth data; the processing unit 702 is specifically used for: determining the first generation value based on the second time and the data size of the fourth data.

[0175] Optionally, the processing unit 702 is specifically used to: increase the value of the second selection rate of the second engine when the value of the first generation is greater than the value of the second generation; and increase the value of the first selection rate of the first engine when the value of the first generation is less than the value of the second generation.

[0176] Optionally, the processing unit 702 is further configured to: correct the updated first selection rate, determine the third selection rate, and correct the updated second selection rate to determine the fourth selection rate; the communication unit 701 is further configured to: acquire fifth data; the processing unit 702 is further configured to: call the first engine to select data from the fifth data based on the third selection rate for processing to determine the third generation value, and call the second engine to select data from the fifth data based on the fourth selection rate for processing to determine the fourth generation value; the processing unit 702 is further configured to: update the first selection rate of the first engine and the second selection rate of the second engine based on the third generation value and the fourth generation value.

[0177] Optionally, processing unit 702 is specifically used for: calling the second engine to select the sixth data from the fifth data based on the fourth selectivity; processing unit 702 is specifically used for: determining the seventh data in the sixth data whose data size is greater than the second threshold; processing unit 702 is specifically used for: calling the second engine to split the seventh data to obtain multiple sub-data of the seventh data; processing unit 702 is specifically used for: performing processing operations on the multiple sub-data of the seventh data respectively; communication unit 701 is specifically used for: obtaining the third time required for the second engine to perform processing operations on the multiple sub-data of the seventh data; processing unit 702 is specifically used for: determining the fourth generation value based on the third time and the data size of the seventh data.

[0178] Optionally, the engine's data processing includes: read processing and / or write processing.

[0179] This application also provides a computer-readable storage medium, which includes computer-executable instructions that, when executed on a computer, cause the computer to perform the performance optimization method provided in the above embodiments.

[0180] This application also provides a computer program that can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, the computer program can implement the performance optimization method provided in the above embodiments.

[0181] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0182] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and other division methods may exist in actual implementation. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate; components shown as units may be one physical unit or multiple physical units, i.e., they may be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the general technology, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0185] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A performance optimization method, characterized in that, include: The system acquires first data, processes the first data using a first engine with a first selection rate, and processes the first data using a second engine with a second selection rate; wherein the selection rate is used to characterize the probability that an engine selects data from the data for processing; the first engine and the second engine process the data in different ways; The process of calling the first engine to select data from the first data based on the first selection rate and processing it to determine the first generation value includes: calling the first engine to select fourth data from the first data based on the first selection rate; performing processing operations on the fourth data; obtaining the second time required for the first engine to perform processing operations on the fourth data; and determining the first generation value based on the second time and the data size of the fourth data. The process of calling the second engine to select data from the first data based on the second selection rate and processing it to determine the second generation value includes: calling the second engine to select second data from the first data based on the second selection rate; identifying third data in the second data whose data size is greater than a first threshold; calling the second engine to split the third data into multiple sub-data; performing processing operations on the multiple sub-data of the third data respectively; obtaining the first time required for the second engine to perform processing operations on the multiple sub-data of the third data; and determining the second generation value based on the first time and the data size of the third data. Wherein, the generation value is used to characterize the time and / or resources required to process the data. Based on the first generation value and the second generation value, update the first selection rate of the first engine and the second selection rate of the second engine, including: when the first generation value is greater than the second generation value, increase the value of the second selection rate of the second engine; when the first generation value is less than the second generation value, increase the value of the first selection rate of the first engine.

2. The method according to claim 1, characterized in that, Also includes: The first selection rate is corrected and updated to determine the third selection rate, and the second selection rate is corrected and updated to determine the fourth selection rate; The process involves obtaining the fifth data, calling the first engine to select data from the fifth data based on the third selection rate for processing to determine the third generation value, and calling the second engine to select data from the fifth data based on the fourth selection rate for processing to determine the fourth generation value. Based on the third-generation value and the fourth-generation value, update the first selection rate of the first engine and the second selection rate of the second engine.

3. The method according to claim 2, characterized in that, The process of calling the second engine to select data from the fifth data based on the fourth selectivity to determine the fourth generation value includes: The second engine is invoked to select the sixth data from the fifth data based on the fourth selectivity; Among the sixth data, the seventh data is determined to be larger than the second threshold. The second engine is invoked to split the seventh data, resulting in multiple sub-data of the seventh data; The processing operations are performed on the multiple sub-data of the seventh data respectively; The third time required for the second engine to process multiple sub-data of the seventh data is obtained; The value of the fourth generation is determined based on the data size of the third time and the seventh data.

4. The method according to any one of claims 1-3, characterized in that, The engine's data processing includes read processing and / or write processing.

5. A performance optimization device, characterized in that, include: Communication unit, processing unit; The communication unit is used to acquire first data, a first selection rate of a first engine processing the first data, and a second selection rate of a second engine processing the first data; wherein, the selection rate is used to characterize the probability that the engine selects data from the data for processing; the first engine and the second engine process the data in different ways; The processing unit is configured to invoke the first engine to select data from the first data based on the first selection rate for processing to determine a first-generation value, including: invoking the first engine to select fourth data from the first data based on the first selection rate; performing processing operations on the fourth data; obtaining the second time required for the first engine to perform processing operations on the fourth data; and determining the first-generation value based on the second time and the data size of the fourth data. The unit also invokes the second engine to select data from the first data based on the second selection rate for processing to determine a second-generation value, including: invoking the second engine to select second data from the first data based on the second selection rate; identifying a third data in the second data whose data size is greater than a first threshold; invoking the second engine to split the third data into multiple sub-data; performing processing operations on the multiple sub-data of the third data respectively; obtaining the first time required for the second engine to perform processing operations on the multiple sub-data of the third data; and determining the second-generation value based on the first time and the data size of the third data. The generation value is used to characterize the time and / or resources required to process the data. The processing unit is further configured to update the first selection rate of the first engine and the second selection rate of the second engine based on the first generation value and the second generation value, including: increasing the value of the second selection rate of the second engine when the first generation value is greater than the second generation value; and increasing the value of the first selection rate of the first engine when the first generation value is less than the second generation value.

6. A performance optimization device, characterized in that, It includes a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory via a bus; when the performance optimization device is running, the processor executes the computer execution instructions stored in the memory, so that the performance optimization device performs the performance optimization method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the performance optimization method as described in any one of claims 1-4.

Images