Message processing method and device
By using the coroutine scheduling strategy on the CPU to judge and process the cache status of data blocks, the timing and resource consumption of prefetching in software forwarding are solved, and the packet processing efficiency and memory access efficiency are improved.
Patent Information
- Application Number
- CN202411001208.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-12-03
AI Technical Summary
In the existing software forwarding technology, prefetch processing has problems such as prefetch timing, resource consumption, memory bandwidth and cache capacity limitations, resulting in low packet processing efficiency.
By using a coroutine scheduling strategy on the CPU, we can determine whether the data block is cached in the preset buffer. If it is not cached, switch to schedule other coroutines to process other messages, and read the data block into the buffer to avoid blind pre-preparation Fetch processing, prefetch processing is only performed when there is a requirement.
It improves the efficiency of message processing, overcomes the limitations of prefetching timing, resource consumption, memory bandwidth and cache capacity, and realizes parallel processing of multiple messages, reducing memory access latency.
Smart Images

Figure CN119065804A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a message processing method and device. Background Art
[0002] Software forwarding (Software Packet Forwarding) refers to a forwarding method that uses the central processing unit (CPU) to forward packets. The software on the CPU is responsible for forwarding the received packets to the correct destination or next hop according to certain rules. Software forwarding is more flexible than hardware forwarding, but its performance is lower than that of hardware forwarding. This is because software forwarding requires the participation of the operating system and is limited by the processing power of the CPU and the memory access speed.
[0003] In software forwarding, memory stalls (i.e., the delay caused by the CPU accessing memory and waiting for the memory response) have a particularly significant impact on performance. In the context of message forwarding, memory stalls may be caused by cache misses. A cache miss means that when the data required by the CPU is not in the cache, the CPU needs to load the data from the main memory, which is much slower than reading the data directly from the cache. To reduce the impact of cache misses, the current forwarding framework mainly uses pre-fetch processing to process messages, that is, the CPU pre-reads the data of the message to be processed into the CPU cache. When the data is actually processed, the data can be directly obtained from the cache, thereby avoiding the delay of accessing the main memory. This pre-fetch processing is usually based on a deep understanding of the message processing flow and can significantly improve the cache hit rate.
[0004] However, the above pre-fetching process has the following difficulties:
[0005] 1) Prefetch timing. It is critical to determine when to prefetch. Prefetching too early may cause data to be evicted from the cache before it is actually used; prefetching too late will not effectively reduce the latency of memory access.
[0006] 2) Resource consumption. Although prefetching can reduce waiting time, it consumes CPU resources. If the prefetching logic is too complex or there are too many prefetching operations, it may affect other CPU operations.
[0007] 3) Memory bandwidth and cache capacity limitations. Prefetching increases the use of memory bandwidth. If memory bandwidth is already a bottleneck, excessive prefetching may exacerbate the bandwidth bottleneck problem. In addition, prefetching is also limited by the size of the cache. Excessive prefetching requests may cause frequent replacement of cache content, reducing efficiency. Summary of the invention
[0008] The purpose of the embodiments of the present application is to provide a message processing method and device to overcome the problems of pre-fetching timing, resource consumption, memory bandwidth and cache capacity limitation in related pre-fetching processing, and improve the efficiency of message processing. The specific technical solution is as follows:
[0009] In a first aspect, an embodiment of the present application provides a message processing method, which is applied to a CPU, and the method includes:
[0010] Scheduling a first coroutine for processing the first message according to a preset scheduling strategy;
[0011] When processing the first data block of the first message by using the first coroutine, determining whether the first data block is cached in a preset cache;
[0012] If the first data block is not cached in the preset buffer, terminating the processing of the first data block by the first coroutine, scheduling a second coroutine for processing the second message, and reading the first data block in the memory into the preset buffer;
[0013] When the first coroutine is scheduled again, the first data block is read from the preset buffer, and the first data block is processed by the first coroutine.
[0014] In some embodiments, the method further comprises:
[0015] If the first data block has been cached in the preset cache, the first data block is read from the preset cache, and the first data block is processed by using the first coroutine.
[0016] In some embodiments, the CPU includes a task queue, and the task queue is used to record the coroutine;
[0017] The step of scheduling the first coroutine for processing the first message according to the preset scheduling strategy specifically includes: scheduling a first number of coroutines in the task queue in sequence starting from the first coroutine in the order in which the coroutines are recorded in the task queue, each coroutine being used as the first coroutine for processing the corresponding first message;
[0018] The scheduling of the second coroutine for processing the second message specifically includes:
[0019] Adding at least one first coroutine that has been terminated to the tail of the task queue;
[0020] At least one coroutine in the task queue that is after the first number of first coroutines is scheduled as a second coroutine, wherein the number of the second coroutines is the same as the number of the at least one terminated first coroutine.
[0021] In some embodiments, before sequentially scheduling the first number of coroutines in the task queue, the method further includes:
[0022] Acquire a first system performance of the CPU and a number of working coroutines matching the first system performance;
[0023] The first number is determined according to the number of working coroutines.
[0024] In some embodiments, after sequentially scheduling the first number of coroutines in the task queue, the method further includes:
[0025] Acquire a second system performance of the CPU;
[0026] The corresponding relationship between the pre-stored system performance and the number of working coroutines is adjusted by using the second system performance, the first number and the Bayesian optimization algorithm.
[0027] In some embodiments, the first coroutine includes a preset keyword and a preset operator;
[0028] The processing of the first data block of the first message by using the first coroutine, determining whether the first data block is cached in a preset cache, specifically includes:
[0029] If the preset keyword is matched and the parameter value of the preset keyword includes the preset operator, it is determined whether the first data block is cached in a preset cache.
[0030] In some embodiments, determining whether the first data block is cached in a preset cache specifically includes:
[0031] Obtaining a first position of the first data block in the memory;
[0032] Determine whether the first position is within a position interval of pre-fetched data, the starting position of the position interval being the position of the first data block when at least one data block of the first message in the memory is read into the preset buffer for the latest time, and the ending position of the position interval being the sum of the amount of data of the first message cached in the preset buffer and the starting position;
[0033] If yes, determining that the first data block has been cached in a preset cache;
[0034] If not, it is determined that the first data block is not cached in the preset cache.
[0035] In some embodiments, the first message is stored in the memory, the first message includes metadata and message content; the metadata includes an identification field, the identification field is used to store the starting position of the location interval, and the message content includes multiple data blocks.
[0036] In a second aspect, an embodiment of the present application provides a message processing device, applied to a CPU, the device comprising: a scheduling module and a judgment module;
[0037] The scheduling module is used to schedule a first coroutine for processing a first message according to a preset scheduling strategy;
[0038] The judgment module is used to judge whether the first data block is cached in a preset buffer when the first data block of the first message is processed by the first coroutine;
[0039] The scheduling module is also used for the scheduling module, and is also used to terminate the processing of the first data block by the first coroutine if the first data block is not cached in the preset cache, schedule the second coroutine to process the second message, and read the first data block in the memory into the preset cache; when the first coroutine is scheduled again, read the first data block from the preset cache, and use the first coroutine to process the first data block.
[0040] In some embodiments, the apparatus further comprises a reading module;
[0041] The reading module is used to read the first data block from the preset buffer if the first data block has been cached in the preset buffer, and process the first data block using the first coroutine.
[0042] In some embodiments, the CPU includes a task queue, and the task queue has a plurality of coroutines recorded therein;
[0043] The scheduling module is specifically used for:
[0044] According to the order of recording the coroutines in the task queue, starting from the coroutine at the top, a first number of coroutines in the task queue are scheduled in sequence, and each coroutine is used as a first coroutine to process the corresponding first message;
[0045] If the corresponding first data block processed by at least one first coroutine is not cached in the preset cache, the processing of the first data block by the at least one first coroutine is terminated; the at least one terminated first coroutine is added to the tail of the task queue; and at least one coroutine in the task queue that is after the first number of first coroutines is scheduled as a second coroutine, and the number of the second coroutines is the same as the number of the at least one terminated first coroutine.
[0046] In some embodiments, the apparatus further comprises:
[0047] an acquisition module, configured to acquire a first system performance of the CPU and a number of working coroutines matching the first system performance before sequentially scheduling a first number of coroutines in the task queue;
[0048] A determination module is used to determine the first number according to the number of working coroutines.
[0049] In some embodiments, the apparatus further comprises an adjustment module;
[0050] The acquisition module is further configured to acquire the second system performance of the CPU after sequentially scheduling the first number of coroutines in the task queue;
[0051] The adjustment module is used to adjust the pre-stored correspondence between the system performance and the number of working coroutines by using the second system performance, the first number and the Bayesian optimization algorithm.
[0052] In some embodiments, the first coroutine includes a preset keyword and a preset operator;
[0053] The judgment module is specifically configured to judge whether the first data block is cached in a preset buffer if the preset keyword is matched and the parameter value of the preset keyword includes the preset operator.
[0054] In some embodiments, the determination module is specifically used to:
[0055] Obtaining a first position of the first data block in the memory;
[0056] Determine whether the first position is within a position interval of pre-fetched data, the starting position of the position interval being the position of the first data block when at least one data block of the first message in the memory is read into the preset buffer for the latest time, and the ending position of the position interval being the sum of the amount of data of the first message cached in the preset buffer and the starting position;
[0057] If yes, determining that the first data block has been cached in a preset cache;
[0058] If not, it is determined that the first data block is not cached in the preset cache.
[0059] In some embodiments, the first message is stored in the memory, the first message includes metadata and message content; the metadata includes an identification field, the identification field is used to store the starting position of the location interval, and the message content includes multiple data blocks.
[0060] In a third aspect, an embodiment of the present application provides a CPU that executes any of the above-mentioned message processing methods.
[0061] Beneficial effects of the embodiments of the present application:
[0062] In the technical solution provided in the embodiment of the present application, when the CPU schedules a data block (such as a first data block) of a message processed by a coroutine (such as a first coroutine), if the first data block is not cached in a preset buffer, it switches to schedule other coroutines (such as a second coroutine) to process data blocks of other messages, and reads the first data block in the memory into the preset buffer; in this way, when the CPU schedules the first coroutine again, the first data block has been read into the preset buffer, and then the CPU can directly read the first data block in the preset buffer for processing. In the embodiment of the present application, the CPU will not blindly perform pre-fetch processing, but will perform pre-fetch processing when the coroutine is used to process the data block and the data block is not in the buffer, that is, the pre-fetch processing will be performed when there is a need for pre-fetch processing. When there are bottlenecks in memory bandwidth and cache capacity, the problems of pre-fetch timing, resource consumption, memory bandwidth and cache capacity in related technologies are effectively overcome.
[0063] In addition, in the embodiment of the present application, when the CPU reads the data block into the cache, the CPU switches to schedule other coroutines to process the message, that is, even if the cache does not hit, the CPU cannot continue to process the data of the message, but will switch the coroutine to process other messages. The CPU achieves the effect of parallel processing of multiple messages by switching coroutines, and the cost of coroutine switching and the cost of CPU accessing the preset cache are much smaller than the cost of CPU accessing the memory. Therefore, when facing the problem of processing multiple messages, the efficiency of message processing is improved as a whole.
[0064] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.
[0066] Figure 1 A schematic diagram of the architecture of a buffer in a device;
[0067] Figure 2 A first flow chart of the message processing method provided in the embodiment of the present application;
[0068] Figure 3 A second flow chart of the message processing method provided in the embodiment of the present application;
[0069] Figure 4 A third flow chart of the message processing method provided in the embodiment of the present application;
[0070] Figure 5 A schematic diagram of coroutine switching provided in an embodiment of the present application;
[0071] Figure 6 A fourth flow chart of the message processing method provided in the embodiment of the present application;
[0072] Figure 7 A fifth flow chart of the message processing method provided in the embodiment of the present application;
[0073] Figure 8 A schematic diagram of a message structure provided in an embodiment of the present application;
[0074] Fig. 9 A sixth flow chart of the message processing method provided in the embodiment of the present application;
[0075] Fig.10 A schematic diagram of the first structure of the message processing device provided in the embodiment of the present application;
[0076] Fig.11 A second structural diagram of the message processing device provided in an embodiment of the present application;
[0077] Fig.12 A schematic diagram of the structure of a network device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0078] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.
[0079] To facilitate understanding, the terms appearing in the embodiments of the present application are explained below.
[0080] Tail Call: Calling another function at the last step of a function's execution.
[0081] Coroutine: A special type of function that can suspend execution at a specific point so that other coroutines can run in the middle. The suspension of execution here can be achieved by using keywords such as co_await and co_return. When a coroutine encounters a co_await expression, the coroutine suspends execution and returns control to the caller; the coroutine can then resume execution at a subsequent point, thus continuing execution from where it left off.
[0082] The C++20 standard introduces support for coroutines. Unlike Go coroutines or Boost coroutines, the C++20 standard coroutines use stackless coroutines. Compared with Go or Boost's stackful coroutines, the switch of stackless coroutines is just an ordinary function call, while stackful coroutines involve resetting various registers in the operating system (OS) thread. In addition, the C++20 standard stipulates that the compiler must perform tail call optimization on the switching of coroutines. Therefore, in the C++20 standard, the cost of switching coroutines is the switching of tail calls, which only takes 2-5 nanoseconds.
[0083] Hardware forwarding: uses hardware such as Application Specific Integrated Circuit (ASIC) or Field-Programmable Gate Array (FPGA) to forward messages.
[0084] Software forwarding: A forwarding method that uses a general-purpose central processing unit (CPU) to forward messages. The software on the CPU is responsible for forwarding received messages to the correct destination or next hop node according to rules such as routing tables. This process involves multiple levels, including the data link layer, network layer, and transport layer.
[0085] Memory stall: The CPU delays accessing memory due to waiting for memory response. Figure 1In the device shown, the CPU can be connected to multiple levels of caches, such as the first level (L1) cache, the second level (L2) cache, and the third level (L3) cache. The purpose of these caches is to reduce the time required for the CPU to access the main memory, thereby improving overall performance. The access speed of each level of cache is as follows: L1 cache access cost: about 0.5-1 nanoseconds; L2 cache access cost: about 2-4 nanoseconds, which is equivalent to the switching cost of the tail call, that is, the switching cost of the coroutine in the C++20 standard; L3 cache access cost: about 10-40 nanoseconds; main memory access cost: about 100 nanoseconds or more.
[0086] Compared with hardware forwarding, software forwarding is more flexible, but has lower performance. This is because software forwarding requires the participation of the operating system and is limited by the processing power of the CPU and the memory access speed. In software forwarding, memory stalls have a particularly significant impact on performance. In the context of packet forwarding, memory stalls may be caused by the following reasons:
[0087] Cache miss: When the data the CPU needs is not in the cache, the CPU needs to load the data from main memory, which is much slower than reading the data directly from the cache.
[0088] Memory bandwidth bottleneck: In a high-speed network environment, the arrival of a large number of messages may cause the memory bandwidth to become a bottleneck, and the CPU needs to wait for the memory controller to complete the reading and writing of data.
[0089] Synchronization operations: In a multithreaded environment, synchronization operations between threads may cause some threads to wait for other threads to release memory resources.
[0090] Memory page fault: When the memory page being accessed is not in physical memory, the operating system needs to load the page from disk into memory, which can cause significant delays.
[0091] Among the reasons that cause memory pauses, the most difficult to tune and solve is the cache miss problem. To reduce the impact of cache misses, the current forwarding framework uses the following two methods to process messages.
[0092] 1) Batch processing. The forwarding framework reduces frequent access to memory by batching messages. When processing messages, the CPU does not process each message one by one, but groups multiple messages into a batch and processes them all at once. This method can significantly reduce the number of times the CPU's instruction cache interacts with the memory, thereby reducing the risk of instruction cache misses.
[0093] 2) Prefetch processing. The forwarding framework utilizes the prefetch function of the CPU. The CPU pre-reads the data of the message to be processed into the CPU cache. When the data is actually processed, it can directly obtain the data from the cache, thus avoiding the delay of accessing the main memory. This prefetch processing is usually based on a deep understanding of the message processing flow and can significantly improve the cache hit rate.
[0094] In the current forwarding framework, prefetch processing is usually done in the form of manual loop unrolling. For example, in a loop with a specified step size (i.e., the distance of prefetch processing), the operation codes corresponding to multiple data in the loop are independently written to achieve loop unrolling. In this way, the operations corresponding to each data can be executed independently, which improves the parallelism of the instruction pipeline. This form of prefetch processing can improve performance in certain scenarios, but it lacks versatility and significantly increases the complexity of the code. Specifically, there are the following problems:
[0095] 1) Code duplication: The prefetch processing logic in the form of manual loop expansion includes a large amount of repeated operation codes, which increases the code volume of the prefetch processing logic and reduces the readability of the code.
[0096] 2) Reduced code maintainability: If the prefetch processing logic needs to be changed, each expanded part in the prefetch processing logic needs to be updated independently, which is prone to errors.
[0097] 3) Hardware Dependency: The distance and timing of prefetch processing need to be adjusted according to the specific behavior of the hardware, which increases the coupling of the code to the specific hardware platform and reduces the portability of the prefetch processing logic.
[0098] 4) Increased difficulty in debugging and testing: More complex code usually means increased difficulty in debugging and testing. Tracking unrolled loops during debugging is more difficult than tracking simple loops.
[0099] 5) Difficulty in performance tuning: Determining the optimal prefetching distance and loop unrolling degree usually requires multiple experiments and performance testing. This process can be very time-consuming, and the optimal configuration may vary for different data sets and hardware platforms.
[0100] In order to solve the above problems, solve the problem of memory pause when the CPU accesses the memory, and improve the efficiency of message processing, the embodiment of the present application provides a message processing method, such as Figure 2 As shown, applied to a CPU, the method comprises the following steps:
[0101] Step S21, scheduling a first coroutine for processing a first message according to a preset scheduling strategy;
[0102] Step S22, when processing the first data block of the first message by using the first coroutine, determining whether the first data block is cached in a preset buffer; if the first data block is not cached in the preset buffer, executing step S23;
[0103] Step S23, terminating the processing of the first data block by the first coroutine, scheduling the second coroutine to process the second message, and reading the first data block in the memory into a preset buffer;
[0104] Step S24, when the first coroutine is scheduled again, the first data block is read from the preset buffer, and the first data block is processed by the first coroutine.
[0105] In the technical solution provided in the embodiment of the present application, when the CPU schedules a data block (such as a first data block) of a message processed by a coroutine (such as a first coroutine), if the first data block is not cached in a preset buffer, it switches to schedule other coroutines (such as a second coroutine) to process data blocks of other messages, and reads the first data block in the memory into the preset buffer; in this way, when the CPU schedules the first coroutine again, the first data block has been read into the preset buffer, and then the CPU can directly read the first data block in the preset buffer for processing. In the embodiment of the present application, the CPU will not blindly perform pre-fetch processing, but will perform pre-fetch processing when the coroutine is used to process the data block and the data block is not in the buffer, that is, the pre-fetch processing will be performed when there is a need for pre-fetch processing. When there are bottlenecks in memory bandwidth and cache capacity, the problems of pre-fetch timing, resource consumption, memory bandwidth and cache capacity in related technologies are effectively overcome.
[0106] In addition, in the embodiment of the present application, when the CPU reads the data block into the cache, the CPU switches to schedule other coroutines to process the message, that is, even if the cache does not hit, the CPU cannot continue to process the data of the message, but will switch the coroutine to process other messages. The CPU achieves the effect of parallel processing of multiple messages by switching coroutines, and the cost of coroutine switching and the cost of CPU accessing the preset cache are much smaller than the cost of CPU accessing the memory. Therefore, when facing the problem of processing multiple messages, the efficiency of message processing is improved as a whole.
[0107] In the above step S21, the forwarding chip receives a message sent by the cloud or other devices and writes the message into the memory. Each time a message is written into the memory, the CPU generates a coroutine corresponding to the received message for processing the message. The coroutine can be a coroutine optimized for tail calls, such as the coroutine of the C++20 standard.
[0108] To facilitate the management of coroutines, the CPU can set a task queue. After generating a coroutine, the CPU adds the coroutine to the task queue. After the message is processed, the CPU can release the coroutine corresponding to the message in the task queue.
[0109] For example, the forwarding chip receives message 1 and writes message 1 to the memory; the CPU generates coroutine 1 corresponding to message 1 and adds coroutine 1 to the task queue. After message 1 is processed, coroutine 1 in the task queue is released; the forwarding chip receives message 2 and writes message 2 to the memory; the CPU generates coroutine 2 corresponding to message 2 and adds coroutine 2 to the task queue. After message 2 is processed, coroutine 2 in the task queue is released.
[0110] The CPU pre-configures the scheduling strategy for coroutines, namely the preset scheduling strategy. The preset scheduling strategy can be to schedule coroutines according to the first-in-first-out principle. For example, the CPU generates coroutine 1 first and then coroutine 2. When the CPU schedules coroutines according to the preset scheduling strategy, it will schedule coroutine 1 first and then coroutine 2.
[0111] The preset scheduling strategy can also be to schedule the coroutines corresponding to the messages in descending order of priority of the messages. For example, the CPU generates coroutine 1 and coroutine 2, coroutine 1 corresponds to message 1, and coroutine 2 corresponds to message 2; the priority of message 1 is lower than the priority of message 2, then the CPU schedules the coroutines according to the preset scheduling strategy, first scheduling coroutine 2, and then scheduling coroutine 1.
[0112] The preset scheduling strategy may also be in other forms, which are not limited thereto.
[0113] When processing a message, the CPU schedules a coroutine, such as a first coroutine, according to a preset scheduling strategy. The CPU schedules the first coroutine, that is, the CPU runs the first coroutine to process data of the first message using the first coroutine.
[0114] In the above step S22, the CPU processes the data of the message in the form of data blocks. For example, the data of message 1 is 64 bytes in total, and the CPU divides the data of message 1 into data blocks 1 to 4 with 16 bytes as one data block, and processes data blocks 1 to 4 in sequence.
[0115] The first data block may be any data block included in the first message. The preset buffer may be the above-mentioned L1 buffer, L2 buffer or L3 buffer.
[0116] When the CPU processes the first data block of the first message using the first coroutine, the CPU determines whether the first data block is cached in a preset buffer. If the judgment result indicates that the first data block is not cached in the preset buffer, that is, the first data block is still stored in the memory, the CPU can execute step S23, switch the called coroutine, and perform pre-fetch processing.
[0117] Among them, the switching called coroutine includes: terminating the first coroutine, scheduling the second coroutine according to the preset scheduling strategy, so that the GPU can use the second coroutine to process the data of the second message. The specific processing method can refer to the above-mentioned use of the first coroutine to process the data of the first message.
[0118] The pre-fetch process includes: reading the first data block in the memory into a preset buffer. Specifically, the CPU reads the data of the first message from the memory, and writes the data of the first message into the preset buffer. Here, the data of the first message read by the CPU from the memory includes the first data block, that is, the amount of data read by the CPU from the memory at one time can be greater than or equal to the size of a data block. If the size of a data block is 16 bytes, the amount of data read by the CPU from the memory at one time can be 64 bytes.
[0119] In the embodiment of the present application, in combination with the coroutine switching and pre-fetch processing, when the CPU schedules the first coroutine again according to the preset scheduling strategy, as in the above step S24, the pre-fetch processing has been completed, that is, the first data block has been cached in the preset buffer, and the CPU can process the first data block from the position point in the first coroutine, and continue to run the first coroutine, that is, read the first data block in the preset buffer, and process the first data block. Before the CPU schedules the first coroutine again, the CPU will schedule other coroutines to process the message data, which makes the CPU as a whole busy. Although the phenomenon of coroutine termination and coroutine switching may occur, each time the coroutine is rescheduled, the coroutine can continue to be executed from the termination position, and the cost of coroutine switching and the cost of the CPU accessing the preset buffer are much less than the cost of the CPU accessing the memory. Therefore, the technical solution provided by the embodiment of the present application improves the overall message processing efficiency of the CPU.
[0120] In order to solve the above problems, solve the problem of memory pause when the CPU accesses the memory, and improve the efficiency of message processing, the embodiment of the present application provides a message processing method, such as Figure 3 As shown, applied to a CPU, the method comprises the following steps:
[0121] Step S31, scheduling a first coroutine for processing a first message according to a preset scheduling strategy;
[0122] Step S32, when using the first coroutine to process the first data block of the first message, determine whether the first data block is cached in a preset cache; if the first data block is not cached in the preset cache, execute step S33; if the first data block is cached in the preset cache, execute step S35.
[0123] Step S33, terminating the processing of the first data block by the first coroutine, scheduling the second coroutine to process the second message, and reading the first data block in the memory into a preset buffer;
[0124] Step S34, when the first coroutine is scheduled again, the first data block is read from the preset buffer, and the first data block is processed by the first coroutine.
[0125] Step S35, reading a first data block from a preset buffer, and processing the first data block using a first coroutine.
[0126] The above steps S31 to S34 are the same as the above steps S21 to S24 and will not be described again here.
[0127] In step S32, when the CPU attempts to process the first data block of the first message, the CPU determines whether the first data block is cached in the preset cache. If the judgment result indicates that the first data block is not cached in the preset cache, the CPU can execute step S33, switch the called coroutine, and perform pre-fetch processing.
[0128] If the judgment result indicates that the first data block has been cached in the preset buffer, the CPU can execute step S35 to read the first data block from the preset buffer and process the first data block using the first coroutine. In this case, there is no need to access the memory, thereby improving the message processing efficiency.
[0129] In some embodiments, the CPU includes a task queue, which is used to record coroutines. The preset scheduling policy may be: scheduling coroutines in the task queue according to the first-in-first-out principle. In order to further improve the message processing efficiency, the CPU may schedule multiple coroutines in parallel. In this case, the embodiment of the present application provides a message processing method, such as Figure 4 As shown, the following steps may be included:
[0130] Step S41, according to the order of recording the coroutines in the task queue, starting from the coroutine at the top, the first number of coroutines in the task queue are scheduled in sequence, and each coroutine is used as the first coroutine to process the corresponding first message;
[0131] Step S42, when processing the first data block of the first message by using the first coroutine, determining whether the first data block is cached in a preset buffer; if not cached in the preset buffer, executing step S43;
[0132] Step S43, terminating the processing of the first data block by the first coroutine;
[0133] Step S44, adding the at least one first coroutine that has been terminated to the tail of the task queue;
[0134] Step S45, scheduling at least one coroutine in the task queue that is after the first number of first coroutines as a second coroutine, wherein the number of the second coroutines is the same as the number of the at least one terminated first coroutine;
[0135] Step S46, reading the first data block processed by at least one first coroutine that has been terminated in the memory into a preset buffer;
[0136] Step S47, when the first coroutine is scheduled again, the first data block is read from the preset buffer, and the first data block is processed by the first coroutine.
[0137] In the technical solution provided by the embodiment of the present application, the CPU schedules multiple coroutines in parallel, that is, multiple coroutines are used in parallel to process the data of the message, which further improves the message processing efficiency. In addition, the coroutines are scheduled according to the first-in-first-out principle, which conforms to the structure of the stack, facilitates the CPU to schedule the coroutines, and reduces the complexity of the solution implementation.
[0138] In the above step S41, the first number can be set according to the system performance of the CPU. For example, when the system performance of the CPU is good, the first number can be set to a larger value; when the system performance of the CPU is poor, the first number can be set to a smaller value. This can avoid the following problems: there are too many concurrent coroutines, and the previously pre-fetched content is replaced by the content pre-fetched by other coroutines; there are too few concurrent coroutines, the pre-fetching process has not been completed, and the previous coroutine is scheduled again, which cannot fully utilize the computing processing power of the CPU, resulting in a decrease in system performance. Among them, system performance may include but is not limited to message throughput (i.e., the number of messages processed within a period of time), processing time of a single data block, etc.
[0139] In order to better solve the above problems, the CPU can pre-store the correspondence between the system performance and the number of working coroutines. The number of working coroutines represents the number of concurrent coroutines. Before executing step S41, the CPU can obtain the first system performance of the CPU and the number of working coroutines matching the first system performance; determine the first number based on the determined number of working coroutines.
[0140] Among them, the number of working coroutines corresponding to the first system performance can be determined according to the correspondence between the pre-stored system performance and the number of working coroutines, and the determined number of working coroutines is the number of working coroutines matching the first system performance. After determining the number of working coroutines, the CPU can directly use the number of working coroutines as the first number, or calculate the product of the number of working coroutines and the preset weight, and use the product result as the first number. The preset weight can be set according to actual needs. For example, the preset weight can be 0.8, 0.9, or 1.1, etc.
[0141] In an embodiment of the present application, the correspondence between system performance and the number of working coroutines can be stored in a table, as shown in Table 1.
[0142] Table 1
[0143] System Performance Number of worker coroutines
[0144] In this case, the CPU can look up the table to obtain a corresponding relationship including the first system performance, or a corresponding relationship including the third system performance, wherein the third system performance is the system performance closest to the first system performance; and determine the first number based on the number of working coroutines included in the corresponding relationship found.
[0145] In an embodiment of the present application, the correspondence between system performance and the number of working coroutines can be represented by a Bayesian optimization algorithm. The Bayesian optimization algorithm is a global optimization method based on the Bayesian statistical principle. The Bayesian optimization algorithm constructs a probability model of the objective function and uses this model to guide the search process to find the optimal solution of the objective function. Using the Bayesian optimization algorithm to represent the correspondence between system performance and the number of concurrent processes can be understood as establishing a performance model, specifically: by monitoring historical data such as system performance under different concurrent numbers of coroutines, using the Bayesian algorithm for performance modeling, and obtaining a performance model, the performance model is used to describe the correspondence between system performance and the number of working coroutines.
[0146] When determining the first number, the CPU uses a Bayesian optimization algorithm and a heuristic expected improvement maximization method based on the currently constructed performance model and the current system performance to select the next number of working coroutines that may achieve better system performance.
[0147] In order to improve the accuracy of the first number and maximize the message processing efficiency, after scheduling the first number of first coroutines, the CPU can also obtain the second system performance of the CPU; and use the second system performance and the first number to adjust the correspondence between the pre-stored system performance and the number of concurrent processes. For example, the CPU can use the second system performance, the first number, and the Bayesian optimization algorithm to adjust the correspondence between the pre-stored system performance and the number of working coroutines.
[0148] In the embodiment of the present application, after the CPU dynamically adjusts the number of working coroutines, new system performance data, such as the second system performance, can be obtained in real time during the actual CPU operation. The CPU uses the newly obtained system performance data to periodically update the performance model to more accurately reflect the relationship between the current system performance and the number of working coroutines, adapt to different network loads and processing requirements, and maximize the message processing capability and efficiency.
[0149] In addition, the CPU uses the Bayesian optimization algorithm to update the performance model, which also has the following advantages: strong adaptability: it can adjust the search strategy according to actual operation feedback and adapt to environmental changes; high sample efficiency: it can still perform effective optimization under limited sample conditions, which is suitable for situations where the cost of sample acquisition is high.
[0150] Here, the periodic duration of updating the performance model can be set according to actual needs. For example, the periodic duration of updating the performance model can be 1 day, 2 days, or 1 week.
[0151] In the above step S41, the CPU calls multiple first coroutines in parallel. When the first data block processed by one or more first coroutines among the multiple first coroutines is not cached in the preset buffer, steps S43 to S46 are executed, specifically: the CPU determines whether the first data block corresponding to each first coroutine is cached in the preset buffer; if not in the preset buffer, the first coroutine is terminated; if in the preset buffer, the first coroutine is continued to be scheduled. When there is a first coroutine whose corresponding first data block is not in the preset cache, the judgment result of the above step S42 is no. In step S43, the first data block to be processed is not cached in at least one first coroutine in the preset cache, and then steps S44 and S45 are executed, that is, at least one first coroutine that has been terminated is added to the tail of the task queue to facilitate the CPU to continue to call other coroutines; at least one coroutine after the first number of first coroutines in the task queue is scheduled as a second coroutine, and the number of second coroutines is the same as the number of at least one first coroutine that has been terminated; in addition, the pre-fetch processing of step S46 is executed to read the first data block in the memory into the preset cache.
[0152] In the embodiment of the present application, for the first coroutine whose first data block to be processed has been cached in the preset buffer, the CPU can continue to schedule and run the first coroutine to process the corresponding first data block. As in the above step S35, the CPU can read the first data block from the preset buffer and use the first coroutine to process the first data block.
[0153] For example, Figure 5As shown in (a), the task queue includes coroutines 1 to coroutine n, and the first number is 3. The CPU schedules the three coroutines at the head of the task queue, namely coroutines 1 to coroutine 3, wherein coroutine 1 corresponds to message 1, coroutine 2 corresponds to message 2, and coroutine 3 corresponds to message 3, that is, the CPU uses coroutine 1 to process message 1, the CPU uses coroutine 2 to process message 2, and the CPU uses coroutine 3 to process message 3. At a certain moment, the CPU uses coroutine 1 to process data block 1 of message 1, uses coroutine 2 to process data block 2 of message 2, and uses coroutine 3 to process data block 3 of message 3, while data block 1 and data block 2 are not cached in the preset buffer, and data block 3 has been cached in the preset buffer, then the CPU terminates coroutine 1 and coroutine 2, and adds coroutine 1 and coroutine 2 to the tail of the task queue, as shown in FIG. Figure 5 As shown in (b) in the figure. The number of terminated coroutines (i.e., coroutine 1 and coroutine 2) is 2, and the CPU schedules the two coroutines after coroutines 1 to 3 in the task queue, i.e., coroutine 4 and coroutine 5. At this time, the CPU continues to use coroutine 3 to process the data block of message 3, and uses coroutine 4 to process message 4 corresponding to coroutine 4, and uses coroutine 5 to process message 5 corresponding to coroutine 5.
[0154] When the CPU terminates several coroutines, it will call several additional coroutines, so that the CPU always schedules the first number of coroutines in parallel, ensuring the efficiency of message processing.
[0155] In the embodiment of the present application, steps S43 to S45 are coroutine switching operations, and step S46 is a pre-fetch processing operation. The coroutine switching operation and the pre-fetch processing operation are not limited in their execution order. Step S47 is the same as the above step S24, and will not be repeated here.
[0156] In some embodiments, the first coroutine may include preset keywords and preset operators. The preset keywords are used to terminate the first coroutine, such as co_await and co_return, and the preset operators are used to indicate the execution of pre-fetch processing, such as [] and ->, and the functions of the operators can be implemented by overloading. Based on the preset keywords and preset operators, the embodiment of the present application provides a message processing method, such as Figure 6 As shown, the following steps may be included:
[0157] Step S61, scheduling a first coroutine for processing a first message according to a preset scheduling strategy;
[0158] Step S62, if a preset keyword is matched and the parameter value of the preset keyword includes a preset operator, determine whether the first data block of the first message processed by the first coroutine is cached in a preset buffer; if the first data block is not cached in the preset buffer, execute step S63;
[0159] Step S63, terminating the processing of the first data block by the first coroutine, scheduling the second coroutine to process the second message, and reading the first data block in the memory into a preset buffer;
[0160] Step S64, when the first coroutine is scheduled again, the first data block is read from the preset buffer, and the first data block is processed by the first coroutine.
[0161] The step S61 is the same as the step S21 , the step S63 is the same as the step S23 , and the step S64 is the same as the step S24 .
[0162] In the above step S62, when the CPU uses the first coroutine to process the first data block of the first message, that is, in the process of scheduling the first coroutine, when the preset keyword is matched, it means that the CPU is processing the first data block of the first coroutine; if the parameter value of the preset keyword includes a preset operator, it is determined whether the first data block is cached in the preset cache.
[0163] For example, the preset keyword is co_await, and the preset operator is []. When the CPU schedules the first coroutine, when scheduling the position of auto result=co_await data[i], the position is the preset keyword position and includes the preset operator []. Therefore, it is determined whether the first data block data[i] of the first message processed by the CPU using the first coroutine is cached in the preset cache.
[0164] Then, subsequent processing is performed based on the judgment result. For example, if the judgment result indicates that data[i] is not cached in the preset cache, the CPU terminates the first coroutine and performs pre-fetch processing to read data[i] into the preset cache. When the CPU schedules the first coroutine again, the CPU can start execution from the position of auto result = co_await data[i]. At this time, the judgment result is that data[i] has been cached in the preset cache, and then reads data[i] in the preset cache and processes data[i].
[0165] In the technical solution provided by the embodiment of the present application, after the CPU uses the coroutine, the coroutine can be suspended and resumed without manually expanding the loop, which greatly simplifies the code structure and improves readability. In addition, the CPU can overload operators and integrate the pre-fetch processing mechanism into the coroutine. When these operators are called, the CPU uses preset keywords to terminate the coroutine and enables the pre-fetch processing mechanism of the operator overload to achieve asynchronous waiting for the pre-fetch processing to complete, so that other coroutines can run in the middle. Through coroutines and overloaded operators, data pre-fetching and data processing logic are separated, making the code easier to maintain and expand.
[0166] In some embodiments, the present application provides a message processing method, such as Figure 7 As shown, the following steps may be included:
[0167] Step S71, scheduling a first coroutine for processing a first message according to a preset scheduling strategy;
[0168] Step S72, when processing the first data block of the first message by using the first coroutine, obtaining a first position of the first data block in the memory;
[0169] Step S73, determining whether the first position is within a position interval of pre-fetched data, the starting position of the position interval being the position of the first data block when at least one data block of the first message in the memory is read into the preset buffer for the latest time, and the ending position of the position interval being the sum of the data volume of the first message cached in the preset buffer and the starting position; if not within the position interval, executing step S74; if within the position interval, executing step S76;
[0170] Step S74, terminating the processing of the first data block by the first coroutine, scheduling the second coroutine to process the second message, and reading the first data block in the memory into a preset buffer;
[0171] Step S75, when the first coroutine is scheduled again, the first data block is read from the preset buffer, and the first data block is processed by the first coroutine;
[0172] Step S76, reading a first data block from a preset buffer, and processing the first data block using a first coroutine.
[0173] The step S71 is the same as the step S21, the step S74 is the same as the step S23, the step S75 is the same as the step S24, and the step S76 is the same as the step S35.
[0174] In the above step S72, when the CPU processes the first data block of the first message using the first coroutine, the CPU obtains the position of the first data block in the memory, that is, the first position. For example, the position of the first data block data[i] in the memory is i×Q, Q is the size of a single data block, i=0,1,2,…,n.
[0175] In step S73, the location interval of the pre-fetched data indicates the address interval of the data cached in the current preset buffer in the memory. The starting position of the location interval is: the position of the first data block in the latest pre-fetched data, and the pre-fetched data is the data read once from the memory to the preset buffer.
[0176] LastPerfetchPosition represents the starting position of the prefetch processing, that is, the starting position of the location interval of the prefetched data. CPU_Cache_Size represents the data amount of the first message cached in the preset cache, that is, the amount of data read by the CPU once. CPU_Cache_Size is a fixed value. The location interval of the prefetched data can be expressed as LastPerfetchPosition~LastPerfetchPosition+CPU_Cache_Size.
[0177] After obtaining the first position, the CPU can determine whether the first position is within the position interval of the pre-fetched data. If the first position is not within the position interval of the pre-fetched data, it is determined that the first data block is not cached in the preset cache, and step S74 is executed; if the first position is within the position interval of the pre-fetched data, it is determined that the first data block is cached in the preset cache, and step S76 is executed.
[0178] In the embodiment of the present application, in order to ensure accurate processing of message data, after each pre-fetching process, the CPU may update the location interval of the pre-fetched data.
[0179] When the CPU determines that the first data block is not in the preset buffer, it starts to perform pre-fetch processing, that is, executes step S74 to read the first data block in the memory into the preset buffer. At this time, the first data block is the first data block read this time.
[0180] In one example, the CPU can record the location interval of the pre-fetched data of each message. After executing step S74, the CPU updates the starting position of the location interval to the first position of the first data block; and updates the ending position of the location interval to the sum of the first position and the data amount of the first message cached in the preset buffer.
[0181] In another example, the first message is stored in the memory, and the first message includes metadata and message content, such as Figure 8 As shown; the metadata includes an identification field, the identification field is used to store the starting position of the position interval, and the message content includes multiple data blocks. After executing step S74, the CPU can update the starting position stored in the identification field included in the metadata to the first position of the first data block. Among them, the metadata can include an input interface, an output interface, a source address, a destination address, and a transmission protocol. After updating the starting position, the CPU can recalculate a new position interval.
[0182] In the embodiment of the present application, the CPU records the starting position in the memory, reducing the burden on the CPU storage space.
[0183] To facilitate the implementation of the message processing method provided in the embodiment of the present application, a scheduler can be configured in the CPU, which is responsible for managing the coroutines waiting for the pre-fetching process to be completed, adding the coroutines waiting for the pre-fetching process to be completed to the tail of the scheduler, and determining the next coroutine to be run according to the scheduling strategy to ensure that the system runs at the optimal level. In order to determine the optimal number of coroutines running in parallel, the scheduler can use a Bayesian optimization algorithm to periodically sample system performance information to adjust the optimal number of coroutines running in parallel, that is, adjust the concurrency level. The scheduler is equivalent to a task queue.
[0184] Below through Fig. 9 The message processing flow shown explains the message processing method provided in the embodiment of the present application.
[0185] Step S91, the CPU schedules a first number of coroutines in the task queue in sequence starting from the first coroutine according to the order in which the coroutines are recorded in the task queue, and each coroutine is used as a first coroutine to process the corresponding first message;
[0186] Step S92, when using each first coroutine to process the first data block of the first message, if a preset keyword is matched and the parameter value of the preset keyword includes a preset operator, the CPU obtains the first position of the first data block processed by the first coroutine in the memory;
[0187] Step S93, the CPU determines whether the first position is within the position interval of the pre-fetched data; if not, step S94 is executed; if within the position interval, step S98 is executed;
[0188] Step S94, the CPU terminates at least one first coroutine of the first data block that is not cached in the preset buffer, and adds the terminated at least one first coroutine to the tail of the task queue;
[0189] Step S95, the CPU schedules at least one coroutine in the task queue after the first number of first coroutines as a second coroutine, and the number of the second coroutines is the same as the number of the at least one terminated first coroutine;
[0190] Step S96, the CPU reads the first data block processed by at least one terminated coroutine in the memory into a preset buffer;
[0191] In the embodiment of the present application, the execution order of step S94, step S95 and step S96 is not limited, as long as step S94, step S95 and step S96 are executed when the judgment result of step S93 is no, and step S97 is executed after step S94, step S95 and step S96 are executed.
[0192] Step S97, when the first coroutine is scheduled again, the CPU reads the first data block from the preset buffer and processes the first data block using the first coroutine;
[0193] In step S98 , the CPU reads the first data block from the preset buffer and processes the first data block using the first coroutine.
[0194] In an embodiment of the present application, when the CPU schedules a first number of first coroutines in parallel, the CPU can execute the above steps S91 to S98 for each first coroutine. For example, for each coroutine, when the first data block of the first message is processed by the first coroutine, if a preset keyword is matched, and the parameter value of the preset keyword includes a preset operator, the CPU obtains the first position of the first data block processed by the first coroutine in the memory; if the first position is not within the corresponding position interval, the first coroutine is terminated and the first coroutine is added to the tail of the task queue; a second coroutine is scheduled to process the second message; when the first coroutine is scheduled again, the CPU reads the first data block from the preset buffer and uses the first coroutine to process the corresponding first data block.
[0195] The description of the above steps S91 to S98 is relatively simple, please refer to the above Figure 2 to Figure 8 Part of the description.
[0196] The technical solution provided in the embodiment of the present application can bring the following beneficial effects by combining the coroutine characteristics of the C++20 standard and the Bayesian optimization algorithm:
[0197] 1. Performance improvement: The non-blocking prefetch processing mechanism implemented by coroutines effectively reduces memory access latency, reduces the possibility of memory pauses, and improves the performance of software forwarding.
[0198] 2. Optimization of resource utilization: Using the Bayesian optimization algorithm, the system can dynamically adjust the number of concurrent coroutines to achieve efficient use of CPU resources, avoiding resource waste and excessive competition.
[0199] 3. Automation and intelligence: With the adaptive characteristics of the Bayesian optimization algorithm, the system can intelligently adjust the system configuration based on real-time performance feedback data, reducing the need for manual intervention and improving the intelligence level of the system.
[0200] 4. Improved code simplicity and maintainability: By using coroutines and operator overloading, the traditional manual optimization code is simplified, and the simplicity and maintainability of the code are improved.
[0201] 5. Reduce the workload of manual tuning: Traditional pre-fetch processing technology requires manual tuning according to specific hardware and workload, while the technical solution provided by the embodiment of the present application reduces the workload of such manual tuning, making performance optimization easier and more predictable.
[0202] 6. Scalability: The lightweight nature of coroutines makes the system easier to expand and able to handle larger network loads while maintaining high performance.
[0203] 7. Strong robustness: Adaptive scheduling strategies can better cope with network traffic fluctuations, ensuring that the system can maintain a high performance level even in an unstable network environment.
[0204] Corresponding to the above-mentioned message processing method, the embodiment of the present application also provides a message processing device, such as Fig.10 As shown, it is applied to a CPU and includes: a scheduling module 101 and a judgment module 102;
[0205] The scheduling module 101 is used to schedule a first coroutine for processing a first message according to a preset scheduling strategy;
[0206] A judgment module 102, configured to judge whether the first data block is cached in a preset buffer when processing the first data block of the first message by using the first coroutine;
[0207] The scheduling module 101 is also used to terminate the processing of the first data block by the first coroutine if the first data block is not cached in the preset cache, schedule the second coroutine to process the second message, and read the first data block in the memory into the preset cache; when the first coroutine is scheduled again, read the first data block from the preset cache, and use the first coroutine to process the first data block.
[0208] In some embodiments, Fig.11 As shown, the message processing device may further include a reading module 103;
[0209] The reading module 103 is configured to read the first data block from the preset buffer if the first data block has been buffered in the preset buffer, and process the first data block using the first coroutine.
[0210] In some embodiments, the CPU includes a task queue, and the task queue has a plurality of coroutines recorded therein;
[0211] The scheduling module 101 may be specifically used for:
[0212] According to the order of recording the coroutines in the task queue, starting from the coroutine at the top, the first number of coroutines in the task queue are scheduled in sequence, and each coroutine is used as the first coroutine to process the corresponding first message;
[0213] If the corresponding first data block processed by at least one first coroutine is not cached in a preset cache, the processing of the first data block by at least one first coroutine is terminated; the at least one terminated first coroutine is added to the tail of the task queue; and at least one coroutine in the task queue that is after the first number of first coroutines is scheduled as a second coroutine, and the number of second coroutines is the same as the number of at least one terminated first coroutine.
[0214] In some embodiments, the message processing device may further include:
[0215] An acquisition module, used for acquiring a first system performance of the CPU and a number of working coroutines matching the first system performance before sequentially scheduling a first number of coroutines in the task queue;
[0216] The determination module is used to determine the first quantity according to the number of working coroutines.
[0217] In some embodiments, the message processing device may further include an adjustment module;
[0218] The acquisition module is further used to acquire the second system performance of the CPU after sequentially scheduling the first number of coroutines in the task queue;
[0219] The adjustment module is used to adjust the correspondence between the pre-stored system performance and the number of working coroutines by using the second system performance, the first number and the Bayesian optimization algorithm.
[0220] In some embodiments, the first coroutine includes a preset keyword and a preset operator;
[0221] The judgment module 102 is specifically configured to judge whether the first data block is cached in a preset buffer if a preset keyword is matched and the parameter value of the preset keyword includes a preset operator.
[0222] In some embodiments, the determination module 102 is specifically configured to:
[0223] Obtaining a first position of a first data block in memory;
[0224] Determine whether the first position is within a position interval of pre-fetched data, the starting position of the position interval being the position of the first data block when at least one data block of the first message in the memory is read into the preset buffer for the latest time, and the ending position of the position interval being the sum of the amount of data of the first message cached in the preset buffer and the starting position;
[0225] If yes, it is determined that the first data block has been cached in the preset cache;
[0226] If not, it is determined that the first data block is not cached in the preset cache.
[0227] In some embodiments, a first message is stored in the memory, the first message includes metadata and message content; the metadata includes an identification field, the identification field is used to store the starting position of the location interval, and the message content includes multiple data blocks.
[0228] In the technical solution provided in the embodiment of the present application, when the CPU schedules a data block (such as a first data block) of a message processed by a coroutine (such as a first coroutine), if the first data block is not cached in a preset buffer, it switches to schedule other coroutines (such as a second coroutine) to process data blocks of other messages, and reads the first data block in the memory into the preset buffer; in this way, when the CPU schedules the first coroutine again, the first data block has been read into the preset buffer, and then the CPU can directly read the first data block in the preset buffer for processing. In the embodiment of the present application, the CPU will not blindly perform pre-fetch processing, but will perform pre-fetch processing when the coroutine is used to process the data block and the data block is not in the buffer, that is, the pre-fetch processing will be performed when there is a need for pre-fetch processing. When there are bottlenecks in memory bandwidth and cache capacity, the problems of pre-fetch timing, resource consumption, memory bandwidth and cache capacity in related technologies are effectively overcome.
[0229] In addition, in the embodiment of the present application, when the CPU reads the data block into the cache, the CPU switches to schedule other coroutines to process the message, that is, even if the cache does not hit, the CPU cannot continue to process the data of the message, but will switch the coroutine to process other messages. The CPU achieves the effect of parallel processing of multiple messages by switching coroutines, and the cost of coroutine switching and the cost of CPU accessing the preset cache are much smaller than the cost of CPU accessing the memory. Therefore, when facing the problem of processing multiple messages, the efficiency of message processing is improved as a whole.
[0230] Corresponding to the above-mentioned message processing method, an embodiment of the present application also provides a CPU, which executes any of the above-mentioned message processing methods.
[0231] Corresponding to the above-mentioned message processing method, the embodiment of the present application also provides a network device, such as Fig.12 As shown, it includes a CPU 121 and a machine-readable storage medium 122, wherein the machine-readable storage medium 122 stores machine-executable instructions that can be executed by the CPU 121, and the CPU 121 is prompted by the machine-executable instructions to implement any of the above-mentioned message processing methods.
[0232] The machine-readable storage medium may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk storage. Optionally, the machine-readable storage medium may also be at least one storage device located away from the aforementioned processor.
[0233] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a CPU, any of the above-mentioned message processing methods is implemented.
[0234] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any message processing method in the above embodiments.
[0235] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.
[0236] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0237] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, CPU, network device, storage medium and program product embodiments, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0238] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.
Claims
1. A message processing method, characterized in that: Applied to a CPU, the method comprises: Scheduling a first coroutine for processing the first message according to a preset scheduling strategy; When processing the first data block of the first message by using the first coroutine, determining whether the first data block is cached in a preset cache; If the first data block is not cached in the preset buffer, terminating the processing of the first data block by the first coroutine, scheduling a second coroutine for processing the second message, and reading the first data block in the memory into the preset buffer; When the first coroutine is scheduled again, the first data block is read from the preset buffer, and the first data block is processed by the first coroutine.
2. The method according to claim 1, characterized in that: The method further comprises: If the first data block has been cached in the preset cache, the first data block is read from the preset cache, and the first data block is processed by using the first coroutine.
3. The method according to claim 1, characterized in that The CPU includes a task queue, and the task queue is used to record the coroutine; Scheduling the first coroutine for processing the first message according to the preset scheduling strategy specifically includes: According to the order of recording the coroutines in the task queue, starting from the coroutine at the top, a first number of coroutines in the task queue are scheduled in sequence, and each coroutine is used as a first coroutine to process the corresponding first message; The scheduling of the second coroutine for processing the second message specifically includes: Adding at least one first coroutine that has been terminated to the tail of the task queue; At least one coroutine in the task queue that is after the first number of first coroutines is scheduled as a second coroutine, and the number of the at least one second coroutine that has been scheduled is the same as the number of the at least one first coroutine that has been terminated.
4. The method according to claim 3, characterized in that Before sequentially scheduling the first number of coroutines in the task queue, the method further includes: Acquire a first system performance of the CPU and a number of working coroutines matching the first system performance; The first number is determined according to the number of working coroutines.
5. The method according to claim 4, characterized in that After sequentially scheduling the first number of coroutines in the task queue, the method further includes: Acquire a second system performance of the CPU; The corresponding relationship between the pre-stored system performance and the number of working coroutines is adjusted by using the second system performance, the first number and the Bayesian optimization algorithm.
6. The method according to claim 1, characterized in that The first coroutine includes a preset keyword and a preset operator; The processing of the first data block of the first message by using the first coroutine, determining whether the first data block is cached in a preset cache, specifically includes: If the preset keyword is matched and the parameter value of the preset keyword includes the preset operator, it is determined whether the first data block is cached in a preset cache.
7. The method according to claim 1, characterized in that The determining whether the first data block is cached in a preset cache specifically includes: Obtaining a first position of the first data block in the memory; Determine whether the first position is within a position interval of pre-fetched data, the starting position of the position interval being the position of the first data block when at least one data block of the first message in the memory is read into the preset buffer for the latest time, and the ending position of the position interval being the sum of the amount of data of the first message cached in the preset buffer and the starting position; If yes, determining that the first data block has been cached in a preset cache; If not, it is determined that the first data block is not cached in the preset cache.
8. The method according to claim 7, characterized in that The first message is stored in the memory, and the first message includes metadata and message content; the metadata includes an identification field, and the identification field is used to store the starting position of the position interval, and the message content includes multiple data blocks.
9. A message processing device, characterized in that: Applied to a CPU, the device comprises: a scheduling module and a judgment module; The scheduling module is used to schedule a first coroutine for processing a first message according to a preset scheduling strategy; The judgment module is used to judge whether the first data block is cached in a preset buffer when the first data block of the first message is processed by the first coroutine; The scheduling module is also used to terminate the processing of the first data block by the first coroutine if the first data block is not cached in the preset buffer, schedule the second coroutine to process the second message, and read the first data block in the memory into the preset buffer; when the first coroutine is scheduled again, read the first data block from the preset buffer, and use the first coroutine to process the first data block.
10. The device according to claim 9, characterized in that The device also includes a reading module; The reading module is used to read the first data block from the preset buffer if the first data block has been cached in the preset buffer, and process the first data block using the first coroutine.
11. The device according to claim 9, characterized in that The CPU includes a task queue, and a plurality of coroutines are recorded in the task queue; The scheduling module is specifically used for: According to the order of recording the coroutines in the task queue, starting from the coroutine at the top, a first number of coroutines in the task queue are scheduled in sequence, and each coroutine is used as a first coroutine to process the corresponding first message; If the corresponding first data block processed by at least one first coroutine is not cached in the preset buffer, terminating the processing of the first data block by the at least one first coroutine; Add at least one terminated first coroutine to the tail of the task queue; schedule at least one coroutine in the task queue after the first number of first coroutines as a second coroutine, wherein the number of the second coroutines is the same as the number of the at least one terminated first coroutine.
12. The device according to claim 11, characterized in that The device also includes: an acquisition module, configured to acquire a first system performance of the CPU and a number of working coroutines matching the first system performance before sequentially scheduling a first number of coroutines in the task queue; A determination module is used to determine the first number according to the number of working coroutines.
13. The device according to claim 12, characterized in that The device also includes an adjustment module; The acquisition module is further configured to acquire the second system performance of the CPU after sequentially scheduling the first number of coroutines in the task queue; The adjustment module is used to adjust the pre-stored correspondence between the system performance and the number of working coroutines by using the second system performance, the first number and the Bayesian optimization algorithm.
14. The device according to claim 9, characterized in that The first coroutine includes a preset keyword and a preset operator; The judgment module is specifically configured to judge whether the first data block is cached in a preset buffer if the preset keyword is matched and the parameter value of the preset keyword includes the preset operator.
15. The device according to claim 9, characterized in that The judgment module is specifically used for: Obtaining a first position of the first data block in the memory; Determine whether the first position is within a position interval of pre-fetched data, the starting position of the position interval being the position of the first data block when at least one data block of the first message in the memory is read into the preset buffer for the latest time, and the ending position of the position interval being the sum of the amount of data of the first message cached in the preset buffer and the starting position; If yes, determining that the first data block has been cached in a preset cache; If not, it is determined that the first data block is not cached in the preset cache.
16. The device according to claim 15, characterized in that The first message is stored in the memory, and the first message includes metadata and message content; the metadata includes an identification field, and the identification field is used to store the starting position of the position interval, and the message content includes multiple data blocks.
