Communication data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a communication data processing method, a communication data processing device, electronic equipment and a computer readable storage medium, and belongs to the technical field of communication. The method comprises the following steps: determining the data size of to-be-processed data included in a target process participating in communication; determining a data classification result of the to-be-processed data according to the data size of the to-be-processed data; and determining a memory processing strategy of the to-be-processed data according to the data classification result, and processing the to-be-processed data by adopting the memory processing strategy. According to the invention, the data can be accurately and reasonably processed according to the size of the data in the communication process.

Description

Background Art

[0002] In high-performance computing applications, inter-process communication is one of the important factors in optimizing application performance. The most commonly used inter-process communication model is MPI (Message Passing Interface), which provides a standardized communication mechanism for processes in distributed systems. In MPI, processes exchange data by sending and receiving messages.

[0003] In the existing technology, in the MPI implementation using shared memory, a shared memory segment is often created and the relevant data is loaded into the memory segment. Other processes can directly access the shared data by mapping the shared memory segment without going through the network stack. However, this method still has major limitations, such as memory management overhead and synchronization complexity, which affect the performance of the application.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a communication data processing method, a communication data processing device, an electronic device and a computer-readable storage medium, thereby overcoming, at least to a certain extent, the problem that the prior art cannot effectively manage data memory during the communication process.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a communication data processing method is provided, including: determining the data size of the data to be processed included in the target process participating in the communication; determining the data classification result of the data to be processed according to the data size of the data to be processed; determining the memory processing strategy of the data to be processed according to the data classification result, and adopting the memory processing strategy to process the data to be processed.

[0008] According to one aspect of the present disclosure, a communication data processing device is provided, including: a data size determination module, used to determine the data size of the data to be processed included in the target process participating in the communication; a classification result determination module, used to determine the data classification result of the data to be processed according to the data size of the data to be processed; a processing strategy determination module, used to determine the memory processing strategy of the data to be processed according to the data classification result, and adopt the memory processing strategy to process the data to be processed.

[0009] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods by executing the executable instructions.

[0010] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the above methods is implemented.

[0011] The exemplary embodiments of the present disclosure have the following beneficial effects:

[0012] Determine the data size of the data to be processed included in the target process participating in the communication; determine the data classification result of the data to be processed according to the data size of the data to be processed; determine the memory processing strategy of the data to be processed according to the data classification result, and adopt the memory processing strategy to process the data to be processed. On the one hand, this exemplary embodiment proposes a new communication data processing method. Compared with the prior art that directly processes the data in the communication process, this exemplary embodiment can provide a variety of memory processing strategies for the data in the communication process according to the size of the data in the process, which can improve the effectiveness of data processing while ensuring communication efficiency; on the other hand, this exemplary embodiment can determine different classification results according to the data size of the data to be processed, and then determine the memory processing strategy of the data to be processed according to the classification result. The process is simple and convenient and can be applied to a variety of communication scenarios.

[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0015] Figure 1 A flow chart schematically illustrates a method for processing communication data in this exemplary embodiment;

[0016] Figure 2 A sub-flow chart schematically illustrates a communication data processing method in this exemplary embodiment;

[0017] Figure 3 A block diagram schematically shows a structure of a communication data processing device in this exemplary embodiment;

[0018] Figure 4 An electronic device for implementing the above method in this exemplary embodiment is schematically shown. DETAILED DESCRIPTION

[0019] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0020] An exemplary embodiment of the present disclosure first provides a communication data processing method.

[0021] The following is combined with Figure 1 This exemplary embodiment is further described as follows. Figure 1 As shown, the communication data processing method may include the following steps S110 to S130:

[0022] Step S110, determining the data size of the to-be-processed data included in the target process participating in the communication.

[0023] When communicating, information can be transmitted between processes to exchange information. MPI is a widely used standard for communicating between processes in a parallel computing environment. MPI allows multiple processes to exchange data through messages. These processes can run on the same computer or distributed on different computers in a network. Due to its high efficiency and flexibility. In this exemplary embodiment, the communication involved can be ALL-to-all communication.

[0024] The target process may be any process in the communication process, for example, one or more selected processes, or the current process, etc. This exemplary embodiment may obtain multiple processes participating in the communication, and determine the data size of the data to be processed included in each process. The data to be processed may refer to information to be transmitted or interacted in the process, such as data that the process needs to send, share, or give access to other processes. In this exemplary embodiment, the process may copy the data to be processed to a designated memory so that other processes can access the shared data, etc. The data size of the data to be processed refers to the size of an indicator that can reflect the amount of data, and can usually be measured in KB as the size unit.

[0025] This exemplary embodiment can obtain the data size of the to-be-processed data at the same time when acquiring the target process of communication, or estimate the data volume based on the acquired to-be-processed data to determine the data size, etc.

[0026] Step S120: determining a data classification result of the data to be processed according to the data size of the data to be processed.

[0027] Then, the data to be processed can be divided into different categories according to the data size of the data to be processed, that is, the data classification result. In this exemplary embodiment, the data classification result can be used to reflect the classification of data size, and the specific category definition may not be limited. For example, it can be a big data category, a small data category, or a first data category, a second data category, etc.

[0028] The data classification result of the data to be processed can be determined according to the data size of the data to be processed. The judgment can be made according to a preset threshold and data size, or according to a dynamically adjusted threshold and data size. The data size can also be judged according to other rules, etc. For example, the data to be processed can be input into a pre-trained classification model to output the data classification result.

[0029] Step S130, determining a memory processing strategy for the data to be processed according to the data classification result, and using the memory processing strategy to process the data to be processed.

[0030] After determining the data classification results of the data to be processed, this exemplary embodiment can determine different memory processing strategies according to different data classification results, and adopt appropriate memory processing strategies for different data to be processed to perform data processing. For data to be processed with a small amount of data, it can be directly copied to the temporary memory. For data to be processed with a large amount of data, it can be copied to the temporary memory in stages according to a preset cycle, or part of the data can be copied to the temporary memory and the other part of the data can be copied to the non-temporary memory, and so on.

[0031] Based on the above description, in this exemplary embodiment, the data size of the data to be processed included in the target process participating in the communication is determined; according to the data size of the data to be processed, the data classification result of the data to be processed is determined; according to the data classification result, the memory processing strategy of the data to be processed is determined, and the data to be processed is processed using the memory processing strategy. On the one hand, this exemplary embodiment proposes a new communication data processing method. Compared with the prior art that directly processes the data in the communication process, this exemplary embodiment can provide a variety of memory processing strategies for the data in the communication process according to the size of the data in the process, which can improve the effectiveness of data processing while ensuring communication efficiency; on the other hand, this exemplary embodiment can determine different classification results according to the data size of the data to be processed, and then determine the memory processing strategy of the data to be processed according to the classification result. This process is simple and convenient and can be applied to a variety of communication scenarios.

[0032] In an exemplary embodiment, if Figure 2As shown, the communication data processing method may further include the following steps:

[0033] Step S210, determining a data classification threshold;

[0034] The above step S120, determining the data classification result of the data to be processed according to the data size of the data to be processed, may include:

[0035] Step S220, comparing the data size of the data to be processed with the data classification threshold, and determining the data classification result of the data to be processed according to the comparison result.

[0036] In this exemplary embodiment, a data classification threshold can be determined, and the data classification result of the data to be processed can be determined based on the data classification threshold. Specifically, the data size of the data to be processed can be compared with the data classification threshold, and the corresponding data classification result can be determined according to the comparison result. The data classification threshold can be one or more. For example, by setting a data classification threshold, data less than the data classification threshold can be determined as first-category data, and data greater than or equal to the data classification threshold can be determined as second-category data; or by setting two data classification thresholds, the data classification result can include first-category data less than the first threshold, second-category data greater than the first threshold and less than the second threshold, and third-category data greater than the second threshold, and so on.

[0037] In an exemplary embodiment, the above-mentioned comparing the data size of the to-be-processed data with the data classification threshold and determining the data classification result of the to-be-processed data according to the comparison result may include:

[0038] If the data size of the data to be processed is smaller than the data classification threshold, the data to be processed is determined to be first-category data;

[0039] If the data size of the data to be processed is greater than or equal to the data classification threshold, it is determined that the data to be processed is the second category of data.

[0040] When the data size of the data to be processed is less than the data classification threshold, it can be considered that the data volume of the current data to be processed is relatively small, and it can be determined as the first type of data, and the first type of data can be regarded as a data classification result of a small data volume; when the data size of the data to be processed is greater than or equal to the data classification threshold, it can be considered that the data volume of the current data to be processed is relatively large, and it can be determined as the second type of data. For the first type of data and the second type of data, this exemplary embodiment can provide different memory processing strategies.

[0041] In an exemplary embodiment, if the data to be processed is the first type of data, determining the memory processing strategy of the data to be processed according to the data classification result, and using the memory processing strategy to process the data to be processed may include:

[0042] The data to be processed is copied to the first memory using a standard data processing method.

[0043] Among them, the standard data processing method can be a conventional memcpy processing method. Memcpy is a memory copy function used in C and C++, which can copy several bytes from the starting position of the source memory address to the target memory address. That is, this exemplary embodiment can adopt the memcpy processing method to copy the first type of data to the first memory. The first memory can be a pre-configured storage area for storing small data volumes. In this exemplary embodiment, the first memory can be a temporary cache area.

[0044] In an exemplary embodiment, if the data to be processed is the second type of data, determining the memory processing strategy of the data to be processed according to the data classification result, and using the memory processing strategy to process the data to be processed may include:

[0045] Dividing the data to be processed into a first part of data and a second part of data;

[0046] The first portion of data is copied to the first memory, and the second portion of data is copied to the second memory.

[0047] When the data to be processed is the second type of data, that is, the amount of data to be processed is large, directly copying the data to the first memory will increase the memory management overhead and cause performance degradation. At the same time, since a large amount of data is located in the first memory, multiple processes may share the memory simultaneously. In order to ensure data consistency, locks or other synchronization mechanisms need to be introduced, which will increase programming complexity and may cause performance bottlenecks.

[0048] Based on this, when determining that the data to be processed is the second type of data, this exemplary embodiment can divide the data to be processed into a first part of data and a second part of data, and then copy the first part of data to the first memory, and copy the second part of data to the second memory. The data division standard can be determined according to the size of the first memory and / or the second memory, for example, the data less than or equal to the first memory can be used as the first part of data, and the remaining data can be used as the second part of data; the data to be processed can also be divided into the first part of data and the second part of data according to the size ratio of the first memory and the second memory, etc., to ensure the balance of data division.

[0049] The second memory may be a memory area different from the first memory, which may accommodate more and more complex data. In this exemplary embodiment, the first part of the data may be copied to the first memory using a memcpy processing method, and then the copying of the second part of the data may be continued by switching to the second memory. In this exemplary embodiment, the first memory may be a temporary storage area, and the second memory may be a non-temporary storage area. The priority of the first memory may be higher than that of the second memory.

[0050] It should be noted that the amount of data copied to the second memory is not necessarily greater than the amount of data copied to the first memory. For example, if the data to be processed is 30KB, the first memory is 25KB, and the second memory is 500M, the data to be processed can be divided into 25KB and 5KB, and 25KB of data can be copied to the first memory, and 5KB of data can be copied to the second memory, and so on.

[0051] In an exemplary embodiment, the above-mentioned determination of the data classification threshold may include:

[0052] Get the number of processes participating in the communication;

[0053] The data classification threshold is determined based on the size of the secondary cache temporary storage and the number of processes.

[0054] In this exemplary embodiment, the data classification threshold can be determined based on the number of processes communicating with parameters and the size of a secondary cache temporary memory, wherein the secondary cache temporary memory can be an L2 cache, which is a temporary memory located between a processor and a memory, with a smaller capacity than the memory but a faster exchange speed. This exemplary embodiment calculates the data classification threshold based on the size of the L2 cache and the number of processes, for example, by dividing the size of the L2 cache by the number of processes to determine the data classification threshold.

[0055] It should be noted that, in practical applications, the data classification threshold may be dynamically changed. For example, when the number of processes changes, the data classification threshold calculated based on the process will also change accordingly.

[0056] In an exemplary embodiment, the communication data processing method may further include:

[0057] During the communication process, monitor the data processing performance;

[0058] If the data processing performance does not meet the preset requirements, the data classification threshold is dynamically adjusted according to the preset rules.

[0059] In addition, this exemplary embodiment can also monitor data processing performance during the communication process, such as transmission rate, delay, bandwidth, etc., and further, the data classification threshold can be dynamically conditioned according to the monitored data processing performance. Specifically, it can be pre-configured to monitor whether the data processing performance meets the preset requirements. If the preset requirements are not met, the data classification threshold can be dynamically adjusted according to the preset rules. For example, when the transmission rate is less than the preset rate threshold, the data classification threshold can be increased. The preset rules can be pre-configured adjustment rules. For example, increasing the data classification threshold can be increased in an arithmetic or geometric manner, such as increasing the data classification threshold to twice the original value, etc.

[0060] It should be noted that different data processing performances may correspond to different preset requirements. For example, transmission rate and delay may correspond to different rate requirements and delay requirements respectively. In addition, failure of data processing performance to meet the preset requirements may mean that one data processing performance fails to meet the preset requirements, or multiple data processing performances fail to meet the preset requirements, or all data processing performances fail to meet the preset requirements, which can trigger dynamic adjustment of data classification thresholds according to preset rules, etc.

[0061] Below, the above content is explained with a specific embodiment. Assume that there is a system with an L2 cache size of 256KB and 16 processes participating in the Alltoall communication. The initial data classification threshold T will be set to 256KB / 16=16KB; in the data classification stage, when a target process needs to send a 20KB data to be processed, since the data to be processed is larger than the data classification threshold T, it is classified as the second type of data and enters a special processing flow; in the memory copy stage, the process can first use memcpy to copy the first 16KB of data to the target buffer, and then use the non-temporary storage strategy to copy the remaining 4KB of data, prompting that the memory area does not need to be cached, thereby reducing RFO requests and improving bandwidth utilization. In the dynamic adjustment stage, if it is monitored that the performance does not meet expectations when processing large messages, the data classification threshold T can be adjusted in real time, for example, increased to 32KB, to better adapt to the current load and hardware characteristics.

[0062] Based on the above approach, this exemplary embodiment can overcome the problem that the traditional Alltoall communication algorithm causes a large number of RFO requests due to frequent memory replication when processing large-scale messages, thereby consuming bandwidth and causing memory pipeline stagnation in a multi-core system. The hybrid memory replication scheme proposed in this exemplary embodiment can effectively reduce the number of memory transactions, and greatly improve the bandwidth and communication performance of data transmission through reasonable threshold setting and non-temporary storage strategy; it can maintain high efficiency in the transmission of messages of different scales, while reducing latency and resource waste, and achieve a significant improvement in global performance.

[0063] This exemplary embodiment can be applied to the Alltoall operation of large-scale parallel computing in a high-performance computing (HPC) environment, and is particularly suitable for various applications in data centers and supercomputers, such as climate simulation, fluid dynamics, molecular dynamics, etc. The network element devices mainly involved may include:

[0064] Compute node: A host used to process and store data, usually composed of a multi-core CPU and large-capacity memory;

[0065] Switch: responsible for data transmission between nodes, supporting low-latency and high-bandwidth communication;

[0066] Storage devices: used to store computational results and intermediate data, such as distributed storage systems or parallel file systems.

[0067] The communication connection relationship between network element devices may include:

[0068] Connection between nodes and switches: Each computing node is connected to a switch through a high-speed network interface card (NIC) to achieve fast data exchange;

[0069] Connections between switches: Multiple switches are interconnected via optical fiber or Ethernet to form a larger network topology to support complex communication patterns;

[0070] The function, form, and type of each device may include:

[0071] Compute nodes: can be standard rack-mounted servers with multi-core processors, each running MPI (Message Passing Interface) tasks;

[0072] Switch: Use high-performance switching chips to support different network protocols, such as InfiniBand, Ethernet, etc.

[0073] Storage device: usually NAS (network attached storage) integrated into the computing cluster, providing high-throughput read and write capabilities.

[0074] The exemplary embodiments of the present disclosure also provide a communication data processing device. Figure 3 The device 300 may include a data size determination module 310, which is used to determine the data size of the data to be processed included in the target process participating in the communication; a classification result determination module 320, which is used to determine the data classification result of the data to be processed according to the data size of the data to be processed; and a processing strategy determination module 330, which is used to determine the memory processing strategy of the data to be processed according to the data classification result, and adopt the memory processing strategy to process the data to be processed.

[0075] In an exemplary embodiment, the communication data processing device also includes: a threshold determination unit for determining a data classification threshold; a classification result determination module 320, including: a classification result determination unit for comparing the data size of the data to be processed with the data classification threshold, and determining the data classification result of the data to be processed based on the comparison result.

[0076] In an exemplary embodiment, the classification result determination unit includes: a first category data determination subunit, used to determine that the data to be processed is first category data if the data size of the data to be processed is less than the data classification threshold; a second category data determination subunit, used to determine that the data to be processed is second category data if the data size of the data to be processed is greater than or equal to the data classification threshold.

[0077] In an exemplary embodiment, if the data to be processed is the first type of data, the processing strategy determination module 330 includes: a first processing unit, which is used to copy the data to be processed to the first memory using a standard data processing method.

[0078] In an exemplary embodiment, if the data to be processed is the second category of data, the processing strategy determination module 330 includes: a data division unit, used to divide the data to be processed into a first part of data and a second part of data; a second processing unit, used to copy the first part of data to the first memory, and copy the second part of data to the second memory.

[0079] In an exemplary embodiment, the threshold determination unit includes: a process number acquisition subunit for acquiring the number of processes participating in the communication; a threshold determination subunit for determining the data classification threshold according to the size of the secondary cache temporary memory and the number of processes.

[0080] In an exemplary embodiment, the communication data processing device also includes: a performance monitoring unit for monitoring data processing performance during the communication process; a threshold adjustment unit for dynamically adjusting the data classification threshold according to preset rules if the data processing performance does not meet the preset requirements.

[0081] The specific details of each module / unit in the above-mentioned device have been described in detail in the embodiments of the method part. The undisclosed details can be found in the embodiments of the method part, so they will not be repeated here.

[0082] Exemplary embodiments of the present disclosure also provide an electronic device capable of implementing the above method.

[0083] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0084] Refer to the following Figure 4 4 to describe the electronic device 400 according to such an exemplary embodiment of the present disclosure. Figure 4 The electronic device 400 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0085] like Figure 4 As shown, the electronic device 400 is in the form of a general computing device. The components of the electronic device 400 may include but are not limited to: the at least one processing unit 410, the at least one storage unit 420, a bus 430 connecting different system components (including the storage unit 420 and the processing unit 410), and a display unit 440.

[0086] The storage unit stores program codes, which can be executed by the processing unit 410, so that the processing unit 410 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 410 can execute Figure 1 or Figure 2 Steps shown etc.

[0087] The storage unit 420 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 421 and / or a cache memory unit 422 , and may further include a read-only memory unit (ROM) 423 .

[0088] The storage unit 420 may also include a program / utility 424 having a set (at least one) of program modules 425, such program modules 425 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0089] Bus 430 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0090] The electronic device 400 may also communicate with one or more external devices 500 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 400, and / or communicate with any device that enables the electronic device 400 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 450. Furthermore, the electronic device 400 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 460. As shown, the network adapter 460 communicates with other modules of the electronic device 400 via a bus 430. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0091] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary embodiment of the present disclosure.

[0092] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.

[0093] The exemplary embodiments of the present disclosure also provide a program product for implementing the above method, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0094] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0095] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0096] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0097] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0098] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0099] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0100] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0101] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A communication data processing method, characterized in that: include: Determining the data size of the to-be-processed data included in the target process participating in the communication; Determining a data classification result of the data to be processed according to the data size of the data to be processed; A memory processing strategy for the data to be processed is determined according to the data classification result, and the data to be processed is processed using the memory processing strategy.

2. The method according to claim 1, characterized in that The method further comprises: Determine data classification thresholds; Determining the data classification result of the data to be processed according to the data size of the data to be processed includes: The data size of the data to be processed is compared with the data classification threshold, and the data classification result of the data to be processed is determined according to the comparison result.

3. The method according to claim 2, characterized in that The comparing the data size of the data to be processed with the data classification threshold, and determining the data classification result of the data to be processed according to the comparison result, includes: If the data size of the data to be processed is smaller than the data classification threshold, determining that the data to be processed is first-category data; If the data size of the data to be processed is greater than or equal to the data classification threshold, it is determined that the data to be processed is second-category data.

4. The method according to claim 3, characterized in that If the data to be processed is the first type of data, determining the memory processing strategy of the data to be processed according to the data classification result, and using the memory processing strategy to process the data to be processed, including: The data to be processed is copied to the first memory using a standard data processing method.

5. The method according to claim 3, characterized in that: If the data to be processed is the second type of data, determining the memory processing strategy of the data to be processed according to the data classification result, and using the memory processing strategy to process the data to be processed, including: Dividing the data to be processed into a first part of data and a second part of data; The first portion of data is copied to a first memory, and the second portion of data is copied to a second memory.

6. The method according to claim 2, characterized in that The step of determining the data classification threshold comprises: Get the number of processes participating in the communication; The data classification threshold is determined according to the size of the secondary cache temporary memory and the number of the processes.

7. The method according to claim 2, characterized in that The method further comprises: During the communication process, monitor the data processing performance; If the data processing performance does not meet the preset requirements, the data classification threshold is dynamically adjusted according to preset rules.

8. A communication data processing device, characterized in that: include: A data size determination module, used to determine the data size of the to-be-processed data included in the target process participating in the communication; A classification result determination module, used to determine a data classification result of the data to be processed according to the data size of the data to be processed; The processing strategy determination module is used to determine the memory processing strategy of the data to be processed according to the data classification result, and adopt the memory processing strategy to process the data to be processed.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images