File reading and writing method and device, equipment, readable storage medium and program product

Abstract

The invention relates to a file reading and writing method and device, equipment, a readable storage medium and a program product. The method is applied to the electronic equipment, a processor of the electronic equipment comprises a plurality of CCXs, and the method comprises the following steps: firstly, receiving a target file read-write request, then, creating a parallel thread group for processing the target file read-write request in parallel for the target file read-write request, and finally, running the parallel thread group and the target file read-write request based on the same CCX in the plurality of CCXs. By the adoption of the method, the parallel thread group and the target file read-write request can run in the same CCX, data exchange among caches of multiple CCXs is avoided, and the single-stream file read-write performance is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a file reading and writing method and device, equipment, readable storage medium and program product. BACKGROUND

[0002] At present, in the file reading and writing process, a single thread is usually used for serial file reading and writing, but when the content of the file to be read and written is relatively large, the single thread serial operation becomes the bottleneck of the file reading and writing performance. In view of this problem, the common method is to split the file reading and writing request of the application program into multiple small requests, and deliver the split requests to the multiple background threads created in advance, and the background threads process the split file reading and writing requests in parallel. After all the split file reading and writing requests are completed, the multiple split requests are restored into a single request, and the request result is returned to the application program.

[0003] However, when multiple background threads run on multiple CPU cores, for the CCX (Core Complex, core unit) of the Haiguang platform, different CCXs do not share the cache, so data exchange will occur between two CCXs, which becomes the performance bottleneck of single stream file reading and writing. SUMMARY

[0004] Therefore, it is necessary to provide a file reading and writing method, device, equipment, readable storage medium and program product capable of improving the performance of single stream file reading and writing.

[0005] In a first aspect, the present application provides a file reading and writing method applied to an electronic device, wherein the processor of the electronic device includes multiple CCXs, and the method includes:

[0006] receiving a target file reading and writing request;

[0007] creating a parallel thread group for the target file reading and writing request; the parallel thread group is used for parallel processing of the target file reading and writing request;

[0008] running the parallel thread group and the target file reading and writing request based on the same CCX in the multiple CCXs.

[0009] In the above embodiment, first, the target file reading and writing request is received, then a parallel thread group for parallel processing of the target file reading and writing request is created, and finally the parallel thread group and the target file reading and writing request are run based on the same CCX in the multiple CCXs. In this way, by running the parallel thread group and the target file reading and writing request in the same CCX, data exchange between the caches of multiple CCXs is avoided, and the performance of single stream file reading and writing is improved.

[0010] In one of the embodiments, before creating the parallel thread group for the target file read-write request, the method further comprises:

[0011] obtaining a resource state of the processor;

[0012] creating the parallel thread group for the target file read-write request according to the resource state.

[0013] In the above embodiments, by obtaining the resource state of the processor, the creation mode of the parallel thread group is determined according to the resource state, and the creation mode is more flexible.

[0014] In one of the embodiments, obtaining the resource state of the processor comprises:

[0015] obtaining a CPU resource utilization rate of each CCX;

[0016] if the CPU resource utilization rate of each CCX is lower than a preset threshold, determining that the resource state is a resource sufficient state;

[0017] otherwise, determining that the resource state is a resource limited state.

[0018] In the above embodiments, the resource state is determined according to the CPU resource utilization rate, which ensures that the CCX resources are in a balanced utilization state and improves the stability of the CCX.

[0019] In one of the embodiments, creating the parallel thread group for the target file read-write request according to the resource state comprises:

[0020] if the resource state is the resource sufficient state, creating one parallel thread group for the target file read-write request in each CCX.

[0021] In the above embodiments, by obtaining the resource state of the processor, the parallel thread group is created for the target file read-write request in each CCX when the CPU resource is sufficient, and efficient parallel processing of the target file read-write request is realized.

[0022] In one of the embodiments, running the parallel thread group and the target file read-write request based on the same CCX in the plurality of CCXs comprises:

[0023] scheduling the target file read-write request to a first CCX in the plurality of CCXs to run;

[0024] controlling the parallel thread group created in the first CCX to run, and prohibiting the parallel thread group in other CCXs except the first CCX to run.

[0025] In the above embodiment, by controlling the running of the parallel thread group in the same CCX as the target file read-write request, and not running the parallel thread group in other CCXs, it is ensured that the target file read-write request and the parallel thread group run in the same CCX, and unnecessary waste of computing resources is prevented.

[0026] In one of the embodiments, the parallel thread group is created for the target file read-write request according to the resource state, including:

[0027] If the resource state is the resource limited state, the parallel thread group is created on the second CCX, and the second CCX is the CCX currently running the target file read-write request.

[0028] The created parallel thread group is thread group bound with the target file read-write request.

[0029] In the above embodiment, in the case of limited resources, a parallel thread group is created at the initiation of the target file read-write request and is thread group bound with the target file read-write request, so that the parallel thread group does not need to be created on each CCX, and resource overhead on other CCXs is saved.

[0030] In one of the embodiments, the parallel thread group and the target file read-write request are run in the same CCX of the multiple CCXs, including:

[0031] Based on the thread group binding relationship, the target file read-write request and the created parallel thread group are scheduled to run in the same CCX.

[0032] In the above embodiment, the bound parallel thread group is scheduled at the same time as the scheduling of the target file read-write request, so that the target file read-write request and the parallel thread group run in the same CCX.

[0033] In one of the embodiments, the parallel thread group and the target file read-write request are run in the same CCX of the multiple CCXs, including:

[0034] The scheduling of the target file read-write request is limited to the second CCX, so as to run the parallel thread group and the target file read-write request based on the second CCX.

[0035] The second CCX is the CCX currently running the target file read-write request.

[0036] In the above embodiment, the scheduling limitation of the target file read-write request is realized by the target control group, so as to reduce data migration across CCXs and reduce performance overhead of file read-write.

[0037] In one of the embodiments, the scheduling of the target file read-write request is limited to the second CCX, including:

[0038] A target control group is created, and the target file read-write request is added to the target control group; the target control group is used to manage the target file read-write request;

[0039] The target file read-write request is scheduled to the second CCX based on the target control group.

[0040] In the above embodiment, the scheduling of the target file read-write request is limited by the target control group, thereby reducing data migration across CCXs and reducing the performance overhead of file read-write.

[0041] In one of the embodiments, the scheduling of the target file read-write request is limited to the second CCX, including:

[0042] The scheduling of the target file read-write request is limited to the second CCX by the kernel scheduler.

[0043] In the above embodiment, the scheduling of the target file read-write request is limited to the second CCX by the kernel scheduler, thereby reducing data migration across CCXs and reducing the performance overhead of file read-write.

[0044] In one of the embodiments, the method further includes:

[0045] A dedicated L3 cache is allocated for the target file read-write request on the second CCX;

[0046] The size of the dedicated L3 cache is dynamically adjusted according to the data size of the target file read-write request.

[0047] In the above embodiment, the size of the dedicated L3 cache can be dynamically adjusted according to the data size of the target file read-write request, ensuring that the target file read-write request can always obtain sufficient cache resource support, further enhancing the performance and stability of request processing.

[0048] In a second aspect, the application also provides a file read-write device applied in an electronic device, wherein the processor of the electronic device includes a plurality of CCXs, and the device includes:

[0049] The receiving module is configured to receive a target file read-write request;

[0050] The creating module is configured to create a parallel thread group for the target file read-write request; the parallel thread group is used to process the target file read-write request in parallel;

[0051] The running module is configured to run the parallel thread group and the target file read-write request based on the same CCX in the plurality of CCXs.

[0052] In a third aspect, the present application also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method according to any one of the first aspect.

[0053] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method according to any one of the first aspect.

[0054] In a fifth aspect, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the method according to any one of the first aspect.

[0055] The above file reading and writing method, device, equipment, readable storage medium and program product are applied to an electronic device, and a processor of the electronic device comprises a plurality of CCXs. First, a target file reading and writing request is received. Then, a parallel thread group for processing the target file reading and writing request in parallel is created for the target file reading and writing request. Finally, the parallel thread group and the target file reading and writing request are run based on a same CCX in the plurality of CCXs. In this way, by running the parallel thread group and the target file reading and writing request in the same CCX, data exchange between caches of the plurality of CCXs is avoided, and the performance of single-stream file reading and writing is improved. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0057] Figure 1 It is a schematic diagram of the single-stream file reading and writing process in the related art;

[0058] Figure 2 It is a schematic diagram of the single-stream file reading and writing process in the related art;

[0059] Figure 3 It is a schematic diagram of the single-stream file reading and writing process in the related art;

[0060] Figure 4 It is a schematic diagram of the single-stream file reading and writing process in the related art;

[0061] Figure 5 It is a schematic diagram of the single-stream file reading and writing process in the related art;

[0062] Figure 6Flowchart of a single-stream file read-write method in another embodiment;

[0063] Figure 7 Flowchart of a single-stream file read-write method in another embodiment;

[0064] Figure 8 Flowchart of a single-stream file read-write method in another embodiment;

[0065] Figure 9 Flowchart of a single-stream file read-write method in another embodiment;

[0066] Figure 10 Flowchart of a single-stream file read-write method in another embodiment;

[0067] Figure 11 Structure block diagram of a single-stream file read-write device in an embodiment;

[0068] Figure 12 Internal structure diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not intended to limit the present application.

[0070] It should be noted that the terms "first", "second", etc. used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "multiple" used in the present application refers to two and more than two. The term "and / or" used in the present application refers to one of the options, or any combination of multiple options.

[0071] When processing a single-stream file read-write request of an application program using a multi-threading scheme, a common practice is to split the single-stream file read-write request into multiple small requests, and deliver the split requests to a plurality of background threads created in advance, and the background threads process the split file read-write requests in parallel. After all the split file read-write requests are completed, the multiple split requests are restored to a single request, and the request result is returned to the application program.

[0072] CCX is the basic module of the processor, and AMD (Advanced Micro Devices, Advanced Micro Devices) and Haiguang platform CPU are taken as examples. Different processor architectures include different numbers of CCXs, each CCX includes multiple CPU cores, and multiple CPU cores share a three-level cache, and different CCXs do not share the cache. As shown in Figure 1 , a single-stream file read-write request initiated by a single-stream application runs on a certain CPU core of CCX 0, and memory A is allocated as a buffer for the single-stream file read-write request to write a file. The parallel copy thread is used to split the single-stream file read-write request into multiple small requests. When the parallel copy thread and the single-stream file read-write request run in different CCXs, as shown in Figure 1 , the single-stream file read-write request runs in CCX 0, and the parallel copy thread runs in CCX 1. Different CCXs do not share the cache. When the single-stream file read-write request running in CCX 0 needs to access memory A, it needs to be loaded into the cache of CCX 0. When the parallel copy thread running in CCX 1 needs to access memory A, it needs to be loaded into the cache of CCX 1. In this way, the cache between the two CCXs will exchange data due to data consistency and other hardware mechanisms. This data exchange becomes a bottleneck that limits the single-stream file read-write.

[0073] Therefore, the present application provides a file read-write method. The processor of the electronic device includes multiple CCXs. First, a target file read-write request is received. Then, a parallel thread group for parallel processing of the target file read-write request is created. Finally, the parallel thread group and the target file read-write request are run based on the same CCX in the multiple CCXs. In this way, by running the parallel thread group and the target file read-write request in the same CCX, data exchange between the caches of the multiple CCXs is avoided, and the performance of the single-stream file read-write is improved.

[0074] In an exemplary embodiment, as shown in Figure 2 , a file read-write method is provided. The method is applied to an electronic device, and the processor of the electronic device includes multiple CCXs. The method includes the following steps 201 to 203. Wherein:

[0075] Step 201, receiving a target file read-write request.

[0076] The target file read-write request can be a single-stream file read-write request initiated by a single-stream application, that is, a file read-write is initiated through a single thread or a single process, and only one file read-write request is initiated each time. The next request is initiated after the request processing is completed.

[0077] Optionally, after receiving the target file read-write request, the target file read-write request can be checked to confirm the legitimacy and authority effectiveness of the source of the target file read-write request, to ensure that the operation conforms to the preset security policy. Subsequently, according to the target file path specified in the target file read-write request, file existence and accessibility are checked, and if the file does not exist, a corresponding error prompt is returned, and if the file exists, the read-write authority of the user to the file is further verified, and after the authority verification is passed, the target file read-write request can be processed accordingly.

[0078] Step 202, creating a parallel thread group for the target file read-write request.

[0079] The parallel thread group is used to process the target file read-write request in parallel. Since the target file read-write request is a single-stream file read-write request, when the content to be read and written is relatively large, in order to improve the efficiency of file read-write, a parallel thread group is created to process the target file read-write request in parallel.

[0080] Optionally, the number of parallel threads in the parallel thread group can be dynamically adjusted according to the system resource situation, to avoid resource competition and blocking phenomenon between different threads. For example, when the data volume to be processed by the target file read-write request is large, the number of parallel threads can be increased, and the parallel thread group can split the large file into multiple data blocks, and each thread processes a data block, thereby reducing the overall operation time.

[0081] Step 203, running the parallel thread group and the target file read-write request based on the same CCX in the plurality of CCXs.

[0082] As known from the above, different CCXs do not share the cache. When the parallel thread group and the target file read-write request run in different CCXs, both CCXs will create a corresponding cache for the same buffer in the memory, and the caches between the two CCXs will exchange data due to data consistency and other hardware mechanisms, affecting the performance of the target file read-write request. Therefore, by limiting the parallel thread group and the target file read-write request to run in one CCX, data exchange between the caches of multiple CCXs can be avoided.

[0083] Optionally, when the system is scheduled, the parallel thread group and the target file read-write request can be scheduled at the same time, so as to realize scheduling of the parallel thread group and the target file read-write request to run in the same CCX. As shown in FIG. 8, at this time, the parallel copy thread can use the data loaded into the cache by the single-stream application program using the buffer area A, avoiding data exchange between the caches, thereby achieving the purpose of improving performance. Figure 3

[0084] ​Optionally, a parallel thread group can be created in each CCX, and according to the scheduling of the target file read-write request, a parallel thread group in the same CCX as the target file read-write request is selected to process the target file read-write request, so that the parallel thread group and the target file read-write request run in the same CCX.

[0085] In the above embodiment, first, the target file read-write request is received, then a parallel thread group for processing the target file read-write request is created for the target file read-write request, and finally the parallel thread group and the target file read-write request are run based on the same CCX in the plurality of CCXs. In this way, by running the parallel thread group and the target file read-write request in the same CCX, data exchange between the caches of the plurality of CCXs is avoided, and the performance of single-stream file read-write is improved.

[0086] In the embodiments of the present application, in order not to affect the normal operation of the processor, before the parallel thread group is created for the target file read-write request, the method further includes: Figure 4 as shown, the method further includes:

[0087] Step 401, obtaining the resource state of the processor.

[0088] In the embodiment, the resource state of the processor can be obtained in real time through a system tool or a programming interface. The resource state can include CPU resource utilization, core temperature, running frequency, cache hit rate, instruction execution efficiency, and load distribution of each core, and other key data. For example, in a multi-task processing scenario, when multiple application programs run simultaneously, the CPU resource utilization may increase significantly, and the system response speed may slow down. Optionally, the resource state of the processor can also include memory access delay or thread scheduling efficiency and other indicators. By obtaining the resource state of the processor, it can be determined whether the processor has sufficient resources to create a parallel thread group.

[0089] Optionally, the step of obtaining the resource state of the processor can include: Figure 5 as shown, the step of obtaining the resource state of the processor can include:

[0090] Step 501, obtaining the CPU resource utilization of each CCX.

[0091] The CPU resource utilization of each CCX is obtained in real time through a system tool, so as to determine the load of each CCX.

[0092] Step 502, if the CPU resource utilization of each CCX is lower than a preset threshold, it is determined that the resource state is a resource sufficient state.

[0093] The preset threshold can be set according to actual conditions of the processor and the number of parallel thread groups, and if the CPU resource utilization of each CCX is lower than the preset threshold, the CPU resource state of each CCX is determined as a resource sufficient state, that is, there is enough CPU resource overhead that can be used to create a parallel thread group for multi-thread task running.

[0094] In step 503, otherwise, the resource state is determined as a resource limited state.

[0095] If the CPU resource utilization of a CCX is lower than the preset threshold, the CPU resource state of the CCX is determined as a resource limited state, that is, the CPU resource overhead cannot meet the creation of a parallel thread group, and then a parallel thread group cannot be created on each CCX.

[0096] In step 402, according to the resource state, a parallel thread group is created for the target file read-write request.

[0097] Optionally, if the resource state is a resource sufficient state, a parallel thread group is created in each CCX for the target file read-write request.

[0098] According to the above, the system load of the current CPU is determined by the preset threshold. When it is detected that the CPU of each CCX is in a resource sufficient state, the creation mechanism of a parallel thread group is automatically triggered in each CCX, and the parallel thread group is affinity bound with the CCX. Optionally, according to the size, read-write frequency and priority of the target file to be operated by the target file read-write request, the optimal number of parallel threads is dynamically calculated. For example, for a batch write request of a large-capacity log file, a CCX can create a parallel thread group containing 8-16 threads, and each parallel thread is responsible for processing different data blocks of the log file, and the read-write efficiency of the target file is significantly improved through multi-thread concurrent operation. When it is detected that the CPU of each CCX is in a resource limited state, a parallel thread group is not created for each CCX.

[0099] In the above embodiment, by obtaining the resource state of the processor, a parallel thread group is created in each CCX for the target file read-write request when the CPU resource is sufficient, and efficient parallel processing of the target file read-write request is realized.

[0100] In the embodiment of the application, running a parallel thread group and a target file read-write request based on the same CCX in a plurality of CCXs can include three ways, the first way as shown in Figure 6 may include:

[0101] In step 601, the target file read-write request is scheduled to a first CCX in a plurality of CCXs for running.

[0102] The first CCX can be any CCX in the processor. During program running, the operating system dynamically schedules program threads among different CPU cores according to the load of hardware resources and other system information. In the single-stream read-write scenario, the single-stream application and the parallel replication threads are randomly scheduled by the operating system among the CPU cores of the multiple CCXs, and the scheduling of the single-stream application and the parallel replication threads is completed independently.

[0103] Optionally, the operating system can monitor and evaluate the current load state, network delay and resource utilization of each CCX in real time through a load balancing algorithm, and under the premise of ensuring data consistency and operation efficiency, preferentially schedule the read-write request of the target file to the first CCX with lower load and faster response speed.

[0104] Step 602, control the parallel thread group created in the first CCX to run, and prohibit the parallel thread group in other CCXs except the first CCX to run.

[0105] In the case that the resource state of each CCX is resource sufficient, a parallel thread group is created in each CCX, when the read-write request of the target file is scheduled to run in the first CCX, the operating system controls the parallel thread group created in the first CCX to run, and processes the read-write request of the target file through the parallel thread group, so that the read-write request of the target file and the parallel thread group run in the same CCX. At the same time, in order to reduce system overhead, the parallel thread group in other CCXs except the first CCX can be prohibited to run. It can be understood that when the read-write request of the target file is scheduled to run in the third CCX, the parallel thread group in the third CCX is controlled to run, and the parallel thread group in other CCXs except the third CCX is prohibited to run.

[0106] Optionally, the running and disabling of the parallel thread group in different CCXs can be controlled through conditional judgment at the code level or switch setting of the configuration file.

[0107] In the above embodiment, by controlling the parallel thread group in the same CCX as the read-write request of the target file to run, and the parallel thread group in other CCXs not to run, the read-write request of the target file and the parallel thread group run in the same CCX are ensured, and unnecessary waste of computing resources is also prevented.

[0108] In the embodiments of the present application, according to the resource state, the parallel thread group is created for the read-write request of the target file, as shown in Figure 7 , further comprising:

[0109] Step 701, if the resource state is a resource limited state, creating a parallel thread group on the second CCX.

[0110] The second CCX is a CCX currently running the target file read-write request. If the resource state of the CCX is a resource-limited state, in order to avoid the load of the CCX being too high and the response delay being increased, a parallel thread group is not created on each CCX, but is only created on the second CCX. It can be understood that when the target file read-write request is initiated, the operating system schedules and runs the current target file read-write request on the second CCX, that is, the second CCX is a CCX with a low current resource usage rate, and therefore, the second CCX has sufficient resources to create a parallel thread group.

[0111] In step 702, the created parallel thread group is thread group bound with the target file read-write request.

[0112] The parallel thread group created on the second CCX is thread group bound with the target file read-write request. At this time, the second way of running the parallel thread group and the target file read-write request based on the same CCX in the plurality of CCXs includes: based on the thread group binding relationship, scheduling the target file read-write request and the created parallel thread group to run in the same CCX.

[0113] Based on the binding relationship between the parallel thread group and the target file read-write request, when the operating system schedules the target file read-write request across the CCX, the parallel thread group in the bound thread group is also scheduled. The CPU resources available to the parallel thread group are all the cores in the CCX to which the target file read-write request is scheduled.

[0114] In the above embodiment, in the case of limited resources, a parallel thread group is created and thread group bound with the target file read-write request when the target file read-write request is initiated, and a parallel thread group does not need to be created on each CCX, thereby saving resource overhead on other CCXs.

[0115] Further, when the target file read-write request is scheduled, the bound parallel thread group is also scheduled, so that the target file read-write request and the parallel thread group run in the same CCX.

[0116] In the embodiment of the present application, the third way of running the parallel thread group and the target file read-write request based on the same CCX in the plurality of CCXs includes: limiting the scheduling of the target file read-write request to the second CCX, so as to run the parallel thread group and the target file read-write request based on the second CCX; wherein the second CCX is a CCX currently running the target file read-write request.

[0117] The first mode and the second mode in the above embodiments solve the problem of how to ensure that the parallel thread group and the target file read-write request run in the same CCX when the parallel thread group and the target file read-write request are scheduled across CCXs. However, when the scheduling is across CCXs, data migration between the three-level caches of different CCXs will still occur, which will also cause certain performance overhead. Therefore, in order to further reduce the performance overhead, on the basis of the first mode and the second mode, a thread scheduling restriction strategy is added, that is, after the parallel thread group is created and allocated for the target file read-write request, the scheduling of the target file read-write request is limited to the second CCX in which the target file read-write request is currently running. In this way, after the target file read-write request and the parallel thread group are scheduled across CCXs, data will not be migrated across CCXs, thereby reducing the performance overhead.

[0118] Optionally, the way of limiting the scheduling of the target file read-write request to the second CCX can be as shown in Figure 8 , which includes:

[0119] Step 801, a target control group is created, and the target file read-write request is added to the target control group.

[0120] The target control group is used to manage the target file read-write request. In the operating system, the control group is usually used to centrally manage and monitor specific file access behaviors to ensure the compliance and security of data operations. Optionally, the way of adding the target file read-write request to the target control group can include creating the target control group and setting the corresponding permission policy. After the control group is created, the target file read-write request is associated with the created target control group through the function of adding to the control group.

[0121] Step 802, the scheduling of the target file read-write request is limited to the second CCX based on the target control group.

[0122] By setting the permissions of the target control group, for example, setting the target control group to prohibit scheduling across CCXs, the target file read-write request is limited to running in the second CCX, and the scheduling of the target file read-write request is avoided.

[0123] In the above embodiments, the scheduling of the target file read-write request is limited through the target control group, thereby reducing data migration across CCXs and reducing the performance overhead of file read-write.

[0124] In another embodiment, the way of limiting the scheduling of the target file read-write request to the second CCX can also be to limit the scheduling of the target file read-write request to the second CCX through the kernel scheduler.

[0125] The kernel scheduler is an important component in the operating system kernel, responsible for managing the execution order and time allocation of processes or threads. By allocating CPU to processes or threads according to the preset scheduling algorithm, multi-task processing is achieved. Since the scheduling of target file read-write requests needs to be limited in the second CCX, the scheduling strategy and rules of the kernel scheduler are modified, such as specifying the scheduling mode of target file read-write requests as being able to be scheduled between cores and prohibiting cross-CCX scheduling, thereby avoiding cross-CCX scheduling of target file read-write requests.

[0126] Exemplarily, target file read-write requests are allocated to specific cores of the second CCX for execution, and these cores work cooperatively to accelerate data access and transmission using local caches, reducing data migration overhead across CCXs. At the same time, the kernel scheduler can also monitor the load situation of the second CCX in real time and dynamically adjust the allocation strategy of other requests to ensure the efficiency and stability of target file read-write requests, thereby improving the I / O throughput and response speed of the overall system.

[0127] In the above embodiment, the kernel scheduler is used to limit the scheduling of target file read-write requests to the second CCX, thereby reducing data migration across CCXs and reducing the performance overhead of file read-write.

[0128] Within a CCX, in addition to running target file read-write requests, requests of other application programs are also run in a single CCX, and the request processing processes of other application programs all occupy CPU cores and level 3 cache resources. When the requests of other application programs and the target file read-write requests compete for level 3 cache resources, the target file read-write requests may be affected to some extent, including a decrease in the number of available level 3 caches or repeated flushing and loading of data in the level 3 cache, thereby affecting the performance of the target file read-write requests. Therefore, in order to avoid the influence of other application programs on the target file read-write requests, the method of the present embodiment, as shown in Figure 9 , further comprises:

[0129] Step 901: allocating a dedicated level 3 cache for the target file read-write requests on the second CCX.

[0130] Optionally, the resources of a CCX can be isolated for the target file read-write request business, including CPU core resources and corresponding cache resources. A dedicated level 3 cache can also be allocated for the target file read-write requests on the second CCX, and the second CCX is the currently running CCX of the target file read-write requests, realizing fine-grained cache resource management, which ensures the performance of the target file read-write requests while avoiding resource waste.

[0131] Step 902: dynamically adjusting the size of the dedicated level 3 cache according to the data size of the target file read-write requests.

[0132] For example, when receiving a target file read-write request, the second CCX identifies the file identifier of the target file read-write request, and obtains the size of the file to be read and written. According to a preset cache allocation strategy, a separate cache area with high priority is allocated for the target file read-write request in the three-level cache architecture managed by the second CCX, so that the read operation can be completed directly from the cache, avoiding frequent access to the underlying storage device. For write requests, the dedicated three-level cache can temporarily store data in the cache, and then write the data to the persistent storage in batches at an appropriate time, effectively reducing the impact of write operations on system performance. At the same time, according to the data size of the target file read-write request, the size of the dedicated three-level cache can be dynamically adjusted to ensure that the target file read-write request can always obtain sufficient cache resource support, further enhancing the performance and stability of request processing.

[0133] In the embodiments of the present application, as shown in Figure 10 A file read-write method is provided, comprising:

[0134] Step 1001, receiving a target file read-write request.

[0135] Step 1002, obtaining the resource state of the processor.

[0136] Step 1003, if the resource state is a resource sufficient state, creating a parallel thread group in each CCX for the target file read-write request.

[0137] Step 1004, scheduling the target file read-write request to a first CCX in the plurality of CCXs for running.

[0138] Step 1005, controlling the parallel thread group created in the first CCX to run, and prohibiting the parallel thread groups in the CCXs other than the first CCX from running.

[0139] Step 1006, if the resource state is a resource limited state, creating a parallel thread group in a second CCX.

[0140] Step 1007, thread group binding the created parallel thread group and the target file read-write request.

[0141] Step 1008, based on the thread group binding relationship, scheduling the target file read-write request and the created parallel thread group to run in the same CCX.

[0142] Step 1009, limiting the scheduling of the target file read-write request to the second CCX, so as to run the parallel thread group and the target file read-write request based on the second CCX.

[0143] In the above embodiment, the method can be applied to single-stream file read-write performance of AMD and Haiguang platform CPUs. Through single-stream file write performance verification using the Linux DD command, the performance on the AMD platform after using the method is twice that before using the method.

[0144] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0145] Based on the same inventive concept, the embodiments of the present application also provide a file reading and writing device for implementing the above-mentioned file reading and writing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more file reading and writing device embodiments provided below can refer to the limitations of the file reading and writing method in the above text, which will not be repeated here.

[0146] In one exemplary embodiment, as shown in Figure 11 A file reading and writing device 1100 is provided, which is applied in an electronic device, and a processor of the electronic device includes a plurality of CCXs, comprising: a receiving module 1101, a creating module 1102, and a running module 1103, wherein:

[0147] The receiving module 1101 is configured to receive a target file reading and writing request.

[0148] The creating module 1102 is configured to create a parallel thread group for the target file reading and writing request; and the parallel thread group is configured to process the target file reading and writing request in parallel.

[0149] The running module 1103 is configured to run the parallel thread group and the target file reading and writing request based on a same CCX in the plurality of CCXs.

[0150] In one embodiment, the device further comprises an obtaining module configured to obtain a resource state of the processor; and the parallel thread group for the target file reading and writing request is created according to the resource state.

[0151] In one of the embodiments, the obtaining module is specifically configured to obtain the CPU resource utilization of each CCX; if the CPU resource utilization of each CCX is lower than a preset threshold, it is determined that the resource state is a resource sufficient state; otherwise, it is determined that the resource state is a resource limited state.

[0152] In one of the embodiments, the obtaining module is specifically configured to, if the resource state is the resource sufficient state, create a parallel thread group in each CCX for the target file read-write request.

[0153] In one of the embodiments, the running module 1103 is specifically configured to schedule the target file read-write request to a first CCX in the plurality of CCXs for running; control the parallel thread group created in the first CCX to run, and prohibit the parallel thread groups in other CCXs except the first CCX to run.

[0154] In one of the embodiments, the obtaining module is further configured to, if the resource state is the resource limited state, create a parallel thread group on a second CCX, the second CCX being a currently running CCX of the target file read-write request; and bind the created parallel thread group with the target file read-write request in a thread group.

[0155] In one of the embodiments, the running module 1103 is further configured to, based on the thread group binding relationship, schedule the target file read-write request and the created parallel thread group to the same CCX for running.

[0156] In one of the embodiments, the running module 1103 is specifically configured to limit the scheduling of the target file read-write request to the second CCX, so as to run the parallel thread group and the target file read-write request based on the second CCX; wherein the second CCX is a currently running CCX of the target file read-write request.

[0157] In one of the embodiments, the running module 1103 is specifically configured to create a target control group, and add the target file read-write request into the target control group; the target control group is used for managing the target file read-write request; and the scheduling of the target file read-write request is limited to the second CCX based on the target control group.

[0158] In one of the embodiments, the running module 1103 is further configured to limit the scheduling of the target file read-write request to the second CCX through a kernel scheduler.

[0159] In one of the embodiments, the apparatus further includes an allocation module configured to allocate a dedicated L3 cache for the target file read-write request on the second CCX; and dynamically adjust the size of the dedicated L3 cache according to the data size of the target file read-write request.

[0160] The modules in the file reading and writing device can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in the electronic device in hardware form, or stored in a memory in the electronic device in software form, so as to be called and executed by the processor to perform operations corresponding to the modules.

[0161] In an example embodiment, an electronic device, which can be a terminal, can have an internal structure as shown in Figure 12 The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the electronic device is configured to exchange information between the processor and external devices. The communication interface of the electronic device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, near field communication (NFC), or other technologies. The computer program is executed by the processor to implement a file reading and writing method. The display unit of the electronic device is configured to form a visually visible screen, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0162] Those skilled in the art can understand that Figure 12 The structure shown in the above figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0163] In an example embodiment, an electronic device is provided, which includes a memory and a processor. The memory stores a computer program. The processor executes the computer program to implement the following steps: receiving a target file reading and writing request; creating a parallel thread group for the target file reading and writing request; the parallel thread group is configured to process the target file reading and writing request in parallel; and running the parallel thread group and the target file reading and writing request based on a same CCX in a plurality of CCXs.

[0164] In one embodiment, the processor, when executing the computer program, further implements the following steps: obtaining a resource state of the processor; and creating a parallel thread group for the target file read-write request according to the resource state.

[0165] In one embodiment, the processor, when executing the computer program, further implements the following steps: obtaining a CPU resource utilization of each CCX; and determining that the resource state is a resource sufficient state if the CPU resource utilization of each CCX is lower than a preset threshold; otherwise, determining that the resource state is a resource limited state.

[0166] In one embodiment, the processor, when executing the computer program, further implements the following steps: creating one parallel thread group in each CCX for the target file read-write request if the resource state is the resource sufficient state.

[0167] In one embodiment, the processor, when executing the computer program, further implements the following steps: scheduling the target file read-write request to a first CCX in the plurality of CCXs to run; and controlling the parallel thread group created in the first CCX to run and prohibiting the parallel thread groups in other CCXs except the first CCX to run.

[0168] In one embodiment, the processor, when executing the computer program, further implements the following steps: creating a parallel thread group on a second CCX if the resource state is the resource limited state, the second CCX being a currently running CCX for the target file read-write request; and thread group binding the created parallel thread group with the target file read-write request.

[0169] In one embodiment, the processor, when executing the computer program, further implements the following steps: scheduling the target file read-write request and the created parallel thread group to the same CCX to run based on the thread group binding relationship.

[0170] In one embodiment, the processor, when executing the computer program, further implements the following steps: limiting the scheduling of the target file read-write request to the second CCX to run the parallel thread group and the target file read-write request based on the second CCX; and the second CCX being a currently running CCX for the target file read-write request.

[0171] In one embodiment, the processor, when executing the computer program, further implements the following steps: creating a target control group and adding the target file read-write request into the target control group; the target control group being used for managing the target file read-write request; and limiting the scheduling of the target file read-write request to the second CCX based on the target control group.

[0172] In one embodiment, the processor, when executing the computer program, further implements the following steps: limiting the scheduling of the target file read-write request to the second CCX by a kernel scheduler.

[0173] In one embodiment, the processor further implements the following steps when executing the computer program: allocating a dedicated level 3 cache for the target file read-write request on the second CCX; and dynamically adjusting the size of the dedicated level 3 cache according to the data size of the target file read-write request.

[0174] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the method in any one of the above method embodiments.

[0175] In one embodiment, a computer program product is provided, and the computer program product includes a computer program. The computer program is executed by a processor to implement the steps of the method in any one of the above method embodiments.

[0176] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0177] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0178] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0179] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A file read-write method, characterized by, The method is applied to an electronic device, a processor of the electronic device includes a plurality of core complexes (CCXs), and the method comprises the following steps: receiving a target file read-write request; creating a parallel thread group for the target file read-write request; the parallel thread group is used for processing the target file read-write request in parallel; running the parallel thread group and the target file read-write request based on a same CCX in the plurality of CCXs.

2. The method of claim 1, wherein, Before the step of creating the parallel thread group for the target file read-write request, the method further comprises the following steps: obtaining a resource state of the processor; creating the parallel thread group for the target file read-write request according to the resource state.

3. The method of claim 2, wherein, The step of obtaining the resource state of the processor comprises the following steps: obtaining a CPU resource utilization rate of each CCX; if the CPU resource utilization rate of each CCX is lower than a preset threshold, determining that the resource state is a resource sufficient state; otherwise, determining that the resource state is a resource limited state.

4. The method of claim 3, wherein, The step of creating the parallel thread group for the target file read-write request according to the resource state comprises the following steps: if the resource state is the resource sufficient state, creating one parallel thread group for the target file read-write request in each CCX.

5. The method of claim 4, wherein, The step of running the parallel thread group and the target file read-write request based on a same CCX in the plurality of CCXs comprises the following steps: scheduling the target file read-write request to a first CCX in the plurality of CCXs for running; controlling the parallel thread group created in the first CCX to run and prohibiting the parallel thread groups in other CCXs except the first CCX from running.

6. The method of claim 3, wherein, The step of creating the parallel thread group for the target file read-write request according to the resource state comprises the following steps: if the resource state is the resource limited state, creating the parallel thread group on a second CCX; the second CCX is a currently running CCX of the target file read-write request; binding the created parallel thread group and the target file read-write request by a thread group.

7. The method of claim 6, wherein, The step of running the parallel thread group and the target file read-write request based on a same CCX in the plurality of CCXs comprises the following steps: based on a thread group binding relationship, scheduling the target file read-write request and the created parallel thread group to a same CCX for running.

8. The method of claim 1, wherein, The step of running the parallel thread group and the target file read-write request based on a same CCX in the plurality of CCXs comprises the following steps: limiting the scheduling of the target file read-write request to a second CCX, so as to run the parallel thread group and the target file read-write request based on the second CCX; wherein, the second CCX is a currently running CCX of the target file read-write request.

9. The method of claim 8, wherein, The step of limiting the scheduling of the target file read-write request to a second CCX comprises the following steps: creating a target control group and adding the target file read-write request into the target control group; the target control group is used for managing the target file read-write request; limiting the scheduling of the target file read-write request to the second CCX based on the target control group.

10. The method of claim 8, wherein, The step of limiting the scheduling of the target file read-write request to a second CCX comprises the following steps: limiting scheduling of the target file read-write request to the second CCX by a kernel scheduler.

11. The method according to any one of claims 6 to 10, characterized in that, The method further comprises: allocating a dedicated L3 cache for the target file read-write request on the second CCX; dynamically adjusting a size of the dedicated L3 cache according to a data volume size of the target file read-write request.

12. A file read-write device, characterized by comprising: The apparatus is applied to an electronic device, and a processor of the electronic device comprises a plurality of CCXs, and the apparatus comprises: a receiving module, configured to receive a target file read-write request; a creating module, configured to create a parallel thread group for the target file read-write request; the parallel thread group is used to process the target file read-write request in parallel; a running module, configured to run the parallel thread group and the target file read-write request based on a same CCX in the plurality of CCXs.

13. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor implements the steps of the method in any one of claims 1 to 11 when executing the computer program.

14. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the steps of the method in any one of claims 1 to 11 when executed by the processor.

15. A computer program product comprising a computer program, characterized in that, The computer program implements the steps of the method in any one of claims 1 to 11 when executed by the processor.

Images