Data processing method and device

Abstract

This disclosure provides a data processing method applicable to the field of computer technology. The method includes: determining memory data to be transferred from the memory when the remaining memory space is less than a preset threshold; dividing the memory data into multiple image files; and storing each of the multiple image files in parallel to a corresponding storage device. This disclosure also provides a data processing device, a computer system, and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more particularly, to a data processing method and device. Background Art

[0002] In order to improve reading and writing efficiency and speed, the Linux kernel caches files in memory. This part of memory is called Cache Memory. Even after the program ends, Cache Memory will not be released automatically. This will cause the available physical memory to become less after the programs in the Linux system frequently read and write files. When the system's physical memory is insufficient, it is necessary to release part of the space in the physical memory for use by the currently running programs.

[0003] In the process of implementing the concept of the present disclosure, the inventors discovered that there is at least one technical problem in the related art of releasing physical memory at a slow speed. Summary of the Invention

[0004] In view of this, the present disclosure provides a data processing method, a data processing device, a computer system, and a storage medium.

[0005] One aspect of the present disclosure provides a data processing method, comprising:

[0006] When the remaining storage space of the memory is less than a preset threshold, determining the memory data to be transferred in the above memory;

[0007] Dividing the memory data into multiple image files; and

[0008] Each of the plurality of image files is stored in parallel in a corresponding storage device.

[0009] According to an embodiment of the present disclosure, the data processing method further includes:

[0010] In response to a call request for the plurality of image files, determining the image file from each of the plurality of storage devices;

[0011] The plurality of image files are stored in parallel in the memory so that the application program can call the plurality of image files from the memory.

[0012] According to an embodiment of the present disclosure, the image file has a second mapping relationship with the storage device; the method further includes:

[0013] Removing the second mapping relationship between the image file and the storage device;

[0014] A first mapping relationship between the image file and the virtual address is generated, wherein the virtual address includes a virtual address in the memory.

[0015] According to an embodiment of the present disclosure, the determining of the memory data to be transferred in the memory includes:

[0016] determining a target application from at least one application running in the memory;

[0017] Obtaining memory description information of the target application, wherein the memory description information includes at least one type of candidate memory data and memory occupancy corresponding to the candidate memory data;

[0018] The memory data to be transferred is determined from the candidate memory data of the target application according to the memory description information.

[0019] According to an embodiment of the present disclosure, determining the memory data to be swapped out from the candidate memory data of the target application according to the memory description information includes:

[0020] Determining anonymous private memory data from the candidate memory data of the target application according to the memory description information, wherein the anonymous private memory data includes memory data that is not shared with other applications;

[0021] The anonymous private memory data is determined as the memory data that needs to be transferred.

[0022] According to an embodiment of the present disclosure, dividing the memory data into multiple image files includes:

[0023] Divide the above memory data evenly into multiple image files; or

[0024] The memory data is divided into a plurality of mirror files according to the size of the storage space of the storage device.

[0025] According to an embodiment of the present disclosure, the image file has a first mapping relationship with a virtual address, wherein the virtual address includes a virtual address in the memory; the method further includes:

[0026] Release the first mapping relationship between the image file and the virtual address;

[0027] A second mapping relationship between the virtual address and the storage device is generated, wherein the storage device includes a storage device corresponding to the mirror file.

[0028] Another aspect of the present disclosure provides a data processing device, comprising:

[0029] A first determining module is used to determine the memory data to be transferred in the above memory when the memory usage exceeds a preset threshold;

[0030] A sharding module is used to divide the memory data into multiple image files; and

[0031] The first swap-out module is configured to store each of the plurality of image files in parallel to a corresponding storage device.

[0032] Another aspect of the present disclosure provides a computer system storing computer-executable instructions, which are used to implement the method described in the embodiments of the present disclosure when executed.

[0033] Another aspect of the present disclosure provides a computer-readable storage medium, wherein the computer program includes computer-executable instructions, and when the instructions are executed, the instructions are used to implement the method described in the embodiment of the present disclosure.

[0034] According to the embodiments of the present disclosure, because a technical means is adopted to divide memory data into multiple image files and store the multiple image files in corresponding storage devices respectively, the technical problem of slow release of physical memory in the related art is at least partially overcome, thereby achieving the technical effect of accelerating the release of physical memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0036] Figure 1 Schematically illustrates an exemplary system architecture to which the data processing method according to an embodiment of the present disclosure can be applied;

[0037] Figure 2 The following schematically shows a flow chart of a data processing method according to an embodiment of the present disclosure;

[0038] Figure 3 A schematic diagram of a data processing method according to an embodiment of the present disclosure is schematically shown;

[0039] Figure 4 The following schematically shows a flow chart of a data processing method according to another embodiment of the present disclosure;

[0040] Figure 5 The following schematically shows a flow chart of a data processing method according to another embodiment of the present disclosure;

[0041] Figure 6 The following schematically shows a flow chart of a data processing method according to another embodiment of the present disclosure;

[0042] Figure 7 A flowchart of determining memory data to be transferred in the memory according to an embodiment of the present disclosure is schematically shown;

[0043] Figure 8 A flowchart of determining the memory data that needs to be swapped out from the candidate memory data of the target application according to the memory description information is schematically shown according to an embodiment of the present disclosure;

[0044] Figure 9 A block diagram schematically shows a data processing device according to an embodiment of the present disclosure; and

[0045] Figure 10 A block diagram of an electronic device suitable for implementing the above-described method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0046] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0047] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0048] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0049] When expressions such as "at least one of A, B and C, etc." are used, they should generally be interpreted in accordance with the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0050] Swap partition is a virtual memory partition under Linux. Its function is to use virtual storage space as memory after the physical memory is used up. The space used by virtual storage space can be an area on the storage space such as the disk, a partition, a file, or a combination of them. When the system physical memory is insufficient, Linux will save the memory data that is not frequently accessed in the memory to the swap partition, so that the system has more physical memory to serve each process, and when the system needs to access the content stored on the swap partition, the data on the swap partition will be loaded into the memory.

[0051] Generally, swapping technology includes swapping in and swapping out. The process of storing memory data from the main memory to the virtual storage space is called swapping out. Conversely, the process of storing memory data in the virtual storage space back to the main memory is called swapping in.

[0052] However, the Linux kernel's swap technology can only execute single-threaded execution during the swap-out and swap-in processes, and the disk access speed is slow. The kernel's swap technology cannot avoid random access. When system memory is very tight, the frequency of reading and writing the swap partition will be very high, which leads to slow reading and writing of the swap partition and poor performance in scenarios where the system requires fast swap.

[0053] In view of this, and to at least partially address the technical issue of slow physical memory release in related technologies, the present disclosure provides a data processing method applicable to the field of computer technology. The data processing method includes: determining memory data to be transferred from the memory when the remaining memory space is less than a preset threshold; dividing the memory data into multiple image files; and storing each of the multiple image files in parallel to a corresponding storage device. The present disclosure also provides a data processing device, a computer system, and a storage medium.

[0054] Figure 1 The following schematically illustrates an exemplary system architecture 100 to which the data processing method according to an embodiment of the present disclosure can be applied. Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0055] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0056] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0057] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0058] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0059] It should be noted that the data processing method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the data processing device provided in the embodiments of the present disclosure can generally be set in the server 105. The data processing method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the data processing device provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Alternatively, the data processing method provided in the embodiments of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or by other terminal devices different from the terminal devices 101, 102, or 103. Accordingly, the data processing device provided in the embodiments of the present disclosure can also be set in the terminal devices 101, 102, or 103, or in other terminal devices different from the terminal devices 101, 102, or 103.

[0060] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0061] Figure 2 The flowchart of the data processing method according to the embodiment of the present disclosure is schematically shown.

[0062] like Figure 2 As shown, the method includes operations S201 to S203.

[0063] In operation S201, when the remaining storage space of the memory is less than a preset threshold, memory data to be transferred in the memory is determined.

[0064] In the embodiments of the present disclosure, the memory may include a random-access memory (RAM) or a cache memory (Cache).

[0065] According to an embodiment of the present disclosure, the preset threshold can be set according to actual business needs, for example, it can be 20% of the rated storage space of the memory, but is not limited to this. The preset threshold can also be 25%, 30%, or 35% of the rated storage space of the memory.

[0066] According to the embodiments of the present disclosure, the preset threshold of the remaining memory space can be flexibly set by those skilled in the art according to the needs of actual applications. The embodiments of the present disclosure do not specifically limit the value and range of the preset threshold.

[0067] According to an embodiment of the present disclosure, the memory data to be transferred in the memory may include memory data in one or more applications that has not been called for more than a preset time.

[0068] According to an embodiment of the present disclosure, memory data may be stored in units of memory pages, and a memory page may store multiple bytes of data. In an embodiment of the present disclosure, the amount of data that a memory page can store may be set based on actual business needs. For example, a memory page may store 4KB, or 4096 bytes of data, but is not limited thereto. Those skilled in the art may use the command $getconf PAGE_SIZE to dynamically adjust the amount of data that each memory page can store based on actual application needs.

[0069] According to an embodiment of the present disclosure, determining the memory data to be transferred in the memory may include determining a storage address and / or a data size of the memory data to be transferred.

[0070] In operation S202 , memory data is divided into a plurality of image files.

[0071] According to an embodiment of the present disclosure, memory data may include multiple memory pages. After the multiple memory pages are divided into multiple image files, each of the multiple image files may include at least one memory page, and the sum of the number of memory pages included in the multiple image files is equal to the number of memory pages included in the memory data.

[0072] In operation S203 , each of the multiple image files is stored in parallel in a corresponding storage device.

[0073] According to an embodiment of the present disclosure, storing the image file in the storage device may be implemented based on the swap technology.

[0074] According to an embodiment of the present disclosure, after the memory data is divided into a plurality of image files, a plurality of storage devices matching the number of the image files may be obtained, and then a corresponding storage device may be allocated to each of the plurality of image files.

[0075] According to an embodiment of the present disclosure, the storage device may include a disk or an external storage device, and the external storage device may include a USB flash drive, a mobile hard disk, etc.

[0076] In the embodiments of the present disclosure, because a technical means is adopted to divide memory data into multiple image files and store the multiple image files in corresponding storage devices respectively, the technical problem of slow release of physical memory in the related art is at least partially overcome, thereby achieving the technical effect of accelerating the release of physical memory.

[0077] In an embodiment of the present disclosure, memory data may be evenly divided into multiple mirror files.

[0078] According to an embodiment of the present disclosure, the memory data may include, for example, 3000 memory pages. Thus, the 3000 memory pages may be evenly divided into 3 mirror files, wherein each of the 3 mirror files may include 1000 memory pages.

[0079] According to an embodiment of the present disclosure, by evenly dividing memory data into multiple mirror files, each mirror file includes an equal number of memory pages. In the process of storing multiple mirror files in corresponding storage devices respectively, the time consumed by swapping out each storage device is close to or the same, thereby ensuring the synchronization of releasing physical memory.

[0080] In an embodiment of the present disclosure, memory data may be divided into a plurality of mirror files according to the size of the storage space of the storage device.

[0081] According to an embodiment of the present disclosure, multiple storage devices can be obtained before dividing the memory data into multiple image files, wherein the storage space sizes of the multiple storage devices may be different, so that the memory data can be divided into multiple image files according to the size of the storage space of the storage devices.

[0082] According to an embodiment of the present disclosure, memory data may, for example, include 10,000 memory pages. Before dividing the memory data into multiple mirror files, five storage devices may be obtained first, wherein the ratio of the storage space sizes of the five storage devices may be 1:1:2:3:3. Thus, the memory data may be divided into five mirror files, wherein the number of memory pages included in the five mirror files may be 1,000, 1,000, 2,000, 3,000, and 3,000, respectively.

[0083] Figure 3 The figure schematically shows a data processing method according to an embodiment of the present disclosure.

[0084] exist Figure 3 In the figure, 301 can be the memory data to be transferred in the determined memory. After the memory data 301 to be transferred is determined, the memory data 301 to be transferred can be divided into multiple image files 302 to 30N, and then the multiple image files 302 to 30N can be stored in parallel in the corresponding storage devices 3021 to 30N1.

[0085] According to an embodiment of the present disclosure, by dividing memory data into multiple mirror files according to the size of the storage space of the storage device, physical memory can still be released when the number of storage devices or storage space is insufficient, thereby improving the robustness of the data processing method provided by the embodiment of the present disclosure.

[0086] Reference below Figures 4 to 8 , combined with specific embodiments Figure 2 and Figure 3 The method shown is further explained.

[0087] According to an embodiment of the present disclosure, the image file has a first mapping relationship with a virtual address, wherein the virtual address includes a virtual address in a memory.

[0088] According to an embodiment of the present disclosure, memory data can be divided into a plurality of mirror files, and a first mapping relationship between the mirror files and the virtual addresses can be generated.

[0089] According to an embodiment of the present disclosure, the first mapping relationship represents the storage location of the image file in the memory.

[0090] According to an embodiment of the present disclosure, for example, image file A has a first mapping relationship with the virtual address 0×10001000 in the memory, indicating that before image file A is swapped out to the storage device, image file A is stored at the address 0×10001000 in the memory.

[0091] Figure 4 The flowchart schematically shows a data processing method according to another embodiment of the present disclosure.

[0092] like Figure 4 As shown, the method includes operations S401 to S402.

[0093] In operation S401 , a first mapping relationship between an image file and a virtual address is released.

[0094] In operation S402 , a second mapping relationship between a virtual address and a storage device is generated, wherein the storage device includes a storage device corresponding to the mirror file.

[0095] According to an embodiment of the present disclosure, before swapping out multiple image files to corresponding storage devices respectively, the first mapping relationship between the image file and the virtual address can be released first. However, releasing the first mapping relationship does not mean deleting the first mapping relationship. After releasing the first mapping relationship, the first mapping relationship can be stored in a mapping relationship table, so that when the image file needs to be swapped from the storage device to the memory, the first mapping relationship can be quickly obtained from the mapping relationship table, so that the image file can be restored from the storage device to the corresponding storage location of the memory.

[0096] According to an embodiment of the present disclosure, after the first mapping relationship between the image file and the virtual address is released, a second mapping relationship between the image file and the corresponding storage device can be generated, wherein the second mapping relationship can represent the storage address of the image file in the storage device.

[0097] According to an embodiment of the present disclosure, after generating a second mapping relationship between each image file and the corresponding storage device, multiple image files can be swapped out in parallel to corresponding storage locations in the corresponding storage devices according to the second mapping relationship.

[0098] According to an embodiment of the present disclosure, by swapping out the image file to the storage device according to the second mapping relationship representing the storage location, the storage location of each image file can be clearly identified, so that when the corresponding image file needs to be called, the storage location of the image file can be quickly determined and swapped into the memory.

[0099] Figure 5 The flowchart schematically shows a data processing method according to another embodiment of the present disclosure.

[0100] like Figure 5As shown, the method includes operations S501 to S502.

[0101] In operation S501 , in response to a call request for a plurality of image files, an image file is determined from each of a plurality of storage devices.

[0102] In operation S502 , the plurality of image files are stored in parallel in the memory so that the application program can call the plurality of image files from the memory.

[0103] According to an embodiment of the present disclosure, when an application system has a page fault exception, that is, the memory page that the application needs to call is stored in the storage device, it is necessary to call the corresponding mirror file from the storage device and swap the mirror file into the memory so that the application can call the memory data corresponding to the mirror file.

[0104] According to an embodiment of the present disclosure, the mirror file has a second mapping relationship with the storage device.

[0105] According to an embodiment of the present disclosure, the second mapping relationship may represent a storage address of the image file in the storage device.

[0106] Figure 6 The flowchart schematically shows a data processing method according to another embodiment of the present disclosure.

[0107] like Figure 6 As shown, the method includes operations S601 to S602.

[0108] In operation S601, the second mapping relationship between the image file and the storage device is released.

[0109] In operation S602 , a first mapping relationship between an image file and a virtual address is generated, wherein the virtual address includes a virtual address in a memory.

[0110] According to an embodiment of the present disclosure, the first mapping relationship stored before the image file is swapped out to the storage device may be searched in the mapping relationship table, thereby generating the first mapping relationship.

[0111] According to an embodiment of the present disclosure, it is also possible to first obtain a free storage address in the memory, and then generate the first mapping relationship according to the free storage address.

[0112] According to an embodiment of the present disclosure, after the first mapping relationship is generated, the image file can be swapped into a corresponding storage address in the memory according to the first mapping relationship.

[0113] According to an embodiment of the present disclosure, by generating a first mapping relationship and swapping the image file into the memory according to the first mapping relationship, the storage address of the swapped-in image file can be clearly determined, so that the application can call the memory page in the image file according to the storage address.

[0114] Figure 7 The flowchart of determining memory data to be transferred in the memory according to an embodiment of the present disclosure is schematically shown.

[0115] like Figure 7 As shown, the method includes operations S701 to S703.

[0116] In operation S701 , a target application is determined from at least one application running in a memory.

[0117] According to an embodiment of the present disclosure, a target application may be determined from at least one application running in a memory according to an idle time of the application.

[0118] According to an embodiment of the present disclosure, since an application in an idle state does not need to quickly call a memory page, the application in an idle state can be determined as a target application.

[0119] According to an embodiment of the present disclosure, for example, there are three applications A, B, and C running in the memory, among which application C has the longest idle time, so application C can be determined as the target application.

[0120] In operation S702 , memory description information of a target application is obtained, where the memory description information includes at least one type of candidate memory data and memory occupancy corresponding to the candidate memory data.

[0121] According to an embodiment of the present disclosure, memory description information may include candidate memory data stored in a storage device and its memory occupancy, memory data shared with other applications in the memory, candidate memory data stored in the memory and its memory occupancy, candidate memory data shared with other applications and not rewritten and its memory occupancy, anonymous private candidate memory data and its memory occupancy, candidate memory data shared with other applications and rewritten and its memory occupancy, etc.

[0122] According to an embodiment of the present disclosure, the memory description information included in the smaps memory description information can be represented by Table (1).

[0123] Table (1)

[0124]

[0125] In operation S703 , memory data that needs to be transferred is determined from candidate memory data of the target application according to the memory description information.

[0126] Figure 8 The flowchart of determining memory data that needs to be swapped out from candidate memory data of a target application according to memory description information according to an embodiment of the present disclosure is schematically shown.

[0127] like Figure 8 As shown, the method includes operations S801 to S802.

[0128] In operation S801 , anonymous private memory data is determined from candidate memory data of a target application according to memory description information, wherein the anonymous private memory data includes memory data that is not shared with other applications.

[0129] In operation S802 , anonymous private memory data is determined as memory data that needs to be transferred.

[0130] According to an embodiment of the present disclosure, anonymous private memory data not shared with other applications is determined as memory data to be transferred, so that the process of swapping out the memory data to be transferred will not affect the call of other applications to the memory data to be transferred.

[0131] Figure 9 The block diagram schematically shows a data processing device according to an embodiment of the present disclosure.

[0132] like Figure 9 As shown, the data processing device 900 may include a first determining module 901 , a sharding module 902 and a first swapping module 903 .

[0133] The first determining module 901 is configured to determine memory data to be transferred in the memory when the memory occupancy exceeds a preset threshold.

[0134] The fragmentation module 902 is used to divide the memory data into multiple image files.

[0135] The first swap-out module 903 is configured to store each of the multiple image files in parallel to a corresponding storage device.

[0136] It should be noted that the embodiments of the device part of the present disclosure are the same or similar to the embodiments of the method part of the present disclosure, and the present disclosure will not repeat them here.

[0137] According to an embodiment of the present disclosure, the data processing device 900 further includes a second determining module and a first calling module.

[0138] The second determining module is configured to determine an image file from each storage device of the plurality of storage devices in response to a call request for the plurality of image files.

[0139] The first calling module is used to store multiple image files in parallel to the memory so that the application program can call the multiple image files from the memory.

[0140] According to an embodiment of the present disclosure, the mirror file has a second mapping relationship with the storage device.

[0141] According to an embodiment of the present disclosure, the data processing device 900 further includes a first releasing module and a first generating module.

[0142] The first releasing module is used to release the second mapping relationship between the image file and the storage device.

[0143] The first generating module is used to generate a first mapping relationship between the image file and the virtual address, wherein the virtual address includes a virtual address in the memory.

[0144] According to an embodiment of the present disclosure, the first determining module 901 includes a first determining unit, a first acquiring unit, and a second determining unit.

[0145] The first determining unit is configured to determine a target application from at least one application running in the memory.

[0146] The first acquiring unit is configured to acquire memory description information of a target application, wherein the memory description information includes at least one type of candidate memory data and memory occupancy corresponding to the candidate memory data.

[0147] The second determining unit is configured to determine the memory data that needs to be transferred from the candidate memory data of the target application according to the memory description information.

[0148] According to an embodiment of the present disclosure, the second determining unit includes a first determining subunit and a second determining subunit.

[0149] A first determining subunit is configured to determine anonymous private memory data from candidate memory data of a target application according to the memory description information, wherein the anonymous private memory data includes memory data that is not shared with other applications;

[0150] The second determining subunit is configured to determine the anonymous private memory data as memory data that needs to be transferred.

[0151] According to an embodiment of the present disclosure, the sharding module 902 includes a first sharding unit or a second sharding unit.

[0152] The first sharding unit is used to evenly divide the memory data into multiple image files.

[0153] The second sharding unit is used to divide the memory data into multiple mirror files according to the size of the storage space of the storage device.

[0154] According to an embodiment of the present disclosure, the image file has a first mapping relationship with a virtual address, wherein the virtual address includes a virtual address in a memory.

[0155] According to an embodiment of the present disclosure, the data processing device 900 further includes a second releasing module and a second generating module.

[0156] The second releasing module is used to release the first mapping relationship between the image file and the virtual address.

[0157] The second generating module is configured to generate a second mapping relationship between the virtual address and the storage device, wherein the storage device includes a storage device corresponding to the mirror file.

[0158] According to the modules, submodules, and units of the embodiments of the present invention, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, submodules, and units of the embodiments of the present invention, any one or more can be split into multiple modules for implementation. According to the modules, submodules, and units of the embodiments of the present invention, any one or more can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in an appropriate combination of any of them. Alternatively, according to the modules, submodules, and units of the embodiments of the present invention, one or more can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is run.

[0159] For example, any multiple of the first determination module 901, the slicing module 902 and the first swapping module 903 can be combined into one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first determination module 901, the slicing module 902 and the first swapping module 903 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the first determining module 901, the sharding module 902 and the first swapping module 903 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0160] Figure 10 A block diagram of an electronic device suitable for implementing the above-described method according to an embodiment of the present disclosure is schematically shown. Figure 10 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0161] like Figure 10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0162] Various programs and data required for the operation of the electronic device 1000 are stored in the RAM 1003. The processor 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 1002 and / or the RAM 1003. It should be noted that the programs may also be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0163] According to an embodiment of the present disclosure, electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to bus 1004. System 1000 may also include one or more of the following components connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a LAN card or a modem. Communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in drive 1010 as needed, so that computer programs read therefrom can be installed into storage section 1008 as needed.

[0164] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0165] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0166] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0167] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1002 and / or the RAM 1003 described above and / or one or more memories other than the ROM 1002 and the RAM 1003 .

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] Those skilled in the art will appreciate that various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, even if such combinations or combinations are not explicitly described in this disclosure. In particular, various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0170] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data processing method, comprising: When the remaining storage space of the memory is less than a preset threshold, determining the memory data to be transferred in the memory; Dividing the memory data into a plurality of mirror files, wherein the mirror files have a second mapping relationship with the storage device; as well as According to the second mapping relationship, each image file in the multiple image files is stored in parallel in a corresponding storage device, wherein the storing of the image files in the storage device is implemented based on a swap technology.

2. The method according to claim 1, further comprising: In response to a call request for the plurality of image files, determining the image file from each of the plurality of storage devices; The plurality of image files are stored in parallel in the memory so that the application program can call the plurality of image files from the memory.

3. The method according to claim 2, wherein: The method further comprises: Release the second mapping relationship between the image file and the storage device; A first mapping relationship between the image file and a virtual address is generated, wherein the virtual address includes a virtual address in the memory.

4. The method according to claim 1, wherein The determining of the memory data to be transferred in the memory includes: determining a target application from at least one application running in the memory; Obtaining memory description information of the target application, wherein the memory description information includes at least one type of candidate memory data and memory occupancy corresponding to the candidate memory data; The memory data to be transferred is determined from the candidate memory data of the target application according to the memory description information.

5. The method according to claim 4, wherein Determining the memory data to be transferred from the candidate memory data of the target application according to the memory description information includes: determining anonymous private memory data from candidate memory data of the target application according to the memory description information, wherein the anonymous private memory data includes memory data that is not shared with other applications; The anonymous private memory data is determined as the memory data to be transferred.

6. The method according to claim 1, wherein The dividing the memory data into a plurality of image files comprises: Divide the memory data evenly into multiple image files; or The memory data is divided into a plurality of mirror files according to the size of the storage space of the storage device.

7. The method according to claim 1, wherein The image file has a first mapping relationship with a virtual address, wherein the virtual address includes a virtual address in the memory; the method further includes: Release the first mapping relationship between the image file and the virtual address; A second mapping relationship between the image file and the storage device is generated, wherein the storage device includes a storage device corresponding to the image file.

8. A data processing device comprising: A first determining module, configured to determine memory data to be transferred in the memory when the memory occupancy exceeds a preset threshold; A sharding module, configured to divide the memory data into a plurality of mirror files, wherein the mirror files have a second mapping relationship with a storage device; as well as The first swap module is configured to store each of the plurality of image files in parallel to a corresponding storage device according to the second mapping relationship, wherein storing the image files to the storage device is implemented based on a swap technology.

9. A computer system comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 7.

Images