Data processing program, data processing device, and data processing method

Abstract

To provide a data processing program, a data processing device and a data processing method which can achieve a long life of a flash memory.SOLUTION: In a data processing device 10, a first storage medium includes a volatile storage unit, and a non-volatile storage unit. A data processing program allows a processing circuit to execute: an overlay application program capable of applying overlay processing to the first storage medium; a data acquisition program capable of acquiring data from a data generation device; a data update program capable of executing data update; a data storage program capable of executing storage of data in the first storage medium; a release determination program capable of determining release of the overlay processing; an overlay release program capable of releasing application of the overlay processing; a lapsed time determination program capable of determining whether data is stored in the non-volatile storage unit according to a lapsed time of data; and a data perpetuation program capable of storing data to be perpetuated in the non-volatile storage unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a data processing program, a data processing device, and a data processing method. [Background technology]

[0002] Many devices used in the Internet of Things (IoT) use flash memory as a secondary storage device. However, flash memory has a limited lifespan, and in industrial and infrastructure fields, it is necessary to reduce the number of writes to memory in order to extend its lifespan. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-101917 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide a data processing program, a data processing device, and a data processing method that can extend the life of flash memory. [Means for solving the problem]

[0005] According to an embodiment, a data processing program is provided in which a storage medium includes a volatile memory unit and a non-volatile memory unit, and a processing circuit is caused to designate the volatile memory unit as an upper layer and the non-volatile memory unit as a lower layer, apply overlay processing to at least a portion of the storage medium, store data in the volatile memory unit, cancel the application of the overlay processing, and, based on predetermined conditions, distinguish between a first data group that is to be persisted and a second data group that is not to be persisted, and store the data of the first data group in the non-volatile memory unit. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of an outline of a data processing device according to an embodiment. [Figure 2] 1 is a flowchart showing an example of the flow of a data processing method according to an embodiment. [Figure 3] FIG. 1 is a schematic diagram showing processing of a storage medium. [Figure 4] FIG. 2 is a block diagram showing an example of an outline of a data processing device and a server device according to the embodiment. [Figure 5] 10 is a flowchart showing a modified example of the flow of the data processing method according to the embodiment. [Figure 6] FIG. 10 is a block diagram showing a schematic modification of the data processing device according to the embodiment. [Figure 7] 10 is a flowchart showing a modified example of the flow of the data processing method according to the embodiment. [Figure 8] FIG. 10 is a block diagram showing a schematic modification of the data processing device according to the embodiment. [Figure 9] 10 is a flowchart showing a modified example of the flow of the data processing method according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, embodiments will be described with reference to the drawings. In the following description, components that perform the same or similar functions are designated by the same reference numerals throughout the drawings, and duplicate descriptions will be omitted. Each drawing is a schematic diagram for explaining and facilitating understanding of the embodiments, and the shapes, dimensions, ratios, etc. may differ from those of an actual device. However, these may be appropriately modified in design, taking into consideration the following description and known techniques.

[0008] The present invention uses a method of applying an overlay to a storage area as a method for reducing the amount of data written to a nonvolatile storage medium such as a flash memory. The upper layer of the overlay allows both reading and writing of data, while the lower layer allows only reading of data. When a nonvolatile storage medium is specified as the lower layer of the overlay and a volatile storage medium is specified as the upper layer, data is not written to the nonvolatile storage medium, and data is written only to the volatile storage medium. This makes it possible to reduce the amount of data written to the nonvolatile storage medium.

[0009] When a device equipped with both volatile and non-volatile storage media is powered off, the data in the volatile storage media is erased. Therefore, to retain the data permanently, the overlay must be removed and the data must be saved to the non-volatile storage media. However, the volatile storage media contains a mixture of data that needs to be persisted and unnecessary data, and writing unnecessary data to the non-volatile storage media can lead to deterioration of the flash memory. Therefore, this invention employs a method that uses an overlay while saving only the data that needs to be persisted to the non-volatile storage media.

[0010] Furthermore, the conditions for determination differ depending on the embodiment, and therefore the types and quantities of data to be persisted may not be the same in each embodiment but may be different.

[0011] The term "permanent" used here means storing the data in a non-volatile storage unit.

[0012] (First embodiment) 1 is a block diagram showing an example of an outline of a data processing device according to an embodiment. The data processing device 10 includes a data processing program 100, a first storage medium 300, and a processing circuit 400. The data processing device 10 may further include a timer 500 and a data generation device 600. The data processing program 100 includes an overlay application program 120 that can apply overlay processing to the first storage medium 300, a data acquisition program 130A that can acquire data from the data generation device 600, a data update program 130B that can update data, a data storage program 140 that can store data in the first storage medium 300, a cancellation determination program 150 that can determine whether to cancel the overlay processing, an overlay cancellation program 160 that can cancel the application of the overlay processing, an elapsed time determination program 170 that can determine whether to store data in a non-volatile storage unit based on the elapsed time of the data, and a data persistence program 180 that can store data to be persisted in a non-volatile storage unit. The first storage medium 300 includes a volatile storage unit 310 and a non-volatile storage unit 320.

[0013] The data processing program 100 can be stored in the first storage medium 300, or in another storage medium not shown. Furthermore, the data processing program 100 only needs to be able to execute instructions issued from the processing circuit 400. Therefore, the programs may be stored in different storage media, or may be run in the cloud.

[0014] The first storage medium 300 is a storage device known as a main storage device or an auxiliary storage device. The first storage medium 300 is, for example, a semiconductor memory. The volatile storage unit 310 is, for example, a random access memory (RAM). The nonvolatile storage unit 320 is, for example, a flash memory. However, to prevent deterioration of the flash memory, an overlay technique is applied to the first storage medium 300. Here, the volatile storage unit 310 is the upper layer, and the nonvolatile storage unit 320 is the lower layer. The overlay technique can be applied to only a portion of the storage medium. Therefore, data that must be persisted, such as software, may be included in the OS (Operating System) of the first storage medium 300 or may be included in another storage medium. In the former case, the overlay is applied to a portion of the storage medium, excluding the storage area for data that must be persisted. In the latter case, the overlay technique can be applied to the entire storage medium.

[0015] The processing circuit 400 is composed of a processor or an integrated circuit, etc., and the processor, etc. constituting the processing circuit 400 includes any of a CPU (Central Processing Unit), ASIC (Application Specific Integrated Circuit), microcontroller unit, FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), etc. The processing circuit 400 may be composed of one processor, etc., or may be composed of multiple processors, etc. The processing circuit 400 executes the data processing program 100 to perform the processing described below in controlling the first storage medium 300.

[0016] The timer 500 can measure the time when the data is updated.

[0017] The data generating device 600 generates data to be stored in the first storage medium. The generated data includes data A and data B, which will be described later.

[0018] The flow of the data processing method according to the embodiment will be described with reference to FIGS. 2 and 3. Here, the case where there are two types of data will be described, but there may be more than two types of data. The same applies to subsequent embodiments. FIG. 2 is a flowchart showing an example of the flow of the data processing method according to the embodiment. Note that this flowchart is only an example, and the order of processing is not limited as long as the required processing results can be obtained. Furthermore, each processing result may be stored sequentially in the first storage medium 300, and each step may acquire the processing result by referring to the first storage medium 300. The same applies to subsequent flowcharts.

[0019] FIG. 3 is a schematic diagram showing data processing on the first storage medium 300 in the first embodiment. FIG. 3(a) shows the state of the first storage medium 300 at S6 in FIG. 2 in the data processing method of the embodiment. The volatile storage unit 310 stores multiple pieces of data. The multiple pieces of data include data A, data B, and data A' and data B', which are obtained by updating data A and data B, respectively. FIG. 3(b) shows the state of the first storage medium 300 after the data processing method of the embodiment is performed (S9). The volatile storage unit 310 stores a first data group and a second data group. The non-volatile storage unit 320 stores a first data group. The first data group includes data A and data A', and the second data group includes data B and data B'. In FIG. 3(a), the volatile storage unit 310 may store only data that becomes the first data group in FIG. 3(b), such as data A and / or data A'.

[0020] Here, data A refers to data that is determined to need to be written to the nonvolatile storage unit 320 in the determination of persistence, which will be described later. For example, data A is important data such as data related to the destination of a software path, certificates, and program code. On the other hand, data B refers to data that is determined not necessarily to need to be written to the nonvolatile storage unit 320, and is data that is updated more frequently than data A. For example, data B is data that is less important than data A, such as maintenance data and temporary files. Reducing the writing of data B can extend the life of the nonvolatile storage unit 320.

[0021] Here we will explain an example where the time elapsed since the last update of data is used as a criterion for determining whether to persist data. When there are multiple types of data, the shorter the elapsed time, the more frequently the data has been updated, and the data is considered to be temporary and is not subject to persistence.

[0022] Returning to FIG. 2, in S2, the processing circuit 400 reads and executes the overlay application program 120 to apply overlay processing to the first storage medium 300, with the volatile storage unit 310 as the upper layer and the non-volatile storage unit 320 as the lower layer. While the overlay processing is being applied, data is stored only in the volatile storage unit 310. If no data exists in the volatile storage unit 310, the process proceeds to S3A. If data already exists in the volatile storage unit 310, the process can proceed to S3A to obtain new data, or to S3B to update the existing data.

[0023] In S3A, the processing circuit 400 acquires the data generated by the data generating device 600 by reading and executing the data acquiring program 130A.

[0024] In S3B, the processing circuit 400 reads and executes the data update program 130B to update the data stored in the volatile storage unit 310. The data update may be performed by overwriting and saving, or by creating new updated data. The latter will be described here.

[0025] In S4, the processing circuit 400 reads and executes the data storage program 140, thereby storing the data acquired in S3A and the data updated in S3B in the volatile storage unit 310. At this time, the acquired date and time is additionally stored using the timer 500. This date and time is the date and time when the data storage program 140 stores the data in the volatile storage unit 310, and is used in the data perpetuation determination described below.

[0026] In S5, the processing circuit 400 reads and executes the cancellation determination program 150 to determine whether or not to cancel the application of overlay processing. The timing for the determination can be, for example, a change in the date. When the timing for the determination is a change in the date, for example, the timer 500 is used to determine whether or not the date has changed. If the date has changed, the process proceeds to S6 (Yes); if not, the process returns to S3A or S3B (No). Data acquisition and / or updating and storage continues until the date changes.

[0027] In S6, the processing circuit 400 reads and executes the overlay removal program 160 to remove the overlay processing from the first storage medium 300. After this, no data is acquired or updated. In addition, data can be written to the non-volatile storage unit 320.

[0028] In S7, the processing circuit 400 reads and executes the elapsed time determination program 170 to determine whether to persist each piece of data in the volatile storage unit 310, in other words, whether to store each piece of data in the volatile storage unit 310 in the nonvolatile storage unit 320. If the elapsed time from the last update time of the data to the execution of the elapsed time determination program 170 is longer than a predetermined threshold (Yes), proceed to S8. If the elapsed time is shorter than the threshold (No), return to S2. This determination is made for all data. Data not stored in the nonvolatile storage unit 320 here may be subject to persistence when the next determination is made after returning to S2. The same threshold value can be used for all data in the volatile storage unit 310. The threshold value can be changed as appropriate depending on the type of data and folder, and the capacity of the first storage medium 300. The same applies to subsequent thresholds.

[0029] This will be explained using Figure 3(b). Data A has a longer elapsed time because it is updated less frequently than data B. The first data group includes data that is not updated frequently, such as data A and data A', which is an updated version of data A.

[0030] In S8, the processing circuit 400 reads and executes the data perpetuation program 180, thereby copying only the first data group from the data stored in the volatile storage unit 310 to the non-volatile storage unit 320. It is sufficient that the first data group stored in the volatile storage unit 310 is stored in the non-volatile storage unit 320, and writing may be performed.

[0031] In this way, according to the data processing program of the first embodiment, data that needs to be persisted is separated from data that does not need to be persisted in the storage area to which overlay processing has been applied, and then the overlay processing is temporarily canceled and only the necessary data is written to the non-volatile storage unit 320, thereby making it possible to suppress deterioration of the flash memory and extend its lifespan.

[0032] (Second embodiment) The data processing method according to the second embodiment is intended to handle a container platform, which is a platform that supports application development and operation using container virtualization technology, a type of server virtualization technology. The data (container) output by the container platform is a combination of the device's operating environment and the application. To execute the container, a container image, which is data such as configuration files and program source code, is required. When the container is executed, a container instance, which is the environment in which the application actually runs, is generated. The state of the container instance may change during execution of the container image, and application log files, temporary files, and the like may be generated. Therefore, the container instance does not need to be stored persistently; only the container image needs to be stored. In other words, the container image is data that needs to be persisted, while the container instance is data that does not need to be persisted. In the first embodiment, the first data group corresponds to the container image, and the second data group corresponds to the container instance.

[0033] 4 is a block diagram showing an example of an outline of a data processing device and a server device according to an embodiment. The data processing device 10 includes a data processing program 100, a first storage medium 300, a processing circuit 400, and a first communication module 700. The server device 11 includes a second storage medium 301, a control circuit 401, and a second communication module 701. The second storage medium 301 includes a container image. The data processing program 100 includes an overlay application program 120, a container update determination program 112 capable of determining whether a container image in the second storage medium 301 has been updated, a container acquisition program 132A capable of acquiring a container from the second storage medium 301, a container startup program 132B capable of starting the acquired container, a container storage program 142 executable to store a container image or a container instance in a volatile storage unit 310, a data update stop program 152 capable of stopping data updates of the container, an overlay removal program 160, and a container persistence program 182 executable to identify a container image stored in the volatile storage unit 310 and store the container image in a non-volatile storage unit 320.

[0034] The second storage medium 301 is a storage device called a main storage device or an auxiliary storage device. The second storage medium 301 may be a magnetic disk, an optical disk (CD-ROM, CD-R, DVD, etc.), a magneto-optical disk (MO, etc.), a semiconductor memory, or the like.

[0035] The control circuit 401 is composed of a processor or an integrated circuit, etc., and the processor constituting the control circuit 401 includes any of a CPU, ASIC, microcomputer, FPGA, DSP, etc. The control circuit 401 may be composed of one processor, etc., or may be composed of multiple processors, etc. The control circuit 401 performs the below-described processing for updating the container image stored in the second storage medium 301 in response to an instruction from the processing circuit 400 in the data processing device 10.

[0036] The first communication module 700 and the second communication module 701 are configured by a communication interface, etc. The processing circuit 400 communicates with the second communication module 701 of the server device 11 via the first communication module 700.

[0037] The flow of the data processing method according to the embodiment will be described. Here, Fig. 5 is a flowchart showing a modified example of the flow of the data processing method according to the embodiment. Since S2 and S6 are the same as those in the data processing method described in the first embodiment, S22 to S25 and S28 will be described here.

[0038] In S22, the processing circuit 400 reads and executes the container update determination program 112, thereby determining whether the container image stored in the second storage medium 301 has been updated using the control circuit 401. If the container image has been updated (Yes), proceed to S23A. If the container image has not been updated (No), repeat S22 until the container image has been updated.

[0039] In S23A, the processing circuitry 400 reads and executes the container acquisition program 132A, thereby acquiring a container image from the second storage medium 301 using the control circuitry 401.

[0040] In S23B, the processing circuit 400 starts the container by reading and executing the container start-up program 132B. When the container is started, a container image and a container instance, which is a collection of temporary data required for the container to operate, are generated.

[0041] In S24, the processing circuit 400 reads and executes the container storage program 142 to store the container image and container instance generated in S23 in the volatile storage unit 310.

[0042] In S25, the processing circuit 400 stops updating the container image by reading and executing the data update stop program 152. After this, acquisition of new data and updating of data are not performed.

[0043] In S28, the processing circuit 400 reads and executes the container perpetuation program 182 to identify whether the data stored in the volatile storage unit 310 is a container image, and copies only the container image to the non-volatile storage unit 320. It is sufficient that the container image stored in the volatile storage unit 310 is stored in the non-volatile storage unit 320, and writing or saving may be performed.

[0044] With the data processing program configured as described above, writing only the container image to the non-volatile storage unit 320 can suppress the writing of data that does not need to be made persistent, thereby avoiding pressure on the capacity of the flash memory.

[0045] (Third embodiment) As a modification of the first embodiment, an example will be described in which data capacity is used as a criterion for determining whether to persist data. Here, when comparing different types of data, the data with the larger capacity is selected as the target for persistence. This is because the larger the data capacity, the more information it contains.

[0046] 6 is a block diagram showing a schematic modification of the data processing device according to the third embodiment. The data processing device 10 according to the third embodiment includes a data capacity determination program 173 that determines whether to persist data according to the capacity of the data, instead of the elapsed time determination program 170 in the data processing program 100 of FIG.

[0047] The flow of the data processing method according to the embodiment will be described. Fig. 7 is a flowchart showing a modified example of the data processing method according to the embodiment. The flowchart in Fig. 7 is the same as the flowchart in Fig. 2 except that it includes S34 and S37 instead of S4 and S7 in the flowchart in Fig. 2. Therefore, S34 and S37 will be described here.

[0048] In S34, the processing circuit 400 reads and executes the data storage program 140 to write the data into the volatile storage unit 310. At this time, the data capacity is increased and stored.

[0049] In S37, the processing circuit 400 reads and executes the data capacity determination program 173 to determine whether or not to make each piece of data in the volatile storage unit 310 permanent. If the capacity of each piece of data is greater than a predetermined threshold (Yes), the process proceeds to S8. If the data capacity is less than the threshold (No), the process returns to S2. This determination is made for all data.

[0050] With the data processing program configured as described above, the determination of persistence is based on the data capacity, so the only time information is exchanged between the timer 500 and the processing circuit 400 is when determining the timing of the date change in S5, reducing the number of times information is exchanged.

[0051] (Fourth embodiment) As a modification of the first embodiment, an example will be described in which data creation time is used as a criterion for determining whether to persist data. Data creation time refers to the time required from when the data is first acquired until a decision is made on whether to persist the data. When comparing different types of data, the data with the longer creation time is selected as the target for persistence. This is because the longer the creation time of the data, the more information it contains.

[0052] 8 is a block diagram showing a schematic modification of the data processing device according to the fourth embodiment. The data processing device according to the fourth embodiment includes a creation time determination program 174 that determines whether to persist data depending on the creation time of the data, instead of the elapsed time determination program 170 in the data processing program 100 of FIG.

[0053] The flow of the data processing method according to the embodiment will be described. Fig. 9 is a flowchart showing a modified example of the data processing method according to the embodiment. The flowchart in Fig. 9 is the same as the flowchart in Fig. 2 except that it includes S44 and S47 instead of S4 and S7 in the flowchart in Fig. 2. Therefore, S44 and S47 will be described here.

[0054] In S44, the processing circuit 400 reads and executes the data storage program 140, thereby writing the results of the data processing to the volatile storage unit 310. At this time, the data creation time is added to the results of the data processing and stored. The data creation time refers to the time from when the data is first acquired from the data generating device 600 in S3 to when the data is stored in the volatile storage unit 310. Each of these times can be obtained using the timer 500.

[0055] In S47, the processing circuit 400 reads and executes the creation time determination program 174 to determine whether to make each piece of data in the volatile storage unit 310 permanent. If the creation time of each piece of data is longer than a predetermined threshold (Yes), the process proceeds to S8. If the creation time is shorter than the threshold (No), the process returns to S2.

[0056] With the data processing program configured as described above, it is possible to determine that old data is to be made permanent.

[0057] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims.

[0058] The following describes the invention in terms of embodiments.

[0059] <1> The storage medium includes a volatile storage unit and a nonvolatile storage unit, The processing circuit designating the volatile storage unit as an upper layer and the non-volatile storage unit as a lower layer, and applying an overlay process to at least a part of the storage medium; storing data in the volatile storage unit; Cancel application of the overlay processing, Based on a predetermined condition, the data is classified into a first data group that is a target for persistence and a second data group that is not a target for persistence; storing the data of the first data group in the nonvolatile storage unit; Data processing program.

[0060] <2> The predetermined condition is the time elapsed since the last update date and time of the data. <1> The data processing program according to claim 1.

[0061] <3> The data in the first data group is a container image. <1> or <2> The data processing program according to claim 1.

[0062] <4> The predetermined condition is a data volume of the data. <1> from <3> 10. The data processing program according to claim 1, wherein:

[0063] <5> The predetermined condition is a creation time of the data. <1> from <4> 10. The data processing program according to claim 1, wherein:

[0064] <6> a storage medium including a volatile storage unit and a nonvolatile storage unit; a processing circuit; Equipped with The processing circuitry designating the volatile storage unit as an upper layer and the non-volatile storage unit as a lower layer, and applying an overlay process to at least a portion of the storage medium; instructing the volatile storage unit to store data; Cancel application of the overlay processing; Based on a predetermined condition, the data is classified into a first data group that is a target for persistence and a second data group that is not a target for persistence; instructing the nonvolatile storage unit to store the data of the first data group; Data processing device.

[0065] <7> The storage medium includes a volatile storage unit and a nonvolatile storage unit, designating the volatile storage unit as an upper layer and the non-volatile storage unit as a lower layer, and applying an overlay process to at least the storage medium; storing data in the volatile storage; canceling the application of the overlay processing; A step of distinguishing the data into a first data group that is a target for persistence and a second data group that is not a target for persistence based on a predetermined condition; storing the first data group of data in the non-volatile storage unit; Including, Data processing methods. [Explanation of symbols]

[0066] 10 Data Processing Device 11 Server equipment 100 Data Processing Programs 112 Container renewal decision program 120 Overlay Application Program 130A Data Acquisition Program 130B Data Update Program 132A Container Acquisition Program 132B Container startup program 140 Data Storage Program 142 Container Memory Program 150 Release Judgment Program 152 Data update stop program 160 Overlay Cancellation Program 170 Elapsed Time Judgment Program 173 Data Capacity Judgment Program 174 Creation time determination program 180 Data Persistence Program 182 Container Persistence Program 300 1st storage medium 301 Second storage medium 310 Volatile memory unit 320 Non-volatile memory unit 400 Processing Circuit 401 Control circuit 500 Timer 600 Data generation device 700 First communication module 701 Second communication module

Claims

1. The storage medium includes a volatile storage unit and a nonvolatile storage unit, The processing circuit designating the volatile storage unit as an upper layer and the non-volatile storage unit as a lower layer, and applying an overlay process to at least a part of the storage medium; storing data in the volatile storage unit; Cancel application of the overlay processing, Based on a predetermined condition, the data is classified into a first data group to be persisted and a second data group not to be persisted; storing the data of the first data group in the nonvolatile storage unit; Data processing program.

2. The predetermined condition is the elapsed time from the last update date and time of the data. The data processing program according to claim 1 .

3. The data in the first data group is a container image. The data processing program according to claim 1 .

4. The predetermined condition is a data volume of the data. The data processing program according to claim 1 .

5. The predetermined condition is a creation time of the data. The data processing program according to claim 1 .

6. a storage medium including a volatile storage unit and a nonvolatile storage unit; a processing circuit; Equipped with The processing circuitry designating the volatile storage unit as an upper layer and the non-volatile storage unit as a lower layer, and applying an overlay process to at least a portion of the storage medium; instructing the volatile storage unit to store data; Cancel application of the overlay processing; Based on a predetermined condition, the data is classified into a first data group that is a target for persistence and a second data group that is not a target for persistence; instructing the nonvolatile storage unit to store the data of the first data group; Data processing device.

7. The storage medium includes a volatile storage unit and a nonvolatile storage unit, designating the volatile storage unit as an upper layer and the non-volatile storage unit as a lower layer, and applying an overlay process to at least the storage medium; storing data in the volatile storage; canceling the application of the overlay processing; A step of distinguishing the data into a first data group that is a target for persistence and a second data group that is not a target for persistence based on a predetermined condition; storing the first data group of data in the non-volatile storage unit; Including, Data processing methods.

Images