File management device, file management method, and file management program

Abstract

A server (20) comprises an environment construction unit (22) that constructs a virtual tenant environment (23) from a use image read from a file storage unit (21), a pipeline processing unit (24) that executes a pipeline file on the virtual tenant environment (23), an image extraction unit (44) that extracts a use image satisfying a prescribed condition in the file storage unit (21), and an image deletion unit (61) that deletes the use image in the file storage unit (21) extracted by the image extraction unit (44).

Description

File management device, file management method, and file management program

[0001] The present invention relates to a file management device, a file management method, and a file management program.

[0002] In the software development process, the code written by the programmer is built to generate an executable application. The application is then run and tested to verify whether the output is correct, thereby identifying any code modifications that need to be made. Applications that pass the tests are released into production.

[0003] Such software development processes are efficiently automated using a CI (Continuous Integration) / CD (Continuous Delivery) pipeline. A CI / CD pipeline enables flexible response and rapid delivery of software development. Patent Document 1 describes GitHub Actions provided by GitHub (registered trademark) as a CI / CD pipeline. Patent Document 2 describes the use of containers as an execution environment for executing GitHub Actions to execute jobs.

[0004] GitHub Docs, "Understanding GitHub Actions," [online], [Retrieved June 23, 2023], Internet <URL: https: / / docs.github.com / ja / actions / learn-github-actions / understanding-github-actions> GitHub Docs, "Running jobs in a container," [online], [Retrieved June 23, 2023], Internet <URL: https: / / docs.github.com / ja / actions / using-jobs / running-jobs-in-a-container>

[0005] To prevent damage to computers due to defects in applications under development, isolated execution environments for trial and error, such as the containers and virtual machines (VMs) described in Patent Literature 2, are often used. Since the execution environment files for these tests are constructed after loading a large number of files onto the computer, preparation takes time. Therefore, we are considering shortening the preparation time for CI / CD pipeline execution by storing execution environment files constructed in the past without deleting them and reusing them for the next test.

[0006] However, since the storage space in the computer where the execution environment files are loaded is limited, it is necessary to delete stored execution environment files according to the remaining capacity. However, if execution environment files that are likely to be reused are also deleted, it will not be possible to shorten the preparation time for CI / CD pipeline execution.

[0007] Therefore, a main object of the present invention is to efficiently free up storage space for execution environment files while increasing the reusability of the execution environment files.

[0008] In order to solve the above problems, the file management device of the present invention has the following features: An environment construction unit that constructs an execution environment from an execution environment file read from a file storage unit, a processing unit that executes the execution file in the execution environment, an extraction unit that extracts the execution environment file from the file storage unit that satisfies predetermined conditions, and a deletion unit that deletes the execution environment file from the file storage unit that has been extracted by the extraction unit.

[0009] According to the present invention, it is possible to efficiently free up storage space for execution environment files while increasing the reusability of the execution environment files.

[0010] FIG. 1 is a configuration diagram of an automation system according to the present embodiment. FIG. 2 is a hardware configuration diagram of each device constituting the automation system according to the present embodiment. FIG. 3 is a configuration diagram showing details of a server according to the present embodiment. FIG. 4 is a flowchart showing pipeline execution processing according to the present embodiment. FIG. 5 is a table showing an example of a management book stored in a management book storage unit according to the present embodiment. FIG. 6 is a flowchart showing details of processing to extract images to be deleted in FIG. 6 according to the present embodiment.

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0012] FIG. 1 is a configuration diagram of an automation system 100. The automation system 100 is configured by connecting a client 10 and a server (file management device) 20 via a network. The client 10 includes a pipeline description unit 11, a pipeline storage unit 12, and an execution instruction unit 13. The pipeline description unit 11 is a processing unit for describing pipeline processing such as a CI / CD pipeline in response to input from a user, and is, for example, a tool such as workflow. The pipeline storage unit 12 is a memory unit for storing executable files for the pipeline processing described by the pipeline description unit 11, and is, for example, network storage such as GitHub or GitHub Lab. The execution instruction unit 13 is a processing unit for reading the executable files for the pipeline processing stored in the pipeline storage unit 12 and instructing the server 20 to execute them, and is, for example, a tool such as GitHub Actions.

[0013] The server 20 includes a file storage unit 21, an environment construction unit 22, a virtual tenant environment (execution environment) 23, and a pipeline processing unit (processing unit) 24. The file storage unit 21 stores the following files as non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD): The usage image is an execution environment file such as a VM or container used by the pipeline processing unit 24, and is read by the environment construction unit 22 to construct the virtual tenant environment 23. The generated image is an execution environment file generated as a result of execution of a build process by the pipeline processing unit 24. The test result 64 (see FIG. 3 described below) is a file generated as a result of execution by the pipeline processing unit 24, for example, an output file of the test result of pytest.

[0014] The environment construction unit 22 is also called a runner because it is a tool that prepares to run pipeline processing. The environment construction unit 22 reads the image to be used from the file storage unit 21 and deploys it on volatile memory (RAM: Random Access Memory), thereby constructing the virtual tenant environment 23. If the corresponding image to be used does not exist in the file storage unit 21, the environment construction unit 22 downloads the corresponding image to be used from an image registry (not shown) to the file storage unit 21. In the case of Docker, for example, the image registry is Docker Hub or Harbor.

[0015] Hereinafter, the process in which the environment construction unit 22 constructs the virtual tenant environment 23 based on the usage image will be referred to as "using the usage image." The pipeline processing unit 24 executes the executable file for the pipeline processing instructed by the execution instruction unit 13 on the virtual tenant environment 23 constructed by the environment construction unit 22. In other words, the pipeline processing unit 24 is executed after the usage image is used. As a result of execution by the pipeline processing unit 24, a generated image which is the execution result of the build process on the code, test results 64 ( FIG. 3 ) which indicate the operation history of the application, etc., are stored in the file storage unit 21.

[0016] FIG. 2 is a hardware configuration diagram of each device constituting the automation system 100. Each device (client 10, server 20) of the automation system 100 is configured as a computer 900 having a CPU 901, RAM 902, ROM 903, HDD 904, communication I / F 905, input / output I / F 906, and media I / F 907. The communication I / F 905 is connected to an external communication device 915. The input / output I / F 906 is connected to an input / output device 916. The media I / F 907 reads and writes data from a recording medium 917. Furthermore, the CPU 901 controls each unit by executing a program (file management program) loaded into RAM 902. This program (also called an application, or an app for short) can be distributed via a communication line or recorded on a recording medium 917 such as a CD-ROM.

[0017] FIG. 3 is a configuration diagram showing the details of the server 20. In FIG. 3, the details of the file storage unit 21 and the pipeline processing unit 24 of the server 20 are particularly described. The file storage unit 21 stores an image deletion unit (deletion unit) 61, an image storage unit 62, a management book storage unit 63, and test results 64 (described in FIG. 1). The image storage unit 62 stores the use image and the generated image described in FIG. 1. The image deletion unit 61 frees up storage space for the execution environment file by deleting the specified image from the image storage unit 62. The management book storage unit 63 stores a management book that describes descriptive information of the use image stored in the image storage unit 62 (details are shown in FIG. 5).

[0018] The pipeline processing unit 24 has a pipeline execution unit 31, a pipeline processing addition unit 32, a usage image grasping unit 41, an image comparison unit 42, an image list acquisition unit 43, an image extraction unit (extraction unit) 44, an image deletion instruction unit 45, a management book entry instruction unit 51, a management book entry unit 52, a management book acquisition unit 53, and a management book creation unit 54.

[0019] The server 20 efficiently frees up storage space for execution environment files while increasing the reusability of the execution environment files, mainly by using the following components: An environment construction unit 22 that constructs a virtual tenant environment 23 from a usage image read from the file storage unit 21. A pipeline processing unit 24 that executes an executable file (hereinafter referred to as a "pipeline file") such as a workflow that describes the processing content of a pipeline to be executed on the virtual tenant environment 23. An image extraction unit 44 that extracts usage images that satisfy predetermined conditions from the file storage unit 21. An image deletion unit 61 that deletes usage images from the file storage unit 21 extracted by the image extraction unit 44. Each component of the server 20 in Figure 3 will be clarified below with reference to Figure 4 and subsequent figures.

[0020] 4 is a flowchart showing the pipeline execution process. The management book creation unit 54 creates a management book for managing the images used in the image storage unit 62 (see FIG. 5 for details) and stores the management book in the management book storage unit 63 (S11). The pipeline description unit 11 stores the described pipeline file in the pipeline storage unit 12 (S12). The pipeline process addition unit 32 then adds a management book append process to the pipeline file stored in S12 (S13). The management book append process is a process of adding, to the management book, descriptive information about the images used in the pipeline processes executed from the pipeline file.

[0021] The execution instruction unit 13 instructs the pipeline processing unit 24 to execute the pipeline file of S13 (S14). The pipeline processing unit 24 extracts the use image required to execute the pipeline file from the image storage unit 62 to the environment construction unit 22, and then executes the pipeline file instructed in S14 on the environment construction unit 22 (S15). The use image understanding unit 41 understands the use image used by the environment construction unit 22 in conjunction with the execution process of S15. Note that the image storage unit 62 may store generated images that were generated in the past. In this case, the pipeline processing unit 24 may use the stored generated image as the current use image.

[0022] The management book entry instruction unit 51 instructs the management book entry unit 52 to add the explanatory information specified in S13 to the management book by executing the management book addition process added to the pipeline file in S13. The management book entry unit 52 follows the instruction and adds the explanatory information of the usage image to the management book in the management book storage unit 63 (S16).

[0023] 5 is a table showing an example of a management book stored in the management book storage unit 63. The management book stores, as explanatory information for the used image in the image storage unit 62, an image ID and management information indicating the usage status of the used image (number of uses and most recent use date and time) in association with each other. The record in the first row indicates that the used image with image ID "V1" in the image storage unit 62 was used on the date and time "2023 / 6 / 1" as the first use result. The record in the second row indicates that the used image with image ID "V2" in the image storage unit 62 was used on the date and time "2022 / 3 / 1" as the fifth use result.

[0024] When a used image is deleted from the image storage unit 62, the management book entry unit 52 also deletes the corresponding record in the management book. For example, consider a case where a pipeline file uses a used image with image ID "V1" at 8:00 AM, 9:00 AM, and 10:00 AM on the same day, and then deletes the used image with image ID "V1" from the image storage unit 62 at 9:30 AM. The record in the management book for image ID "V1" at each time changes over time as follows: [7:00 AM] = No record for image ID "V1" exists in the management book. At this point, no used image with image ID "V1" exists in the image storage unit 62. [8:00 AM] = A new record is created with image ID "V1," the number of uses = 1, and the most recent use date and time = 8:00 AM. The used image with image ID "V1" is loaded into the image storage unit 62 from an external device. [9:00 AM] = The existing record for image ID "V1" is updated to the number of uses = 2 and the most recent use date and time = 9:00 AM. [9:30 AM] = The used image with image ID "V1" is deleted from the image storage unit 62, and the record with image ID "V1" is also deleted from the management book. [10:00 AM] = A new record is created with image ID "V1", number of uses = 1, and most recent use date and time = 10:00 AM. Again, the used image with image ID "V1" is read from the external device into the image storage unit 62.

[0025] FIG. 6 is a flowchart showing the image deletion process. This flowchart is initiated, for example, when the remaining capacity in the image storage unit 62 falls below a predetermined value (e.g., 10% or less). The image list acquisition unit 43 acquires images (used images and generated images) stored in the image storage unit 62 (S21). The image extraction unit 44 extracts images to be deleted from the images acquired in S21 and notifies the image deletion instruction unit 45 of the extracted images (S22; see FIG. 7 for details). The image deletion instruction unit 45 then notifies the image deletion unit 61 of an instruction to delete the notified image (S23). The image deletion unit 61 deletes the image notified in S23 from the image storage unit 62 (S24). The management book entry unit 52 also deletes the description information of the image deleted in S24 from the management book in the management book storage unit 63.

[0026] Fig. 7 is a flowchart showing details of the process (S22) of extracting images to be deleted in Fig. 6. This flowchart explains an example of executing one of the following four methods. The method to be adopted is set in advance by the user in the image extraction unit 44. [Method 1] = All images (used images, generated images) are to be deleted. [Method 2] = Generated images are to be deleted. [Method 3] = Used images that have been used infrequently are to be deleted. [Method 4] = Used images with the oldest date and time of use are to be deleted.

[0027] First, when Method 1 is adopted (Yes in S201), the image extraction unit 44 selects all images (used images and generated images) acquired in S21 as deletion targets (S211). If No in S201, the management book acquisition unit 53 acquires description information of the used images from the management book storage unit 63 (S202).

[0028] When Method 2 is adopted (Yes in S203), the image extraction unit 44 designates the generated image as a deletion target (S212). Therefore, the image comparison unit 42 compares all images (usage image, generated image) acquired in S21 with the use image acquired in S202. Then, the image comparison unit 42 identifies, among the images stored in the image storage unit 62, images that are not registered in the management book storage unit 63 as generated images to be deleted. To this end, the pipeline processing unit 24 stores, in the file storage unit 21, a newly generated generated image that is generated by executing a build process in the pipeline file. Then, the image extraction unit 44 extracts the generated image generated by the pipeline processing unit 24 as a use image that satisfies predetermined conditions.

[0029] If the answer is No in S203 and Method 3 is adopted (Yes in S204), the image extraction unit 44 targets images to be deleted that have been used fewer than a predetermined number of times (e.g., three times) (S213). Therefore, the management book acquisition unit 53 acquires the number of times each image has been used from the management information in the management book shown in FIG. 5 . As a result, the pipeline processing unit 24 records the number of times each image has been used as a usage history of the image used when executing the pipeline file. The image extraction unit 44 then extracts images that have been used fewer than the predetermined number of times as images that satisfy predetermined conditions.

[0030] If the answer is No in S204 and Method 4 is adopted (Yes in S205), the image extraction unit 44 targets images for deletion whose most recent use date and time is older than a predetermined date and time (e.g., three months ago) (S214). Therefore, the management book acquisition unit 53 acquires the most recent use date and time for each image for use from the management information in the management book shown in FIG. 5 . As a result, the pipeline processing unit 24 records the most recent use date and time for each image for use as the usage history of the image used when executing the pipeline file. The image extraction unit 44 then extracts images whose most recent use date and time is older than the predetermined date and time as images for use that satisfy the predetermined conditions.

[0031] As described above with reference to FIG. 7 , the data referenced by the image extraction unit 44 to determine the image to be deleted differs depending on the method 1 to method 4. FIG. 3 illustrates a configuration of the server 20 capable of executing any of methods 1 to 4 (not only a single method but also a combination of multiple methods is possible). Meanwhile, as shown below, the components can be simplified by configuring the server 20 to execute only individual methods. For [Method 1], the management book is not necessary at all. Therefore, the pipeline processing addition unit 32, the management book entry unit 52, the management book acquisition unit 53, the management book creation unit 54, the management book storage unit 63, the used image grasping unit 41, the image comparison unit 42, and the management book entry instruction unit 51 can be omitted from the server 20.

[0032] In [Method 2], in the management book appending process (S13, S16) and the management book reading process (S202), the image ID of the image to be used is required, but reading and writing of management information (number of uses, latest use date and time) is not required.

[0033] In [Method 3], in the management book appending process (S13, S16) and the management book reading process (S202), the image ID and number of uses of the used image are necessary, but reading and writing of the most recent used date and time data is unnecessary. Also, because generated images are not subject to deletion, the image comparison unit 42 can be omitted from the server 20. Furthermore, the process in S21 in which the image list acquisition unit 43 acquires images stored in the image storage unit 62 can be substituted by the process in S202 in which the management book acquisition unit 53 acquires description information of the used image from the management book, so in that case the image list acquisition unit 43 can also be omitted from the server 20.

[0034] Regarding [Method 4], in the management book appending process (S13, S16) and the management book reading process (S202), the image ID of the image to be used and the most recent date and time of use are necessary, but reading and writing data on the number of uses is not necessary. Also, because generated images are not subject to deletion, the image comparison unit 42 can be omitted from the server 20. Furthermore, the process in S21 in which the image list acquisition unit 43 acquires images stored in the image storage unit 62 can be substituted by the process in S202 in which the management book acquisition unit 53 acquires description information of the image to be used from the management book, so in that case the image list acquisition unit 43 can also be omitted from the server 20.

[0035] In the automation system 100 described above, an image file stored in the image storage unit 62 has been exemplified as an execution environment file for constructing the virtual tenant environment 23. However, other data structures, such as a disk volume (area), may also be used as the execution environment file. Furthermore, with regard to the process (S13) in which the pipeline processing addition unit 32 adds a management book appending process to the pipeline file, the process of S13 may be executed from the pipeline processing addition unit 32 present inside the pipeline file to be added.

[0036] [Effect] The server 20 of the present invention is characterized by having an environment construction unit 22 that constructs a virtual tenant environment 23 from a usage image read from a file storage unit 21, a pipeline processing unit 24 that executes a pipeline file on the virtual tenant environment 23, an image extraction unit 44 that extracts usage images that satisfy predetermined conditions in the file storage unit 21, and an image deletion unit 61 that deletes the usage images in the file storage unit 21 extracted by the image extraction unit 44.

[0037] As a result, the server 20 can improve the reusability of the execution environment file and shorten the execution time by using the use image read from the file storage unit 21. Furthermore, the server 20 can efficiently free up storage space for the execution environment file by deleting the use image extracted by the image extraction unit 44.

[0038] The present invention is characterized in that the pipeline processing unit 24 stores a newly generated generated image in the file storage unit 21 by executing the build process in the pipeline file, and the image extraction unit 44 extracts the generated image generated by the pipeline processing unit 24 as a usable image that satisfies predetermined conditions.

[0039] This allows the server 20 to recognize the generated image that it has generated itself in the build process as an image that may be deleted.

[0040] The present invention is characterized in that the pipeline processing unit 24 records the number of times each use image is used as a usage history of the use image used when executing the pipeline file, and the image extraction unit 44 extracts use images whose number of uses is less than a predetermined number as use images that satisfy predetermined conditions.

[0041] This allows the server 20 to recognize images that are rarely used as images that may be deleted.

[0042] The present invention is characterized in that the pipeline processing unit 24 records the most recent use date and time for each use image as a usage history of the use image used when executing the pipeline file, and the image extraction unit 44 extracts use images whose most recent use date and time is older than a predetermined date and time as use images that satisfy predetermined conditions.

[0043] This allows the server 20 to recognize images that have not been used recently as images that may be deleted.

[0044] 10 Client 11 Pipeline description unit 12 Pipeline storage unit 13 Execution instruction unit 20 Server (file management device) 21 File storage unit 22 Environment construction unit 23 Virtual tenant environment (execution environment) 24 Pipeline processing unit (processing unit) 31 Pipeline execution unit 32 Pipeline processing addition unit 41 Usage image understanding unit 42 Image comparison unit 43 Image list acquisition unit 44 Image extraction unit (extraction unit) 45 Image deletion instruction unit 51 Management book entry instruction unit 52 Management book entry unit 53 Management book acquisition unit 54 Management book creation unit 61 Image deletion unit (deletion unit) 62 Image storage unit 63 Management book storage unit 64 Test result 100 Automation system

Claims

1. A file management device comprising: an environment construction unit that constructs an execution environment from an execution environment file read from a file storage unit; a processing unit that executes the execution file on the execution environment; an extraction unit that extracts the execution environment file from the file storage unit that satisfies specified conditions; and a deletion unit that deletes the execution environment file from the file storage unit extracted by the extraction unit.

2. The file management device of claim 1, wherein the processing unit stores the newly generated execution environment file in the file storage unit by executing a build process within the executable file, and the extraction unit extracts the execution environment file generated by the processing unit as the execution environment file that satisfies the specified conditions.

3. The file management device of claim 1, wherein the processing unit records the number of times each execution environment file is used as a usage history of the execution environment file used when executing the executable file, and the extraction unit extracts the execution environment file whose number of times it has been used is less than a predetermined number as the execution environment file that satisfies the predetermined condition.

4. The file management device of claim 1, wherein the processing unit records the most recent usage date and time for each execution environment file as a usage history of the execution environment file used when executing the executable file, and the extraction unit extracts the execution environment file whose most recent usage date and time is older than a specified date and time as the execution environment file that satisfies the specified condition.

5. A file management method comprising: a file management device having an environment construction unit, a processing unit, an extraction unit, and a deletion unit; the environment construction unit constructs an execution environment from an execution environment file read from a file storage unit; the processing unit executes the execution file on the execution environment; the extraction unit extracts the execution environment file in the file storage unit that satisfies predetermined conditions; and the deletion unit deletes the execution environment file in the file storage unit extracted by the extraction unit.

6. A file management program for causing a computer to function as the file management device according to any one of claims 1 to 4.

Images