Information processing device, information processing method, and program

Abstract

To realize a new technique for organizing files stored in a storage device.SOLUTION: An information processing device disclosed herein includes an extraction unit. The extraction unit extracts unnecessary files from among files stored in a storage device based on at least one of criteria related to file usage frequency, file data quality, and file redundancy.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A technology related to this disclosure is disclosed in Patent Document 1. Patent Document 1 discloses a technology for searching for and deleting similar files that are similar to an original file. This technology searches for similar files based on the file name, extension, file size, file contents, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-32617 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology disclosed in Patent Document 1, it is possible to search for and delete files similar to an original file. However, there are cases where simply deleting files similar to an original file is insufficient to organize files stored in a storage device.

[0005] One example of a purpose of this disclosure is to provide new techniques for organizing files stored in a storage device. [Means for solving the problem]

[0006] According to this disclosure: An information processing device is provided that has an extraction means for extracting unnecessary files from among files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy.

[0007] Also, according to this disclosure: One or more computers An information processing method is provided for extracting unnecessary files from files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy.

[0008] Also, according to this disclosure: Computer, A program is provided that functions as an extraction means for extracting unnecessary files from among files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy. [Effects of the Invention]

[0009] According to one aspect of the present disclosure, a new technique for organizing files stored on a storage device is provided. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional block diagram of an information processing device. [Figure 2] FIG. 2 is a flowchart showing an example of the flow of processing by the information processing device. [Figure 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of an information processing device. [Figure 4] FIG. 4 is a flowchart showing another example of the flow of processing by the information processing device. [Figure 5] FIG. 5 is a flowchart showing another example of the flow of processing by the information processing device. [Figure 6] FIG. 6 is an example of a functional block diagram of a system using an information processing device. [Figure 7] FIG. 7 is a flowchart showing another example of the flow of processing by the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate.

[0012] <<First embodiment>> Fig. 1 is a functional block diagram showing an overview of an information processing device 10. Fig. 2 is a flowchart showing an example of the flow of processing executed by the information processing device 10.

[0013] 1, the information processing device 10 includes an extraction unit 11. This functional unit executes the processing of the flowchart in FIG.

[0014] In S10, the extraction unit 11 extracts unnecessary files from among the files stored in the storage device based on at least one of criteria related to the frequency of file use, criteria related to file data quality, and criteria related to file redundancy.

[0015] In this way, the information processing device 10 can extract unnecessary files based on characteristic criteria, such as criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy. For example, the information processing device 10 can extract files with low usage frequency as unnecessary files. Furthermore, the information processing device 10 can extract files with low data quality as unnecessary files. Furthermore, the information processing device 10 can extract files that overlap with other files as unnecessary files. Such information processing device 10 realizes a new technology for organizing files stored in a storage device.

[0016] <<Second embodiment>> <Summary> The information processing device 10 of the second embodiment is a specific implementation of the configuration of the information processing device 10 of the first embodiment, which will be described in detail below.

[0017] <Hardware configuration> First, an example of the hardware configuration of the information processing device 10 will be described. Each functional unit of the information processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. The software includes programs that are pre-loaded when the device is shipped, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0018] FIG. 3 is a block diagram illustrating an example of the hardware configuration of an information processing device 10. As shown in FIG. 3, the information processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The information processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the information processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices can have the above hardware configuration.

[0019] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a central processing unit (CPU) or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, and touch panel. Examples of output devices include a display, projector, speaker, printer, and mailer. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0020] <Functional configuration> Next, a detailed description will be given of the functional configuration of the information processing device 10. Fig. 1 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an extraction unit 11. The functional unit will be described below.

[0021] The extraction unit 11 extracts unnecessary files from among the files stored in a storage device. Hereinafter, this storage device will be referred to as a "file storage device."

[0022] A file storage device stores a large number of files. As shown in the following examples, an example of a file storage device is a data lake that stores a large amount of various data (including structured data and unstructured data) acquired from various sources. Note that the file storage device may be a system other than a data lake.

[0023] The type of file stored in the file storage device is not particularly limited. As shown in the following embodiment, an example of a file is a CSV file. Note that the files stored in the file storage device may be files other than CSV files, or multiple types of files may be mixed.

[0024] The extraction unit 11 extracts unnecessary files from among the files stored in the file storage device based on at least one of the following criteria: "criteria related to file usage frequency," "criteria related to file data quality," and "criteria related to file redundancy." The process of extracting unnecessary files based on each criterion will be described below.

[0025] "Extracting unnecessary files based on file frequency criteria" The extraction unit 11 can extract, as unnecessary files, files that satisfy one of the following use frequency criteria 1 to 5, or files that satisfy a predetermined number of criteria.

[0026] (Usage Frequency Criteria 1) The time elapsed since the file creation date or last access date indicated in the file metadata is greater than or equal to the reference value. (Usage Frequency Criterion 2) The time elapsed since the last access date of the file, as shown in the file access log, is greater than or equal to the reference value. (Usage Frequency Criterion 3) The number of users granted access to the file is below the threshold. (Frequency of Use Criterion 4) Backup files exist (Usage Frequency Criterion 5) The file access frequency is below the standard value

[0027] First, (Use Frequency Criterion 1) will be explained. File metadata (sometimes called properties, etc.) is linked to the file and stored in a file storage device. The metadata indicates the file creation date and the file access date. The extraction unit 11 refers to the metadata linked to each file to identify the file creation date or the last access date of each file. Then, the extraction unit 11 determines whether the elapsed time from the file creation date or the last access date of each identified file is equal to or greater than a reference value. The reference value is a predetermined value.

[0028] A file that has been in use for a long time since its creation date is an old file. In a file storage device where new data is constantly being accumulated, the frequency of use of old files decreases. Also, a file that has been in use for a long time since its last access date is a file that has not been used often. The extraction unit 11 extracts such infrequently used files as unnecessary files.

[0029] Next, we will explain (Frequency of Use Criterion 2). A file access log records the date and time of access for each file. A file storage device may have the function of monitoring access to each file and recording the access log. Also, a device other than the file storage device may work in conjunction with the file storage device to monitor access to each file and record the access log.

[0030] The extraction unit 11 refers to the access log to identify the last access date of each file. Then, the extraction unit 11 determines whether the elapsed time since the last access date of each identified file is equal to or greater than a reference value. The reference value is a predetermined value.

[0031] A file with a large amount of time elapsed since its last access date is a file that is not used often, and the extraction unit 11 extracts such a file that is not used often as an unnecessary file.

[0032] Next, we will explain (Use Frequency Criterion 3). Users who have been granted access rights to a file are registered in the file's metadata or file management information. The file management information may be stored in the file storage device or in another device.

[0033] The extraction unit 11 refers to the metadata of each file or the management information of the file to identify the number of users who have been granted access rights to each file. Then, the extraction unit 11 determines whether the identified number is equal to or less than a reference value. The reference value is a predetermined value, such as 1.

[0034] A file to which only a small number of users have been granted access rights can be used by only a small number of users. Such files are used less frequently. The extraction unit 11 extracts such files that are used less frequently as unnecessary files.

[0035] Next, (Use Frequency Criterion 4) will be explained. The extraction unit 11 checks whether a backup file of each file is stored in a backup storage device different from the file storage device. Then, the extraction unit 11 extracts, as unnecessary files, files whose backup files are stored in the backup storage device.

[0036] If a file with a file name matching the file to be processed stored in the file storage device is stored in the backup storage device, the extraction unit 11 can determine that a backup file of the file to be processed exists.

[0037] If a backup file exists, data can be restored using the backup file even if the file stored in the file storage device is deleted. Furthermore, a file for which a backup file exists is likely to have been processed at least once by a user. It is presumed that the user determined the need for a backup file of the file based on the results of that processing and created the backup file. Such files that have been used once may become less frequently used. Therefore, the extraction unit 11 extracts files for which a backup file exists as unnecessary files.

[0038] Next, we will explain (Frequency of Use Criterion 5). The access frequency of a file is indicated by the number of accesses within a given period of time.

[0039] The extraction unit 11 refers to the access log and identifies the number of times each file has been accessed within a certain period of time. The extraction unit 11 then determines whether the identified number of times each file has been accessed (access frequency) is equal to or less than a reference value. The reference value is a predetermined value.

[0040] A file that has a low access frequency within a fixed period of time is a file that has not been used often, and the extraction unit 11 extracts such a file that has a low access frequency as an unnecessary file.

[0041] Extracting unwanted files based on file data quality criteria The extraction unit 11 can extract files that satisfy one or more of the following data quality criteria 1 and 2 as unnecessary files.

[0042] (Data Quality Criterion 1) The percentage of missing data values ​​is above the standard value. (Data Quality Criterion 2) The file's rating based on user feedback is below the standard value.

[0043] First, (Data Quality Standard 1) will be described. The proportion of missing values ​​in the data is calculated by dividing the number of missing values ​​in the data by a predetermined number. For example, the extraction unit 11 refers to the contents of the file and counts the predetermined missing values ​​(e.g., null values) for each file, thereby identifying the number of missing values ​​in the data. Furthermore, the extraction unit 11 identifies the predetermined number based on the contents of the file. The predetermined number is a value indicating the volume of the data. For example, the extraction unit 11 may refer to the contents of the file (e.g., a CSV file) and identify the number of rows and columns of the data. Then, the extraction unit 11 may identify the product of the number of rows and the number of columns as the predetermined number.

[0044] The extraction unit 11 calculates the "proportion of missing data values" by dividing the "number of missing data values" thus identified by the "predetermined number." The extraction unit 11 then determines whether the calculated proportion of missing data values ​​for each file is equal to or greater than a reference value. The reference value is a predetermined value.

[0045] A file with a proportion of missing data values ​​equal to or greater than a reference value is a low-quality file with a high proportion of missing data values, and the extraction unit 11 extracts such low-quality files as unnecessary files.

[0046] Next, (Data Quality Standard 2) will be described. In one example, a user who uses a file stored in a file storage device can register a rating score as feedback for the file they used. The rating score may be indicated, for example, on a five-point scale from 1 to 5, or may be indicated by other criteria. The file storage device, or another device linked to the file storage device, presents a UI (User Interface) screen for feedback to the user who used the file stored in the file storage device, and accepts the input and registration of various information. This UI screen includes a UI component that accepts the rating score for the used file. The file storage device, or another device linked to the file storage device, then stores the rating score registered by the user for each file. Note that when multiple users use the same file, multiple rating scores are registered in association with that single file.

[0047] The extraction unit 11 calculates statistical values ​​of the evaluation points registered by the user for each file. The statistical values ​​include, but are not limited to, the average value, maximum value, minimum value, mode value, median value, etc. The extraction unit 11 then determines whether the calculated evaluation points (statistical values) of each file are equal to or greater than a reference value. The reference value is a predetermined value.

[0048] A file with an evaluation score equal to or less than the reference value is a low-quality file that has been evaluated poorly by the user, and the extraction unit 11 extracts such low-quality files as unnecessary files.

[0049] "Extracting unwanted files based on file duplication criteria" The extraction unit 11 can extract files that satisfy the following redundancy criterion 1 as unnecessary files.

[0050] (Duplicity Criterion 1) There is another file whose contents match the file.

[0051] The extraction unit 11 can determine the identity of the file contents by using, for example, a checksum value. This process is suitable when the file to be processed is a CSV file. Note that the extraction unit 11 may also determine the identity of the file contents by using other well-known methods.

[0052] Then, the extraction unit 11 extracts, as unnecessary files, files for which other files with the same (that is, matching) content exist in the file storage device.

[0053] If there are multiple files with the same contents, only one needs to be kept and the others are unnecessary. Therefore, the extraction unit 11 searches for multiple files with the same contents and extracts the remaining files except for one of them as unnecessary files.

[0054] There are various methods for extracting unnecessary files from multiple files with matching contents, but one example is shown below.

[0055] For example, when multiple files with identical contents have different file names, the extraction unit 11 may extract the file with the older file creation date as an unnecessary file. That is, the extraction unit 11 may extract files other than the file with the most recent file creation date as unnecessary files.

[0056] When multiple files with identical file contents have different file names, it is likely that the user created a new file by copying and pasting a file with an older file creation date, and then gave the new file a name of their choice. In such a case, the file with the newer file creation date and the desired file name is naturally more important. Therefore, the extraction unit 11 extracts the file with the older file creation date as an unnecessary file.

[0057] On the other hand, when the file names of multiple files whose contents are identical match, the extraction unit 11 may extract the file with the newer file creation date as an unnecessary file. That is, the extraction unit 11 may extract the files other than the file with the oldest file creation date as unnecessary files.

[0058] When multiple files with identical file contents have the same filenames, there is no difference between the files. In such a case, the extraction unit 11 keeps the file with the older file creation date and extracts the file with the newer file creation date as an unnecessary file. By keeping the older file in this way, the number of files extracted as unnecessary in the "extraction of unnecessary files based on the criteria related to file usage frequency" described above increases. As a result, the efficiency of organizing the file storage device is improved.

[0059] In addition, the extraction unit 11 can extract files that satisfy one of the above-mentioned usage frequency criteria 1 to 5, data quality criteria 1 and 2, and redundancy criteria 1, or files that satisfy a specified number of criteria, as unnecessary files.

[0060] The extraction unit 11 can perform various processes on the extracted unnecessary files.

[0061] For example, the extraction unit 11 may delete the extracted unnecessary files from the file storage device. Also, as will be described in the third embodiment, the extraction unit 11 may move the extracted unnecessary files from the file storage device to another storage device. By deleting or moving the extracted unnecessary files, they no longer exist in the file storage device. As a result, the files stored in the file storage device are organized.

[0062] Alternatively, the extraction unit 11 may generate a list of the extracted unnecessary files and output the list to the user. The output is realized via an output device such as a display, a projector, or a printer. The user can refer to the files in the list one by one and delete them from the file storage device or move them to another storage device as necessary.

[0063] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of FIG.

[0064] The information processing device 10 extracts unnecessary files from among the files stored in the file storage device based on at least one of a criterion related to the frequency of file use, a criterion related to file data quality, and a criterion related to file redundancy (S10).

[0065] Although not shown in Figure 2, after S10, the information processing device 10 can delete the extracted unnecessary files from the file storage device, move the extracted unnecessary files to another storage device, or output a list of the extracted unnecessary files to the user.

[0066] The information processing device 10 can execute the extraction process of S10 at a predetermined timing.

[0067] For example, the information processing device 10 may execute the extraction process of S10 in response to receiving a process start instruction input from the user.

[0068] Alternatively, the information processing device 10 may execute the extraction process of S10 at predetermined time intervals. For example, the information processing device 10 may execute the extraction process of S10 daily, weekly, or monthly, or at other time intervals.

[0069] Alternatively, the information processing device 10 may execute the extraction process of S10 when the number of files stored in the file storage device exceeds a reference value.

[0070] <Action and effect> According to the information processing device 10 of the second embodiment, the same effects as those of the information processing device 10 of the first embodiment are achieved.

[0071] Furthermore, the information processing device 10 can extract unnecessary files based on the above-mentioned characteristic usage frequency criteria 1 to 5, data quality criteria 1 and 2, and redundancy criteria 1. By making a judgment based on the usage frequency of the file and the contents of the file (quality, redundancy), unnecessary files can be extracted with high accuracy.

[0072] <<Third embodiment>> An information processing device 10 of the third embodiment moves extracted unnecessary files from a file storage device to another storage device. After the move, the information processing device 10 deletes the unnecessary files from the other storage device when a deletion condition is met. Furthermore, if a user inputs an instruction before the deletion condition is met, the information processing device 10 returns the unnecessary files moved to the other storage device to the file storage device. This will be described in detail below.

[0073] The extraction unit 11 moves the extracted unnecessary files from the file storage device to another storage device. As a result of this movement, the extracted unnecessary files no longer exist in the file storage device.

[0074] In one example, the file storage device is a public repository. Users can access files in the file storage device via a predetermined API (Application Programming Interface). In this case, by moving extracted unnecessary files from the file storage device to another storage device, access to the unnecessary files via the predetermined API becomes impossible. By preventing access to unnecessary files, user convenience is improved.

[0075] The extraction unit 11 then deletes the unnecessary files that have been moved to another storage device from the other storage device when the deletion condition is satisfied. The deletion condition is, for example, "a predetermined time has elapsed since the file was moved to the other storage device," but is not limited to this.

[0076] If the user inputs an instruction before the deletion condition is met, the extraction unit 11 moves the unnecessary files that were moved to another storage device to the file storage device. This movement causes the unnecessary files to reside again in the file storage device. As a result, the unnecessary files can once again be accessed via the above-mentioned predetermined API.

[0077] In one example, a user can access unnecessary files in another storage device via an API for the other storage device that is different from the API for the file storage device described above. If necessary, the user can access unnecessary files in the other storage device via the API for the other storage device and find a necessary file. The user can then input an instruction to move the found file to the file storage device. In response to the instruction, the extraction unit 11 moves the file specified by the user input from the other storage device to the file storage device. As a result of this move, the file no longer exists in the other storage device.

[0078] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of FIG.

[0079] First, the information processing device 10 extracts unnecessary files from among the files stored in the file storage device based on at least one of criteria related to the frequency of file use, criteria related to file data quality, and criteria related to file redundancy (S20).

[0080] Next, the information processing device 10 moves the extracted unnecessary files from the file storage device to another storage device (S21).

[0081] The information processing device 10 can execute the processes of S20 and S21 at the predetermined timing described in the second embodiment.

[0082] Next, another example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of FIG.

[0083] The information processing device 10 determines whether any unnecessary files moved to another storage device include a file that satisfies the deletion condition (S30). If any file satisfies the deletion condition (Yes in S30), the information processing device 10 deletes the file from the other storage device (S33) and proceeds to S34.

[0084] If there is no file that satisfies the deletion condition (No in S30), the information processing device 10 proceeds to S31. Then, if the information processing device 10 receives an instruction input from the user to move the specified file to the file storage device (Yes in S31), the information processing device 10 moves the file specified by the instruction input to the file storage device (S32), and proceeds to S34.

[0085] If an instruction input to move the designated file to the file storage device has not been received from the user (No in S31), the information processing device 10 proceeds to S34.

[0086] If there is no instruction to end the process (No in S34), the information processing device 10 returns to S30 and repeats the process.

[0087] Other configurations of the information processing device 10 of the third embodiment are similar to those of the information processing device 10 of the first and second embodiments.

[0088] According to the information processing device 10 of the third embodiment, the same effects as those of the information processing device 10 of the first and second embodiments are realized.

[0089] Furthermore, the information processing device 10 does not immediately delete the extracted unnecessary files, but temporarily saves them in another storage device. Then, the information processing device 10 deletes the unnecessary files when the deletion conditions are met after the temporary saving. Furthermore, if a user instructs it before the deletion conditions are met, the unnecessary files can be returned to the file storage device.

[0090] According to the information processing device 10, unnecessary files can be extracted accurately by extracting unnecessary files according to the criteria described in the first and second embodiments. However, a file that was unnecessary at the time of extraction may later become a necessary file. If the extracted unnecessary file is immediately deleted, no action can be taken even if such a change in situation occurs.

[0091] Unlike the information processing device 10, extracted unnecessary files are not immediately deleted, but are temporarily saved in another storage device, so that when a change in the situation occurs as described above, the unnecessary files can be restored to the file storage device. With this information processing device 10, it is possible to prevent the inconvenience of deleting necessary files.

[0092] <<Example>> A description will be given of an embodiment of a data providing system including the information processing device 10. Note that the embodiment described here is merely an example, and the present invention is not limited to this.

[0093] 6 shows a functional block diagram of a data providing system including an information processing device 10. As shown in the figure, the data providing system includes the information processing device 10, a data lake 20, a data preprocessing system 30, an AP-side system 40, and an API 50.

[0094] The data lake 20 corresponds to the file storage device described in the above embodiment. The data lake 20 is a public repository. The data lake 20 stores a large amount of various data (including structured data and unstructured data) acquired from various sources. In the illustrated example, the data lake 20 stores CSV files.

[0095] The data preprocessing system 30 includes a file database, an ETL / Crawler, and a data catalog database.

[0096] The file database stores files that have undergone predetermined formatting for files stored in the data lake 20. The data catalog database stores related information (catalog information) for each file stored in the file database. The related information includes file characteristics, evaluation scores for the file based on user feedback, etc.

[0097] The ETL / Crawler reads files stored in the data lake 20, performs necessary formatting, and stores the files in the file database. The ETL / Crawler can also crawl the data catalog stored in the data catalog database.

[0098] The AP side system 40 includes a data analysis system and a data visualization system. A user can access a file database via the data analysis system. A user can also access a data catalog database via the data visualization system.

[0099] The configuration of the information processing device 10 is as described in the above embodiment. In the example of Fig. 6, the information processing device 10 has a sandbox for storing unnecessary files. The sandbox corresponds to the other storage device described in the above embodiment.

[0100] API50 provides access to unwanted files stored in the sandbox.

[0101] Next, an example of the flow of processing by the information processing device 10 in the embodiment will be described with reference to the flowchart of FIG.

[0102] First, the information processing device 10 reads a file from the data lake 20 (S400). Next, the information processing device 10 extracts attribute information from the read file (S401). The attribute information includes at least one of metadata for each file, an access log, content information for the file, sharing information for the file, and evaluation information for the file.

[0103] The metadata indicates, for example, the file name, the file extension, the last access date, and the like. The access log indicates, for example, the date and time of access. The file content information is information about the contents of the file, and indicates, for example, the number of rows, the number of columns, the percentage of missing values, the checksum value, and the like. The file sharing information indicates, for example, users who have been granted access rights to the file. The file rating information of a file indicates, for example, a rating score of the file based on feedback from users.

[0104] After S401, determinations in S402 to S404 are made based on criteria relating to the frequency of file use.

[0105] In S402, the information processing device 10 determines whether the last access date is X years or more. In S403, the information processing device 10 determines whether there are other co-owners (that is, whether there are a plurality of users who have been granted access authority). In S404, the information processing device 10 determines whether a backup file exists.

[0106] The information processing device 10 extracts the file for which the determination in all of S402 to S404 is Yes as an unnecessary file, and moves it to the sandbox (S411).

[0107] On the other hand, for a file determined as No in at least one of S402 to S404, the information processing apparatus 10 performs determinations in S405 and S406. The determinations in S405 and S406 are based on criteria related to the data quality of the file.

[0108] In S405, the information processing device 10 determines whether the rate of missing values ​​is Z % or more. In S406, the information processing device 10 determines whether the evaluation score based on the feedback from the user is equal to or less than a reference value.

[0109] The information processing device 10 extracts the file for which it has been determined as Yes in at least one of S405 and S406 as an unnecessary file, and moves it to the sandbox (S411).

[0110] On the other hand, for a file for which both S405 and S406 have been determined as No, the information processing apparatus 10 performs the determinations of S407 to S410. The determinations of S407 and S408 are based on criteria relating to file duplication.

[0111] In S407, the information processing device 10 determines whether there are multiple files with the same checksum value. In S408, the information processing device 10 determines whether or not there are multiple files with the same file name among multiple files with the same checksum value.

[0112] If there are multiple files with the same checksum value and file name (Yes in S407 and Yes in S408), the information processing device 10 extracts the file with the newer file creation date from among the multiple files as an unnecessary file. Then, the information processing device 10 moves the extracted unnecessary file to the sandbox (S411).

[0113] On the other hand, if there are multiple files with the same checksum value but different file names (Yes in S407 and No in S408), the information processing device 10 extracts the file with the oldest file creation date from among the multiple files as an unnecessary file.Then, the information processing device 10 moves the extracted unnecessary file to the sandbox (S411).

[0114] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0115] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of the content.

[0116] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. 1. An information processing device having an extraction means for extracting unnecessary files from among files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy. 2. The extraction means The time elapsed since the file creation date or the last access date indicated in the file metadata is greater than or equal to a threshold value. The time elapsed since the last access date of the file shown in the file access log is greater than or equal to the reference value. The number of users granted access to the file is below a threshold, A backup file exists, and The file access frequency is below the standard value, 2. The information processing device according to claim 1, wherein files that satisfy at least one of the above are extracted as unnecessary files. 3. The extraction means The proportion of missing data is equal to or greater than the standard value, and The file's rating based on user feedback is below a certain threshold, 3. The information processing device according to 1 or 2, wherein a file that satisfies at least one of the above conditions is extracted as an unnecessary file. 4. The extraction means 4. The information processing device according to any one of 1 to 3, wherein a file having contents matching other files is extracted as an unnecessary file. 5. The extraction means 5. The information processing device according to 4, wherein when there are multiple files whose contents are identical but whose file names are different, the file with the oldest file creation date is extracted as an unnecessary file. 6. The file to be processed is a CSV file. The extraction means 6. The information processing device according to 4 or 5, which determines the identity of the contents of a file by using a checksum value. 7. The extraction means 7. The information processing device according to any one of 1 to 6, wherein the extracted unnecessary files are moved from the storage device to another storage device. 8. The extraction means deleting the unnecessary file that has been moved to the other storage device from the other storage device when a deletion condition is satisfied; 8. The information processing device according to 7, wherein, if a user inputs an instruction before the deletion condition is satisfied, the unnecessary files that have been moved to the other storage device are moved to the storage device. 9. One or more computers: An information processing method for extracting unnecessary files from files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy. 10. Computer A program that functions as an extraction means for extracting unnecessary files from among files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy.

[0117] Some or all of Supplements 2 to 8 that are dependent on the information processing device of Supplement 1 described above may also be dependent on the information processing method of Supplement 9 and the program of Supplement 10 in the same dependent relationship as Supplement 1 and Supplements 2 to 8. Furthermore, within the scope of each of the above-mentioned embodiments, some or all of the configurations described as Supplements can be realized in various hardware, software, various recording means for recording software, or systems. [Explanation of symbols]

[0118] 10. Information processing equipment 11 Extraction part 20 Data Lake 30 Data Preprocessing System 40 AP side system 50 API50 1A processor 2A Memory 3A input / output I / F 4A peripheral circuit 5A Bus

Claims

1. An information processing device having an extraction means for extracting unnecessary files from files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy.

2. The extraction means The time elapsed since the file creation date or the last access date indicated in the file metadata is greater than or equal to a threshold value. The time elapsed since the last access date of the file shown in the file access log is greater than or equal to the reference value. The number of users granted access to the file is below a threshold, A backup file exists, and The file access frequency is below the standard value, 2. The information processing apparatus according to claim 1, wherein a file that satisfies at least one of the following conditions is extracted as an unnecessary file.

3. The extraction means The proportion of missing data is equal to or greater than the standard value, and The file's rating based on user feedback is below a certain threshold, 2. The information processing apparatus according to claim 1, wherein a file that satisfies at least one of the following conditions is extracted as an unnecessary file.

4. The extraction means 2. The information processing apparatus according to claim 1, wherein a file having the same contents as another file is extracted as an unnecessary file.

5. The extraction means 5. The information processing apparatus according to claim 4, wherein when there are a plurality of files whose contents are identical but whose file names are different, the file with the oldest file creation date is extracted as an unnecessary file.

6. The file to be processed is a CSV file. The extraction means 5. The information processing apparatus according to claim 4, wherein the checksum value is used to determine whether the contents of the files are identical.

7. The extraction means 2. The information processing apparatus according to claim 1, wherein the extracted unnecessary files are moved from the storage device to another storage device.

8. The extraction means deleting the unnecessary file that has been moved to the other storage device from the other storage device when a deletion condition is satisfied; 8. The information processing apparatus according to claim 7, wherein, when a user inputs an instruction before the deletion condition is satisfied, the unnecessary files that have been moved to the other storage device are moved to the storage device.

9. One or more computers An information processing method for extracting unnecessary files from files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy.

10. Computer, A program that functions as an extraction means for extracting unnecessary files from files stored in a storage device based on at least one of criteria related to file usage frequency, criteria related to file data quality, and criteria related to file redundancy.

Images