Information processing device, information processing method and program

Abstract

To enable data to be retrieved at a high speed by suppressing a capacity of a storage area necessary for retrieval.SOLUTION: An information processing device includes holding means for holding a first trie structure generated on the basis of a uniquely identifiable retrieval key, and retrieval means for retrieving an attribute value corresponding to the retrieval key from the retrieval key with reference to the first trie structure held by the holding means. In the first trie structure, the attribute value is associated with each node, and the retrieval means performs retrieval with respect to the first trie structure on the basis of the retrieval key, and makes the attribute value associated with a node that has reached deepest a retrieval result.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is a high demand for technology to search data quickly in systems that retrieve data from data names, such as image databases, news article sites, and data management systems for machine learning. From an economic perspective, these systems are required to store data efficiently in data storage. On the other hand, from the perspective of operability, there is also a demand to semantically divide data by data name and store it in data storage.

[0003] Figure 20 shows an example of a typical storage state of image data for machine learning. Here, the image data is assumed to be stored in PNG file format in a file system with a hierarchical tree structure. Also, the storage locations " / Volume1 / " and " / Volume2 / " are assumed to be physically different volumes. Figure 20 shows that an image file with an identification name in the first column is stored in the volume of the corresponding storage location in the second column.

[0004] In this example, image files with identifier name prefixes "Photo_Img1_001" through "Photo_Img1_580" are stored in " / Volume1 / ". Image files with identifier name prefixes "Photo_Img1_583" through "Photo_Img1_999" or "Photo_Img2" are stored in " / Volume2 / ". Additionally, the storage destinations of two image files with identifier name prefixes "Photo_Img1_581" and "Photo_Img1_582" are irregular and are stored in different locations from the files before and after them.

[0005] In cases where such identifiers often have a common prefix but slightly different suffixes, a Trie suitable for common prefix searches is known as a database technology for quickly searching storage locations from identifiers. To search for storage locations from identifiers, a Key-Value type database or a database to which Trie technology is applied may be used. However, both of these methods have the problem that the capacity of primary storage (memory, etc.) and secondary storage (hard disk, etc.) required for searches is large. Patent Document 1 proposes a technology that uses Tries to achieve high-speed searches while reducing the required capacity of primary storage (memory). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2008-134688 A Summary of the Invention [Problem to be solved by the invention]

[0007] The technology described in Patent Document 1 has the problem that while frequently searched identifiers can be searched quickly, it takes a long time to search for identifiers that are rarely searched. In addition, although it is possible to reduce the amount of primary storage (memory) required for a search, the amount is stored in secondary storage (hard disk, etc.), so there is a problem that the total amount of storage area required for a search is large. The present invention has been made in view of such circumstances, and has an object to reduce the amount of storage area required for a search and enable high-speed data search. [Means for solving the problem]

[0008] The information processing device of the present invention comprises a storage means for storing a first trie structure generated based on a uniquely identifiable search key, and a search means for searching the first trie structure stored in the storage means for an attribute value corresponding to the search key from the search key, wherein the attribute value is associated with each node in the first trie structure, and the search means performs a search in the first trie structure based on the search key, and the attribute value associated with the deepest reached node is set as a search result. Effect of the Invention

[0009] According to the present invention, it is possible to reduce the amount of storage area required for a search and to search for data at high speed. [Brief description of the drawings]

[0010] [Figure 1] 11 is a diagram illustrating an example of a correspondence relationship between a search key and an attribute value. FIG. [Diagram 2] FIG. 2 is a diagram illustrating a trie structure for performing the search in the example shown in FIG. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device according to the first embodiment. [Diagram 5] FIG. 2 is a diagram illustrating a trie structure according to the first embodiment. [Figure 6] FIG. 11 is a diagram illustrating an example of a functional configuration of an information processing device according to a second embodiment. [Figure 7] FIG. 11 is a diagram illustrating a trie structure according to the second embodiment. [Figure 8] FIG. 11 is a diagram illustrating the flow of a registration process in the second embodiment. [Figure 9] FIG. 11 is a diagram illustrating a trie structure according to the second embodiment. [Figure 10] FIG. 13 is a diagram illustrating a trie structure according to the third embodiment. [Figure 11]FIG. 13 is a diagram illustrating the flow of a registration process in the third embodiment. [Figure 12] FIG. 13 is a diagram illustrating an information processing device according to a fourth embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a functional configuration of an information processing device according to a fifth embodiment. [Figure 14] FIG. 13 is a diagram illustrating the flow of registration processing and search processing in the fifth embodiment. [Figure 15] FIG. 23 is a diagram illustrating the flow of a registration process in the sixth embodiment. [Figure 16] FIG. 23 is a diagram illustrating the flow of a registration process in the sixth embodiment. [Figure 17] FIG. 23 is a diagram illustrating the flow of a decision process in the seventh embodiment. [Figure 18] FIG. 23 is a diagram illustrating an example of a correspondence relationship between a search key and a plurality of attribute values ​​in the eighth embodiment. [Figure 19] FIG. 23 is a diagram illustrating a trie structure according to the eighth embodiment. [Figure 20] FIG. 13 is a diagram showing an example of the storage state of image data for machine learning. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] First, to clarify the difference from a conventional general trie, we will explain a search in a general trie. Note that a trie is sometimes written as a trie structure or a trie tree, but they all have the same meaning.

[0013] FIG. 1 is a diagram showing an example of the correspondence between search keys and their corresponding attribute values. Here, a file name is used as the search key, and a folder path related to the storage location (folder) of the file is used as the attribute value. Any value can be used for the search key and attribute value, but the search key must be a uniquely identifiable key (name) and a character string of concatenated words. A word is typically a single character or a reserved, fixed-form character string. Trie performs high-speed search processing by performing state transitions for each word, using words as input elements. The detailed operation is explained below.

[0014] FIG. 2(a) is a diagram showing a trie structure constructed to perform the search of the example shown in FIG. 1. In the diagram (state transition diagram) showing the trie structure, the circles in the diagram are nodes that represent the state of the search process, and the arrows from node to node represent state transitions. The letters attached to the arrows are transition words that cause the state transition. This is also true for the subsequent diagrams showing trie structures. A trie is an automaton that performs state transitions based on the current node and the input word.

[0015] For example, when searching for an attribute value corresponding to the search key "C000.png", first divide this search key into words. Here, 1 word = 1 character. The search key "C000.png" is divided into 9 words, starting from the beginning: "C", "0", "0", "0", ".", "p", "n", "g", and "#". Here, the last "#" is a special word (terminal word) that indicates the end of the search key.

[0016] The search starts from the root node 100. First, in node 100, the first word matches the transition word "C", so the state transitions from root node 100 to node 101. Next, in node 101, the second word matches the transition word "0", so the state transitions from node 101 to node 102. Similarly, the third to eighth words each cause a state transition word by word, to node 103, node 104, node 105, node 106, node 107, and node 108. Then, in node 108, the terminal word "#" of the ninth word causes a state transition from node 108 to node 109.

[0017] A node that has undergone a state transition due to a terminal word is called a terminal node, and an attribute value is associated with a terminal node. In the following, "being associated" with an attribute value may also be expressed as "having" the attribute value. In this example, node 109 is a terminal node, and has " / mnt / vol0 / 00 / " as the attribute value. A search is performed in this manner, and the attribute value " / mnt / vol0 / 00 / " corresponding to the search key "C000.png" is obtained as the search result.

[0018] Note that nodes having attribute values ​​are shown with different hatching patterns for each attribute value (the same applies below). An example of the correspondence between patterns and attribute values ​​is shown in Fig. 2(b). For example, node 109 with horizontal stripes indicates that it has an attribute value of " / mnt / vol0 / 00 / ".

[0019] Similarly, when searching for an attribute value corresponding to the search key "C001.png", the search key "C001.png" is divided into the words "C", "0", "0", "1", ".", "p", "n", "g", and "#" in order from the beginning. The first to third words cause a state transition for each word, from node 100 to node 101, node 102, and node 103. Up to this point, the state transition is the same as that for the search key "C000.png", but the fourth word "1" causes a state transition from node 103 to node 130. Then, the fifth to ninth words cause a state transition for each word, from node 131, node 132, node 133, and node 134, and finally the state transitions to the terminal node 135 having " / mnt / vol0 / 00 / " as the attribute value. In this way, the attribute value " / mnt / vol0 / 00 / " corresponding to the search key "C001.png" is obtained as a search result.

[0020] Similarly, in a search for the search key "C100.png", the first word "C" causes a state transition from node 100 to node 101. After that, the second word "1" causes a state transition from node 101 to node 110, and the third word "0" causes a state transition from node 110 to node 111. Then, the fourth to ninth words cause a state transition for each word, from node 112, node 113, node 114, node 115, and node 116, and finally the state transitions to the terminal node 117 having the attribute value " / mnt / vol0 / 01 / ". In this way, the attribute value " / mnt / vol0 / 01 / " corresponding to the search key "C100.png" is obtained as the search result.

[0021] Furthermore, in a search for the search key "C120.png", the first and second words cause a state transition from node 100, to node 101, to node 110 for each word. After that, the third word "2" causes a state transition from node 110 to node 120. Then, the fourth to ninth words cause a state transition from node 121, to node 122, to node 123, to node 124, to node 125 for each word, and finally the state transitions to the terminal node 126 having the attribute value " / mnt / vol1 / 00 / ". In this way, the attribute value " / mnt / vol1 / 00 / " corresponding to the search key "C120.png" is obtained as the search result.

[0022] The above is an overview of the operation of a search using try, and a search using try has the following features. Feature 1: The number of state transitions per search is the number of words (i.e. the number of characters in the search key + 1) Feature 2: Searching for a common prefix causes state transitions in the same node Feature 1 indicates that searches using a trie are fast, and that no matter how many search keys can be searched using a trie, search results can be obtained by simply making state transitions for the number of words.

[0023] Feature 2 indicates that, for example, for the search keys "C000.png" and "C001.png", the same state transition (node ​​100, node 101, node 102, node 103 in the example shown in FIG. 2) is performed up to the common prefix "C00". Therefore, when the search keys have a common prefix, one node is used to search for multiple search keys, so the storage area (primary storage, secondary storage) required for the search can be reduced. However, while feature 2 can reduce the storage area required for the search, one node is required per word to search for words following the common prefix. In the trie structure illustrated in FIG. 2(a), a total of 31 nodes are created.

[0024] (First embodiment) 3 is a block diagram showing an example of a hardware configuration of the information processing device 30 in this embodiment. The information processing device 30 has a CPU 31, a ROM 32, a RAM 33, a storage device 34, an operation unit 35, a display unit 36, a communication unit 37, and a system bus 38. The CPU 31, the ROM 32, the RAM 33, the storage device 34, the operation unit 35, the display unit 36, and the communication unit 37 are communicatively connected via the system bus 38. Note that the information processing device 30 in this embodiment may further have other configurations.

[0025] The CPU (Central Processing Unit) 31 realizes, for example, each function of the information processing device described below by controlling the entire information processing device 30 using computer programs and data stored in the ROM 32 and the RAM 33. The information processing device 30 may have one or more dedicated hardware different from the CPU 31, and at least a part of the processing by the CPU 31 may be executed by the dedicated hardware. Examples of the dedicated hardware include an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit).

[0026] The ROM (Read Only Memory) 32 stores programs and the like that do not require modification. The RAM (Random Access Memory) 33 temporarily stores programs and data supplied from the storage device 34, and data supplied from the outside via the communication unit 37. The RAM 33 also functions as a main memory, work area, and the like for the CPU 31. The storage device 34 is composed of, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various data. The storage device 34 stores, for example, various data required when the CPU 31 performs processing related to the programs, and various data obtained by the CPU 31 performing processing related to the programs.

[0027] The operation unit 35 is composed of, for example, a keyboard, a mouse, a joystick, a touch panel, etc., and receives operations by a user to input various instructions to the CPU 31. The display unit 36 ​​is composed of, for example, a liquid crystal display, an LED, etc., and displays a GUI (Graphical User Interface) for the user to operate the information processing device 30, processing results by the CPU 31, etc. Note that the operation unit 35 and the display unit 36 ​​may be present as separate devices outside the information processing device 30. The communication unit 37 connects the information processing device 30 to a network and controls communication with other devices, etc.

[0028] 4 is a block diagram showing an example of a functional configuration of an information processing device that performs a search using a trie structure in the first embodiment. The information processing device in the first embodiment has a search unit 41 and a storage unit 42. The search unit 41 receives a search key as an input, and outputs an attribute value corresponding to the search key by referring to the trie structure stored in the storage unit 42. The storage unit 42 stores the trie structure used for the search.

[0029] In this embodiment, an example of a trie structure stored in the storage unit 42 is shown in FIG. 5. The trie structure shown in FIG. 5 corresponds to the four sets of search keys and attribute values ​​shown in FIG. 1. Note that the node patterns represent the attribute values ​​as shown in FIG. 2(b), similar to FIG. 2(a). In the trie structure shown in FIG. 2(a), attribute values ​​are associated only with the terminal nodes, but in the trie structure shown in FIG. 5, attribute values ​​are set for all nodes. Also, the trie structure shown in FIG. 5 does not have a terminal node that undergoes a state transition due to a terminal word.

[0030] The search performed by the search unit 41 will be described with reference to Fig. 5. The trie structure shown in Fig. 5 is a state transition diagram starting from the root node 200. For example, a search for an attribute value corresponding to the search key "C000.png" is performed as follows. Word division for the search key is as described above, and division is performed with 1 word = 1 character. First, the search unit 41 performs a state transition from the root node 200 to node 201 by the first word "C". Next, the search unit 41 attempts to perform a state transition by the second word "0", but there is no transition destination. In other words, when the search key "C000.png" is input, the node that has been reached the deepest (hereinafter also referred to as the "longest reached node") is node 201. Therefore, the search unit 41 outputs the attribute value " / mnt / vol0 / 00 / " associated with node 201, which is the longest reached node, as a search result for the search key "C000.png".

[0031] The same is true for searching an attribute value corresponding to the search key "C001.png". The search unit 41 performs a state transition from the root node 200 to node 201 due to the first word "C". Since there is no transition destination due to the second word "0", node 201 becomes the longest reached node. Therefore, the search unit 41 outputs the attribute value " / mnt / vol0 / 00 / " associated with node 201, which is the longest reached node, as the search result for the search key "C001.png".

[0032] It is worth noting that the search key "C000.png" and the search key "C001.png" have the same node state transitions. In other words, the number of additional nodes required to search for the search key "C001.png" is 0, and no storage space is consumed.

[0033] Similarly, in a search for the search key "C100.png", the search unit 41 performs a state transition from the root node 200 to node 201 due to the first word "C". Next, the search unit 41 performs a state transition from node 201 to node 210 due to the second word "1". Next, the search unit 41 attempts to perform a state transition due to the third word "0", but since there is no transition destination, node 210 becomes the longest reached node. Therefore, the search unit 41 outputs the attribute value " / mnt / vol0 / 01 / " associated with node 210, which is the longest reached node, as the search result for the search key "C100.png".

[0034] Furthermore, in a search for the search key "C120.png", the search unit 41 performs a state transition from the root node 200 to node 201 due to the first word "C". Next, the search unit 41 performs a state transition from node 201 to node 210 due to the second word "1". Next, the search unit 41 performs a state transition from node 210 to node 220 due to the third word "2". Next, the search unit 41 attempts to perform a state transition due to the fourth word "0", but since there is no transition destination, node 220 becomes the longest reached node. Therefore, the search unit 41 outputs the attribute value " / mnt / vol1 / 00 / " associated with node 220, which is the longest reached node, as a search result for the search key "C120.png".

[0035] In this embodiment, when a search is performed using a search key not shown in FIG. 1, for example, "C300.png", the state transitions to the root node 200 and node 201. The search unit 41 outputs the attribute value " / mnt / vol0 / 00 / " of the node 201, which is the longest reachable node, as an erroneous search result. In the trie structure shown in FIG. 2(a), if the terminal node cannot be reached, it can be determined that there is no attribute value corresponding to the search key, but this determination cannot be made in the trie structure of this embodiment shown in FIG. 5. That is, in this embodiment, it cannot be determined whether the search key is registered or not. This is a system that returns an attribute value related to the search key if the search key is registered.

[0036] This restriction does not pose a problem, for example, when searches are not performed using any search keys other than the registered search keys (in this example, the search keys shown in FIG. 1). For example, this is the case when a higher-level system generates an intermediate file name (search key), saves the results of a certain process, and notifies another process of the name to perform the subsequent process. Alternatively, it may be possible that the search key is not registered from the search results. This example will be described in the fourth embodiment described later.

[0037] For the four search keys shown in FIG. 1, the total number of state transitions when searching using the normal TRIE structure shown in FIG. 2(a) is 36. On the other hand, when searching using the TRIE structure of this embodiment shown in FIG. 5, the total number of state transitions is 7. In this way, when searching using the TRIE structure of this embodiment, a significant speed improvement is achieved, and data can be searched at high speed. Also, the number of nodes required for a search is 31 in the normal TRIE structure shown in FIG. 2(a), but is only 4 in the TRIE structure of this embodiment shown in FIG. 5, which allows a significant reduction in the storage capacity.

[0038] Second embodiment In the first embodiment, a search using a trie structure is described, but in the second embodiment, a generation of a trie structure stored in the storage unit 42 is described. Note that the hardware configuration of the information processing device in the second embodiment is similar to the hardware configuration of the information processing device in the first embodiment shown in FIG. 3, and therefore a description thereof will be omitted.

[0039] Fig. 6 is a block diagram showing an example of a functional configuration of an information processing device that generates a trie structure to be held in a holding unit 42. In Fig. 6, components similar to those shown in Fig. 4 are given the same reference numerals, and duplicated explanations will be omitted. The information processing device in this embodiment has a search unit 41, a holding unit 42, a registration unit 43, and an enumeration unit 44.

[0040] The registration unit 43 takes a pair of a new search key and an attribute value corresponding to that search key as input, and updates the trie structure held in the holding unit 42 so that searches for that input are also possible. As described above, the search system exemplified in the embodiment is a system that returns an attribute value related to a search key "if the search key is registered," and the trie structure used in the search does not explicitly hold all registered search keys. Therefore, when the trie structure held in the holding unit 42 is updated, search results for registered search keys may result in erroneous attribute values, which must be re-registered. The enumeration unit 44 enumerates candidates for registered search keys that will result in erroneous search results. The enumeration unit 44 is called by the registration unit 43 and used to enumerate all search keys associated with a specific attribute value. The operations of the registration unit 43 and the enumeration unit 44 will be described in detail later.

[0041] In the following, we will explain an example in which a new search key "C200.png" and its corresponding attribute value " / mnt / vol1 / 00 / " are added and registered as shown in FIG. 7(a) to the trie structure shown in FIG. 5, which corresponds to the search key and attribute value shown in FIG. 1. FIG. 8 is an activity diagram for explaining the flow of processing by the registration unit 43 to register a search key and an attribute value.

[0042] First, the registration unit 43 receives a pair of a first key K1 and a first attribute value V1 as an input. In this example, the first key K1 is "C200.png" and the first attribute value V1 is " / mnt / vol1 / 00 / ".

[0043] In step S1, the registration unit 41 performs a search process using the trie structure already stored in the storage unit 42, with the first key K1 as the search key. The contents of the search process are the same as those in the first embodiment. The search result in step S1 is the second attribute value V2, and the longest reached node at that time is the longest reached node N1. Note that, referring to the trie structure shown in FIG. 5, the longest reached node N1 in the search for the search key "C200.png" is node 201, and the second attribute value V2 is " / mnt / vol0 / 00 / ".

[0044] Next, in step S2, the registration unit 43 compares the first attribute value V1 with the second attribute value V2. If the registration unit 43 determines that the first attribute value V1 and the second attribute value V2 match, this means that the first attribute value V1 can be correctly searched for from the first key K1 in the current trie structure stored in the storage unit 42, and the registration process is completed.

[0045] On the other hand, if the registration unit 43 determines that the first attribute value V1 and the second attribute value V2 are different, it means that the current trie structure stored in the storage unit 42 cannot correctly search for the first attribute value V1 from the first key K1, and therefore the trie structure needs to be updated.

[0046] First, the registration unit 43 modifies the current trie structure stored in the storage unit 42 so that the first attribute value V1 can be correctly searched for from the first key K1. In step S3, the registration unit 43 creates a transition node beyond the longest reach node N1 (node ​​201) to perform a new state transition from the longest reach node N1 (node ​​201). In this example, the registration unit 43 creates a node 230 that performs a state transition by the word "2" beyond the node 201 that is the longest reach node N1, as shown in FIG. 7(b). The attribute value of the node 230 is the first attribute value V1. This makes it possible to obtain the first attribute value V1 " / mnt / vol1 / 00 / " as a search result by a search process from the first key K1 "C200.png".

[0047] However, by making a change to the TRIE structure, an incorrect search result (attribute value) may be obtained for a search key that has already been registered, i.e., a search key that has been correctly searched. Therefore, the registration unit 43 checks the updated TRIE structure, and if an incorrect value is returned, the registration unit 43 further modifies the TRIE structure.

[0048] First, in step S4, the enumeration unit 44 enumerates search keys that may return erroneous search results based on instructions from the registration unit 43. It is believed that erroneous search results are returned because a new transition node was added beyond the longest reach node N1 in step S3, causing an erroneous state transition to that node. Therefore, search keys associated with the attribute value of the longest reach node N1, i.e., the second attribute value V2, are enumerated. In this example, two keys, "C000.png" and "C001.png", are enumerated as search keys having the second attribute value V2 " / mnt / vol0 / 00 / ".

[0049] The method of enumeration performed by the enumeration unit 44 in step S4 is not particularly limited. For example, a key-value type database in which a search key can be obtained from an attribute value may be constructed and used at the same time as registering a pair of a search key and an attribute value in the search system. In this case, a storage area the size of the database is required, but in a system in which the registration phase and the search phase are separated, this key-value type database is not required at the time of search, so there is no increase in the storage area. Other examples will be described in the fourth embodiment described later.

[0050] The following process SS1 is performed for each of the search keys listed in step S4. The process SS1 will be described below using the search keys "C000.png" and "C001.png" as examples.

[0051] First, the registration unit 43 sets the search key "C000.png" as the third key K3. In step S5, the registration unit 43 refers to the TRIE structure updated in step S3, performs a search process with the third key K3 "C000.png", and obtains the third attribute value V3 " / mnt / vol0 / 00 / " as the search result.

[0052] In step S6, the registration unit 43 compares the second attribute value V2 with the third attribute value V3. If the registration unit 43 determines that the second attribute value V2 and the third attribute value V3 are equal, the search result of the third key K3 is correct, and the process for this key is terminated. On the other hand, if the registration unit 43 determines that the second attribute value V2 and the third attribute value V3 are different, the search result of the third key K3 is incorrect, and the registration process is performed for this key in step S7. In this example, the second attribute value V2 " / mnt / vol0 / 00 / " and the third attribute value V3 " / mnt / vol0 / 00 / " are equal, and the process is terminated. The same process is performed on the remaining listed search key "C001.png", but the comparison result in step S6 is "equal" so the process ends.

[0053] This completes the process SS1 for all the enumerated search keys. This completes the registration process in the registration unit 43, and creates a trie structure as shown in Fig. 7(b) that returns correct attribute values ​​for all the search keys shown in Fig. 7(a).

[0054] An example of performing the registration process in step S7 in process SS1 will be described below. In the following, an example will be described in which the search key "C010.png" and the corresponding attribute value " / mnt / vol1 / 00 / " are registered as shown in FIG. 9(a) in addition to the example shown in FIG. 7(a).

[0055] When the same processing as steps S1 to S3 described above is performed with the first key K1 set to "C010.png" and the first attribute value V1 set to " / mnt / vol1 / 00 / ", the trie structure is updated to a trie structure with node 240 added as shown in FIG. 9(b).

[0056] In step S4, the enumeration unit 44 enumerates all search keys having the attribute value of the node 201, which is the longest reached node N1, i.e., the second attribute value V2 " / mnt / vol0 / 00 / ". In this example, two keys, "C000.png" and "C001.png", are enumerated as search keys having the second attribute value V2 " / mnt / vol0 / 00 / ". Then, process SS1 is performed.

[0057] In step S5, the registration unit 43 performs a search process with the search key "C000.png" as the third key K3 by referring to the TRIE structure shown in Fig. 9(b) updated in step S3. At this time, since the node 240 is the longest reached node, the third attribute value V3 " / mnt / vol1 / 00 / " is obtained as the search result.

[0058] Next, in step S6, the registration unit 43 compares the second attribute value " / mnt / vol0 / 00 / " with the third attribute value " / mnt / vol1 / 00 / " and obtains a result that the attribute values ​​are different. This means that a state transition occurs to the node 240 created in step S3, and as a result, an incorrect attribute value is returned. To correct this so that a correct search is performed, in step S7, the third key K3 "C000.png" and the second attribute value V2 " / mnt / vol0 / 00 / " are paired and registered again. The updated trie structure is shown in FIG. 9(c). In the trie structure shown in FIG. 9(c), a node 241 having an attribute value " / mnt / vol1 / 00 / " is newly created after the node 240 in the trie structure shown in FIG. 9(b).

[0059] The update of the trie structure performed here will now be described in detail. First, when a search is performed using the search keys "C000.png" and "C001.png" using the trie structure shown in FIG. 7(b), the longest reached node is node 201. Similarly, in a search for the search key "C010.png", node 201 is also the longest reached node, but because this has an incorrect attribute value, node 240 is added beyond node 201, which is the longest reached node, as shown in FIG. 9(b). The addition of node 240 changes the longest reached node of a portion (all of the search keys in this example) of the search keys whose final reached node was 201 to node 240. Therefore, these (parts of the search keys with node 240 as the longest reached node) are further state-transitioned beyond node 240.

[0060] The process SS1 is also performed on the remaining listed search key "C001.png." In this case, the longest reached node is node 241, and the comparison in step S6 results in "equal."

[0061] With the above, the registration process by the registration unit 43 is completed, and a trie structure as shown in Fig. 9(c) is created, which returns correct attribute values ​​for all search keys shown in Fig. 9(a). In this way, even when adding search keys and their corresponding attribute values, it is possible to build a trie structure that suppresses an increase in the storage area required for searches and enables high-speed data search.

[0062] In the above explanation, a new search key (first key K1) and attribute value (first attribute value V1) are added and registered to an existing trie structure, but when registering in a search system for the first time, for example, the following can be done. The initial state is a state in which no trie structure exists, and when registering a new one, a root node is created and the first attribute value V1 is associated with the root node. Also, the initial state may be a state in which there is one root node, and the attribute value may be an indefinite value.

[0063] (Third embodiment) As described in the second embodiment, by sequentially registering pairs of search keys and attribute values, it is possible to update the trie structure held in the holding unit 42. However, the updated trie structure does not necessarily have good search efficiency. For example, in order to correctly perform a search related to the search key shown in FIG. 9(a), the trie structure shown in FIG. 9(c) is reconstructed in the second embodiment. However, a correct search result can be obtained even with the trie structure shown in FIG. 10 in which the redundant node (node ​​230) has been deleted. The reduction in the number of nodes contributes to a reduction in the storage area, and the number of state transitions is also reduced, which is expected to improve the search speed.

[0064] In the third embodiment, a case where a process of deleting redundant nodes is added to the registration process in the registration unit 43 will be described with reference to Fig. 11. Note that the configuration of the information processing device in the third embodiment is similar to the hardware configuration of the information processing device shown in Fig. 3 and the functional configuration of the information processing device shown in Fig. 6, and therefore description thereof will be omitted.

[0065] FIG. 11 is an activity diagram for explaining the flow of a process of registering a search key and an attribute value by the registration unit 43 in the third embodiment. First, in step S10, the registration unit 43 (and the enumeration unit 44) performs a registration process. The registration process in step S10 is the entire process shown in Fig. 8, and the details are as described in the second embodiment. The longest reached node N1 and the second attribute value V2, which are the results of the registration process in step S10, are used in the subsequent processes.

[0066] In step S11, the enumeration unit 44 enumerates search keys having the second attribute value V2 based on an instruction from the registration unit 43. This is exactly the same as the process in step S4 in Fig. 8, and the processing result in step S4 in Fig. 8 may be used as is. In this example, two search keys, "C000.png" and "C001.png", are enumerated.

[0067] Next, in step S12, the registration unit 43 performs a search process for each search key listed in step S11, lists each longest reached node N2, and obtains the longest reached node list L1. In this example, a search is performed using two search keys, "C000.png" and "C001.png", and the longest reached node list L1 includes node 241.

[0068] In step S13, the registration unit 43 judges whether the longest reaching node N1 is included in the longest reaching node list L1. If the registration unit 43 judges that the longest reaching node N1 is included in the longest reaching node list L1, the longest reaching node N1 is used to obtain the attribute value that is the search result, so there is no redundant node in the trie structure. Then, the registration process shown in FIG. 11 ends.

[0069] On the other hand, if the registration unit 43 determines that the longest reached node N1 is not included in the longest reached node list L1, the longest reached node N1 is used only for state transitions, so the registration unit 43 attempts to delete the node to minimize the number of nodes. In this example, the longest reached node N1 is node 201, so it is not included in the longest reached node list L1. Therefore, the registration unit 43 attempts to delete the node.

[0070] In step S14, the registration unit 43 selects one leaf node from among the child nodes of the longest reach node N1, and sets it as child leaf node N3. If there is no child node that is a leaf node, the registration process shown in Fig. 11 ends. In this example, as shown in Fig. 9(c), the child nodes of node 201, which is the longest reach node N1, are node 210, node 230, and node 240, but among them, node 230 is the only leaf node that does not have a child. Therefore, the registration unit 43 selects node 230, and sets it as child leaf node N3.

[0071] Then, in steps S15 and S16, the registration unit 43 changes and reconstructs the trie structure. The registration unit 43 changes the trie structure so that the longest reached node in a search for a search key that has undergone a state transition to the child leaf node N3 becomes the longest reached node N1. Specifically, in step S15, the registration unit 43 rewrites the attribute value associated with the longest reached node N1 with the attribute value of the child leaf node N3. Also, in step S16, the registration unit 43 deletes the node that has become the child leaf node N3 from the trie structure.

[0072] As described above, by deleting redundant nodes from the TRIE structure and reconstructing it to minimize the number of nodes, the storage capacity can be reduced and the number of state transitions can be reduced, thereby improving search speed.

[0073] (Fourth embodiment) In the first to third embodiments, an example of an abstract search system that handles an arbitrary search key and an arbitrary attribute value is shown. In this embodiment, a description is given based on a specific example of a search key and an attribute value. In this embodiment, a hierarchical file system having a tree-structured folder structure and files stored therein are described.

[0074] Generally, users who use a file system have a need to handle files with path names that are easy for users to understand, regardless of the folder structure of the file system. On the other hand, there is a need to save files in appropriate folders regardless of path names so that data storage can be used as efficiently as possible. In order to meet each of these needs, for example, the semantic path used by the user can be managed separately as a "virtual path" and the latter path as a "real path", and the two can be associated with each other. The operation related to this association needs to be high-speed and memory-saving so as not to put a strain on the operation of the entire system. A search system obtained by applying the present invention can be used as such a mechanism.

[0075] The search system exemplified in this embodiment uses a virtual path of a file (hereinafter also referred to as a "virtual file path") as a search key. Also, the real path of the file storage location (folder) (hereinafter also referred to as a "real folder path") is used as an attribute value. Here, the term "file path" is used, but in this specification, the name of a file including the folder structure will be called a "file path" and the name of a file not including the folder structure will be called a "file name" to distinguish between them. Similarly, the name of a folder including the folder structure will be called a "folder path" and the name of a folder not including the folder structure will be called a "folder name" to distinguish between them.

[0076] As the virtual file path, for example, character strings such as " / Photo / Myself / Img_001.png", " / Drama / WhatHappened / Img_001.png", and " / Photo / Paul / Img_001.png" are considered. In this way, it is assumed that the virtual file path includes folder names that are easy to semantically classify, such as "Photo" and "Paul". In addition, here, the real folder paths (attribute values) corresponding to the virtual file paths (search keys) are respectively character strings " / mnt / vol0 / 00 / ", " / mnt / vol0 / 01", and " / mnt / vol1 / 00 / ". In this way, where the files of the virtual file paths are saved depends on the correspondence between the virtual file path and the real folder path. Therefore, any two files may be saved in the same real folder or in different real folders.

[0077] In a general file system, multiple files with the same file name cannot be saved in one folder. Therefore, file names are converted in advance so that there are no multiple identical file names. Here, as shown in FIG. 12(a), a conversion unit 46 is used to obtain a new real file name F2 from a virtual file path F1. The conversion unit 46 converts an arbitrary virtual file path F1 and a real file name F2 so that they correspond one-to-one, and thus if the virtual file path F1 is different, the real file name F2, which is the conversion result, will also be different. In other words, if the real file name F2 is used as a search key, it can be directly applied to the systems shown in the first to third embodiments.

[0078] An example of the functional configuration of the information processing device in this case is shown in Fig. 12(b). In Fig. 12(b), the same components as those shown in Fig. 4 and Fig. 6 are given the same reference numerals, and duplicated explanations are omitted. Note that the hardware configuration of the information processing device in this embodiment is the same as the hardware configuration of the information processing device in the first embodiment shown in Fig. 3.

[0079] In this embodiment, the conversion unit 46 converts the virtual file path F1 into a real file name F2, and the search unit 41 and the registration unit 43 use the real file name F2 as a search key. Here, the conversion unit 46 wants to use the virtual file path F1 as the real file name F2 as it is, but characters that cannot be used as a file name must be removed. The separator between folder names and the characters used to separate folder names and file names (hereinafter referred to as "folder separators") are determined depending on the file system. Therefore, it is necessary that the real file name F2 output by the conversion unit 46 does not include a folder separator. Furthermore, since the virtual file path F1 represents the top folder, the first character is always a folder separator. Therefore, the conversion unit 46 may ignore the first folder separator.

[0080] For the above reasons, the conversion unit 46 may convert the virtual file path F1 into the real file name F2 by, for example, applying the following rules 1 to 3 in order. Note that the folder separator here is " / ". (Rule 1) Remove the leading " / " (Rule 2) Convert "#" to "#n" (Rule 3) Replace " / " with "##" Rule 1 deletes the leading folder separator that indicates the root folder of the virtual file path. Rule 3 converts the file separator to another character string ("##"). This alone makes it impossible to distinguish whether the "##" included in the converted real file name F2 is the result of the conversion by rule 3 or was included in the virtual file path F1. Therefore, rule 2 is applied before rule 3 to make the distinction possible. Note that the conversion by the conversion unit 46 is not limited to the above-mentioned method, and any conversion method can be used as long as there is a one-to-one correspondence between an arbitrary virtual file path F1 and a converted real file name F2.

[0081] According to rules 1 to 3, the conversion unit 46 converts, for example, the virtual file path " / Photo / Myself / Img_001.png" into the real file name "Photo##Myself##Img_001.png". That is, assume that the file with the virtual file path " / Photo / Myself / Img_001.png" is saved in the real folder path " / mnt / vol0 / 00 / ". In this case, the file path that is actually saved (i.e., the real file path) is " / mnt / vol0 / 00 / Photo##Myself##Img_001.png".

[0082] Note that the search key must be uniquely identifiable, but by using the conversion unit 46, if the virtual file path is uniquely identifiable, the search key (real file name F2) can also be uniquely identifiable. However, various conversion rules for such conversion unit 46 are possible and are not limited to one form. In this way, the conversion unit 46 obtains the real file name from the virtual file path.

[0083] As described above, the exemplified search system is a system that returns an attribute value related to a search key if the search key is registered, and it is usually not possible to determine whether the search key (here, the virtual file path) is registered in the system. However, in this embodiment, the search key (virtual file path) and the real file name are in one-to-one correspondence. That is, the search key and the real file path that combines the real folder path and the real file name are also in one-to-one correspondence. Therefore, whether the search key is registered and whether the real file path has a file correspond to each other. In other words, whether the search key is registered can be known by checking whether the file in the search result (real file path) is present or not, or by actually opening the file. Checking whether the file is present is a command to the file system, and can be realized, for example, in Python, by the os.is_file() API.

[0084] In this embodiment, the enumeration process performed by the enumeration unit 44 is to enumerate search keys (real file names) associated with attribute values ​​(real folder paths). This can be achieved by enumerating file names (real file names) that actually exist in the folder represented by the real folder path, which is the attribute value. Many file systems provide such an instruction, and in Python, for example, this can be achieved by the os.listdir() API.

[0085] According to this embodiment, it is possible to efficiently use data storage and to handle files with path names that are easy for users to understand, regardless of the folder structure of the file system.

[0086] Fifth embodiment In the fourth embodiment, a pair of a search key (virtual file path) and an attribute value (real folder path) is registered in the search system, but a system in which the user does not need to specify a real folder path is preferable because it places less of a burden on the user. For example, a user makes a request to the system to create a file identified by the virtual file path " / Photo / Myself / Img_001.png". The system that receives this request returns the real file path " / mnt / vol0 / 00 / Photo##Myself##Img_001.png". The user can create a file with the returned real file path. After that, the user makes a request to the system to read a file identified by the virtual file path " / Photo / Myself / Img_001.png". The system that receives this request returns the real file path " / mnt / vol0 / 00 / Photo##Myself##Img_001.png". The user can read the file with the returned real file path.

[0087] An example of the functional configuration of the information processing device in this case is shown in Fig. 13. In Fig. 13, the same components as those shown in Fig. 4, Fig. 6, and Fig. 12(b) are given the same reference numerals, and duplicated explanations are omitted. Note that the hardware configuration of the information processing device in this embodiment is the same as the hardware configuration of the information processing device in the first embodiment shown in Fig. 3.

[0088] The registration process and search process performed by the information processing device in this embodiment will be described below. The registration process is a process in which, when a user inputs a virtual file path to the information processing device, the information processing device returns a real file path that can actually be written. The information processing device updates the trie structure so as to return a real file path from this virtual file path. The user writes a file to the obtained real file path. The search process is a process in which, when a virtual file path is input to the information processing device, the information processing device returns the real file path registered by the registration process. The user reads from the file of the obtained real file path.

[0089] FIG. 14(a) is an activity diagram illustrating the flow of the registration process in this embodiment. In the registration process, first, the user inputs the virtual file path F21 to the information processing device. In this example, the virtual file path F21 is " / Photo / Myself / Img_001.png".

[0090] In step S21, the conversion unit 46 performs a conversion process to convert the virtual file path F21 into a real file name F22. In this example, the conversion is performed by sequentially applying the above-mentioned rules 1 to 3, and the real file name F22 after conversion becomes "Photo##Myself##Img_001.png".

[0091] Next, in step S22, the determination unit 45 performs a determination process. In the determination process in step S22, the determination unit 45 receives the real file name F22 and determines the destination folder path F23 in which the file should be saved based on the real file name F22. The method of determining the destination folder path F23 in the determination process is not limited in this embodiment, but may be, for example, as follows. FIG. 14(c) shows the correspondence between the file name prefix and the destination folder path. When the real file name F22 starts with the "file name prefix" in the first column, the determination unit 45 may refer to the correspondence shown in FIG. 14(c) and set the "destination folder path" in the second column of the corresponding row as the destination folder path F23. According to this, when the real file name F22 "Photo##Myself##Img_001.png" is given, the destination folder path F23 is determined to be " / mnt / vol0 / ".

[0092] Next, in step S23, the search unit 41 and the registration unit 43 perform registration processing S23 in the trie structure held in the holding unit 42, with the real file name F22 as a search key and the destination folder path F23 as an attribute value. The registration processing in step S23 is the processing shown in Fig. 8 or Fig. 11, and the details have already been described.

[0093] Finally, in step S24, the information processing device synthesizes a real file path based on the real file name F22 and the destination folder path F23, and returns the resulting path to the caller. For example, the real file path can be obtained by concatenating the destination folder path F23 and the real file name F22. In this example, the real file path is " / mnt / vol0 / Photo##Myself##Img_001.png". The user obtains this real file path and writes to this file.

[0094] Next, the search process will be described with reference to Fig. 14(b) which is an activity diagram illustrating the flow of the search process in this embodiment. In the search process, the user inputs the virtual file path F31 to the information processing device.

[0095] In step S31, the conversion unit 46 performs a conversion process to convert the virtual file path F31 into a real file name F32. The conversion process in step S31 is similar to the conversion process in step S21 shown in FIG. 14(a). The virtual file path F31 is specified by the user during the registration process shown in FIG. 14(a). In this example, the virtual file path F31 is " / Photo / Myself / Img_001.png". Therefore, "Photo##Myself##Img_001.png" is obtained as the real file name F32.

[0096] In step S32, the search unit 41 performs a search process using the real file name F32 as a search key, using the trie structure stored in the storage unit 42, and obtains the destination folder path F33. The search process in step S32 is the same as the search process in step S1 shown in Fig. 8. In this example, " / mnt / vol0 / " is obtained as the destination folder path F33.

[0097] Finally, in step S33, the information processing device synthesizes a real file path based on the real file name F32 and the destination folder path F33, and returns the resulting path to the caller. The process in step S33 is the same as the process in step S24 shown in FIG. 14(a). In this example, " / mnt / vol0 / Photo##Myself##Img_001.png" is returned to the caller as the real file path. The user obtains this real file path and reads from this file.

[0098] As a result, the user can write to and read from the real file using only the virtual file path. Note that, although the real file path is simply returned in the registration process here, the real file path may be returned after the file is actually created.

[0099] Sixth embodiment In the fifth embodiment, the user can write to and read from real files using only the virtual file path. However, in the fifth embodiment, the destination folder path determined by the determination unit 45 is predetermined by the file name prefix. Therefore, in the registration process, if there are many virtual file paths starting with " / Photo / ", real files will be created only in " / mnt / vol0 / ".

[0100] For example, suppose that physically different storage areas (volumes) are located directly under / mnt / . In this case, the volume " / mnt / vol0 / " has almost no free space, but other volumes such as " / mnt / vol1 / " and " / mnt / vol2 / " are hardly used. In this case, even though other volumes have plenty of free space, file writing will fail due to insufficient free space on the volume.

[0101] Therefore, in the sixth embodiment, the data size to be written to the file is also passed to the determination process, and the determination unit 45 determines the actual folder path of a volume with free space equal to or larger than the data size to be written as the destination folder path. In this way, the volume capacity can be used effectively, and furthermore, failure of file writing due to a lack of free space in a specific volume can be prevented.

[0102] Fig. 15(a) is an activity diagram explaining the flow of the registration process in this embodiment. The differences from the process shown in Fig. 14(a) are: (1) the data size DS1 is input, (2) the determination process (S22) is changed to a determination process (S221), and (3) the data size DS1 is used as the input for the determination process instead of the real file name F22.

[0103] The determination process in step S221 will be described below. Figure 15(b) shows the flow of the determination process in step S221. Figure 15(c) shows the correspondence between the current destination folder path and its free space. First, in step S41, the determination unit 45 acquires a list of destination folder paths. As a result, a list consisting of four elements, " / mnt / vol0 / ", " / mnt / vol1 / ", " / mnt / vol2 / ", and " / mnt / vol3 / ", is obtained.

[0104] Next, in step S42, the determination unit 45 acquires the free space of the folder of each element (destination folder path). Acquiring the free space is a command to the file system, and in Python, for example, it can be acquired by shutil.disk_usage() API.

[0105] In step S43, the determination unit 45 determines whether or not each destination folder has sufficient free space. For example, when "1 GB" is input as the data size DS1, it is determined whether or not the free space is 1 GB or more. Then, a list of destination folder paths that are determined to have free space of 1 GB or more is output. In this example, a list consisting of four elements, " / mnt / vol0 / ", " / mnt / vol1 / ", " / mnt / vol2 / ", and " / mnt / vol3 / ", is obtained.

[0106] For example, if "3GB" is entered as the data size DS1, it will be determined that " / mnt / vol0 / " does not have enough free space. Therefore, a list of destination folder paths consisting of three elements, " / mnt / vol1 / ", " / mnt / vol2 / ", and " / mnt / vol3 / ", will be obtained. Similarly, if "300GB" is entered as the data size DS1, a list of destination folder paths consisting of two elements, " / mnt / vol2 / " and " / mnt / vol3 / " will be obtained.

[0107] When a list of destination folder paths consisting of multiple elements is obtained, in step S44, the determination unit 45 selects one from the list and ends the determination process in step S221. There is no particular limitation on which element in the list of destination folder paths to select, and for example, if the first element is selected, it will be " / mnt / vol0 / ". In this case, a specific volume will be used preferentially. Also, if elements in the list of destination folder paths are selected randomly, all volumes will be used evenly.

[0108] As a result, the volume capacity can be used efficiently, and file writing can be prevented from failing due to a lack of free space in a particular volume.

[0109] In the above example, the determination process in step S221 uses only the data size DS1 as input, but both the real file name F22 and the data size DS1 may be input. FIG. 16(a) is an activity diagram for explaining the flow of the registration process in this case. The difference between the process shown in FIG. 16(a) and the process shown in FIG. 15(a) is that the determination process S221 is replaced by the determination process S222, and not only the data size DS1 but also the real file name F22 is used as the input for the determination process. FIG. 16(b) shows the flow of the determination process performed in step S222. The specific process of the determination process in step S222 is almost the same as the process shown in FIG. 15(b), except for the part where step S41 is replaced by step S411.

[0110] Figure 16(c) shows the correspondence between file prefixes and destination folder paths. In the determination process S22 already described, the correspondence shown in Figure 15(c) was used, but in Figure 16(c), the destination folder path in the second column is not a single folder path but a list of folder paths.

[0111] In step S411, the determination unit 45 compares the file name prefix in the first column in FIG. 16(c) with the real file name F22, and creates a list of destination folder paths by listing the destination folder paths (second column) of the rows where the prefixes match. For example, if the real file name F22 is "Photo##Myself##Img_001.png", the prefix "Photo##" matches the first row in FIG. 16(c). Therefore, in this case, a list of destination folder paths consisting of two elements, " / mnt / vol0 / " and " / mnt / vol3 / ", is obtained. Subsequent processing is the same as in the determination processing S221.

[0112] In the case where the first element is selected in step S44, if "1 GB" is input as the data size DS1, in the determination process in step S222, the determination unit 45 outputs " / mnt / vol0 / " as the destination folder path. Also, if "3 GB" is input as the data size DS1, in the determination process in step S222, the determination unit 45 outputs " / mnt / vol3 / " as the destination folder path.

[0113] In this way, when both the real file name and data size are used as inputs for the decision process, a real file name starting with "Photo##" will use " / mnt / vol0 / " for file writing. Also, a real file name starting with "Drama##" will use " / mnt / vol1 / " for file writing. However, if there is not enough free space on the volume, it is possible to use " / mnt / vol3 / ", and so on.

[0114] Although the actual file path is simply returned in the registration process here, the actual file path may be returned after actually creating a file (whether the content is irrelevant) whose size is data size DS1.

[0115] For example, suppose that " / mnt / vol0 / " has 2GB of free space, and two different programs register to write 1.5GB with a real file name starting with "Photo##". In this case, the real file path within the " / mnt / vol0 / " volume is returned to both programs. In this case, one or both programs will fail to write the file due to insufficient free space. However, such failures can be avoided by creating a file of the actual size and then returning the real file path.

[0116] Seventh embodiment In the fourth to sixth embodiments, the form of saving files in a file system has been described. In this case, when a large number of files, such as tens of thousands to millions, are registered, tens of thousands or more files will be created in one folder. In such a case, even if a user directly browses the folder using a file browser or the like, the browsability is poor, making it difficult to confirm and operate the desired file. In addition, there is a problem that it takes a lot of time to enumerate the files in the folder in the process by the enumeration unit 44 (step S4 shown in FIG. 8 and step S11 shown in FIG. 11), which increases the time for the registration process.

[0117] These issues can be resolved by limiting the number of files per folder to a certain number or less. To achieve this, a process of adding a subfolder name to the determined destination folder path may be added at the end of the determination process of the determination unit 45.

[0118] Fig. 17 shows an activity diagram for explaining the flow of the decision process S500 in this case. This decision process S500 replaces the decision process S22 in Fig. 14(a) (or the decision process S221 in Fig. 15(a) or the decision process S222 in Fig. 16(a)).

[0119] First, in step S50, the determination unit 45 performs a determination process to obtain a destination parent folder path F51. Here, the determination process S50 is the same as the determination process S22 (or the determination process S221, or the determination process S222), and the destination folder path F23, which is the result of these processes, is set as the destination parent folder path F51 here. In this example, the destination parent folder path F51 is set to " / mnt / vol0 / ".

[0120] Next, in step S51, the determination unit 45 lists the names of subfolders (subfolder names) that already exist in the folder identified by the destination parent folder path F51. There are no particular restrictions on the subfolder names. However, in order to simplify the explanation of the process, the subfolder names are assumed to be represented by numeric character strings from "0000" to "9999". Subfolders are created in ascending order of numbers, and when a predetermined number of files are created (registered) in a subfolder, a folder with the next number is created. To list the subfolder names in a folder, for example, this can be achieved by the os.listdir() API in Python. Although the subfolder names are character strings, in this example they are converted to numeric values ​​for later processing.

[0121] If at least one subfolder exists, in step S52, the determination unit 45 sets the largest number among the subfolder names as the maximum folder number F52. If no subfolders exist, in step S53, the determination unit 45 sets the folder number F53, which will be described later, to 0. In this example, it is assumed that 101 subfolders, from "0000" to "0100", have been found. In this case, the maximum value, 100, becomes the value of the maximum folder number F52.

[0122] Next, in step S54, the determination unit 45 calculates the number of files in the folder with the largest subfolder name "0100". For example, in Python, the files in the subfolder can be enumerated using the os.listdir() API and the number of elements can be counted. In this example, it is assumed that 5000 files exist.

[0123] In step S55, the determination unit 45 determines whether the number of files calculated in step S54 is equal to or greater than a predetermined number. For example, if the predetermined number is 10,000, since 5,000<10,000, the determination unit 45 transitions to step S56 and sets the folder number F53 to the maximum folder number F52 (here, 100). For example, if the number of files in the subfolder is 10,000, the determination unit 45 proceeds to step S57 and sets the folder number F53 to the maximum folder number F52 plus 1 (here, 101).

[0124] Finally, in step S58, the determination unit 45 converts the folder number F53 into a character string and adds it to the end of the destination parent folder path F51. In this example, this is " / mnt / vol0 / 0100 / ".

[0125] As a result of the above, files created under the destination parent folder path F51 will be created in order from the lowest numbered folder, with 10,000 files directly under "0000", then 10,000 files directly under "0001", etc. Once 10,000 files have been created in each subfolder, the subfolder with the next number will be created. For example, if the number of files per folder (predetermined number) is 10,000, and up to 10,000 subfolders can be created in the destination parent folder path, then up to 100 million files can be saved per destination folder path.

[0126] In this embodiment, a numeric character string is used as the subfolder name, but other naming rules may be used. There is no particular restriction on the naming rule, and the feature here is that files are placed under a subfolder. If more files are to be saved, a subfolder may be created under the subfolder. In this case, for example, if the number of folders per folder is 1000 and the number of files is 10000, 10 billion files can be saved per destination folder path.

[0127] In the determination process S500, the destination folder path including the subfolder name is returned as a character string, but if the subfolder does not exist, it may be created.

[0128] As described above, by limiting the number of files per folder to a certain number or less, file browsing is improved and increases in registration processing time can be prevented.

[0129] Eighth embodiment In the first to seventh embodiments, one attribute value is searched from the search key. However, there are cases where multiple attributes are desired to be searched from the search key. For example, in face recognition learning, a face image set called a mini-batch is often created and learned. In this case, in addition to the destination folder path, it may be desired to know the residential area of ​​a person from the search key. This makes it possible to dynamically create a mini-batch of a face set of people with a balanced residential area.

[0130] Figure 18 shows an example of the correspondence between a search key and two types of attribute values ​​(destination folder path and residence area). For example, assume that you want to search for two types of attribute values ​​as shown in Figure 18. In this case, you can search for the destination folder path using the trie structure shown in Figure 10, and search for the residence area using the trie structure shown in Figure 19(a).

[0131] Here, the trie structure shown in Fig. 10 is a trie structure that searches for a first type attribute value (destination folder path) from a search key, and the trie structure shown in Fig. 19(a) is a trie structure that searches for a second type attribute value (residential area) from a search key. The attribute values ​​of the nodes in the trie structure shown in Fig. 19(a) are shown by letters inside the nodes (circles). Also, state transitions are shown by dashed arrows.

[0132] Comparing the trie structures shown in Fig. 10 and Fig. 19(a), it can be seen that state transitions are the same for many nodes. Therefore, these two trie structures are overlapped and integrated as shown in Fig. 19(b). In this way, nodes that represent common prefixes in searches for the first type attribute value and the second type attribute value are consolidated into one node. In this example, the total number of nodes in each trie structure shown in Fig. 10 and Fig. 19(a) is 12, but the number of nodes in the trie structure shown in Fig. 19(b) that overlaps them is 8. Since the amount of information attached to each node increases, the memory amount before and after integration is not completely proportional to the number of nodes, but such integration often allows a significant reduction in the storage capacity (memory amount) required for searches.

[0133] In the case of the trie structure shown in Figure 19(b), there are four types of nodes: (1) two state transitions occur for the same node, (2) two state transitions occur for different nodes, (3) only one state transition occurs, and (4) no state transition occurs. On the other hand, in the case of the trie structure shown in Figure 10, there are only two types of nodes, (3) and (4). Since the trie structure shown in Figure 19(b) has more types of transitions, the logic for deciding the state transitions becomes more complex, and the amount of calculation required to decide the state transitions increases. However, when looking at the overall operating time of an automaton that generally performs state transitions, it is the number of state transitions that dominates.

[0134] When all the search keys in the first column shown in Fig. 18 are searched once each, the number of state transitions is 16 in the Trie structures of Fig. 10 and Fig. 19(a), totaling 32. On the other hand, when two attribute values ​​are searched simultaneously in the Trie structure shown in Fig. 19(b), the number of state transitions is 19. Although the number of state transitions and the search time are not simply proportional to each other, it is possible to reduce the time required for searches in many cases by integrating them.

[0135] (Another embodiment of the present invention) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiment is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0136] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0137] The disclosure of the present embodiment includes the following configurations, methods, etc. (Configuration 1) A storage means for storing a first trie structure generated based on a uniquely identifiable search key; a search means for searching for an attribute value corresponding to the search key from the search key by referring to the first trie structure stored in the storage means, the first trie structure has each node associated with the attribute value; The information processing device is characterized in that the search means searches the first trie structure based on the search key, and sets the attribute value associated with the deepest reached node as a search result. (Configuration 2) The information processing device according to configuration 1, characterized in that the search means searches the first trie structure in order for each word constituting the search key, and if a node corresponding to the next word is not present in the first trie structure, the attribute value associated with the current node is set as a search result. (Configuration 3) 3. The information processing device according to configuration 1 or 2, further comprising a registration means for performing a registration process for updating the first trie structure based on a new search key and an attribute value corresponding to the search key. (Configuration 4) The registration means includes: The information processing device according to configuration 3, characterized in that when a first attribute value corresponding to the new search key is different from a second attribute value obtained by searching the first trie structure based on the new search key, a node corresponding to the new search key associated with the first attribute value is created in the first trie structure, to which a transition is made from the node reached deepest in a search using the new search key. (Configuration 5) The registration means includes: The information processing device according to configuration 4, characterized in that, when the first attribute value and the second attribute value are different, for each search key whose corresponding attribute value is the second attribute value, the first trie structure is searched based on the search key to obtain a third attribute value, and a registration process is performed for the search key whose second attribute value and the third attribute value are different and the attribute value corresponding to the search key. (Configuration 6) The information processing device according to configuration 4 or 5, wherein the registration means, when creating a node corresponding to the new search key, reconstructs the node reached deepest in a search with the new search key and the node transitioning from that node so as to minimize the number of nodes in the first trie structure. (Configuration 7) The system has a hierarchical file system that manages files having uniquely identifiable file names in a tree-structured storage location, 3. The information processing apparatus according to configuration 1 or 2, wherein a file name is used as the search key, and a storage location of a file having that file name is used as the attribute value. (Configuration 8) The system has a hierarchical file system that manages files having uniquely identifiable file names in a tree-structured storage location, The information processing device according to configuration 3, further comprising a determination means for determining a storage location of a file having the file name by inputting at least one of the file name and data size, and inputting the file name and the determined storage location of the file to the registration means as the search key and the attribute value. (Configuration 9) The storage means stores a second trie structure generated based on the search key, The second trie structure has each node associated with an attribute value of a different type from that of the first trie structure, 3. The information processing device according to configuration 1 or 2, wherein a node representing the same common prefix in the first trie structure and the second trie structure is configured as one node. (Method 1) An information processing method executed by an information processing device, a retaining step of retaining the first trie structure generated based on the uniquely identifiable search key; a search step of searching for an attribute value corresponding to the search key from the search key by referring to the first trie structure stored in the storage; the first trie structure has each node associated with the attribute value; An information processing method characterized in that, in the search step, a search is performed on the first trie structure based on the search key, and the attribute value associated with the deepest reached node is set as a search result. (Program 1) The computer of the information processing device a retaining step of retaining the first trie structure generated based on the uniquely identifiable search key; a search step of searching for an attribute value corresponding to the search key from the search key by referring to the first trie structure stored in the storage; the first trie structure has each node associated with the attribute value; In the search step, a search is performed on the first trie structure based on the search key, and the attribute value associated with the deepest reached node is set as a search result. [Explanation of symbols]

[0138] 30: Information processing device 41: Search unit 42: Storage unit 43: Registration unit 44: Enumeration unit 45: Determination unit 46: Registration unit

Claims

1. A storage means for storing a first trie structure generated based on a uniquely identifiable search key; a search means for searching for an attribute value corresponding to the search key from the search key by referring to the first trie structure stored in the storage means, the first trie structure has each node associated with the attribute value; The information processing apparatus is characterized in that the search means searches the first trie structure based on the search key, and determines the attribute value associated with the deepest reached node as a search result.

2. 2. The information processing apparatus according to claim 1, wherein the search means searches the first trie structure in order for each word constituting the search key, and if a node corresponding to the next word is not present in the first trie structure, the attribute value associated with the current node is set as the search result.

3. 2. The information processing apparatus according to claim 1, further comprising: a registration unit that performs a registration process for updating the first trie structure based on a new search key and an attribute value corresponding to the search key.

4. The registration means includes:

4. The information processing device according to claim 3, further comprising: a node corresponding to the new search key associated with the first attribute value, which transitions from the node that is deepest reached in a search using the new search key in the first trie structure when a first attribute value corresponding to the new search key is different from a second attribute value obtained by searching the first trie structure based on the new search key.

5. The registration means includes:

5. The information processing apparatus according to claim 4, characterized in that, when the first attribute value and the second attribute value are different, for each search key whose corresponding attribute value is the second attribute value, a third attribute value is obtained by searching the first trie structure based on the search key, and a registration process is performed for the search key whose second attribute value and the third attribute value are different and the attribute value corresponding to the search key.

6. The information processing device according to claim 4, characterized in that, when creating a node corresponding to the new search key, the registration means reconstructs the node reached deepest in a search with the new search key and the node to which the transition is made from that node so as to minimize the number of nodes in the first trie structure.

7. The system has a hierarchical file system that manages files having uniquely identifiable file names in a tree-structured storage location, 2. The information processing apparatus according to claim 1, wherein a file name is used as the search key, and a storage location of a file having the file name is used as the attribute value.

8. The system has a hierarchical file system that manages files having uniquely identifiable file names in a tree-structured storage location, 4. The information processing device according to claim 3, further comprising a determination means for determining a storage location of a file having the file name by inputting at least one of the file name and data size, and inputting the file name and the determined storage location of the file into the registration means as the search key and the attribute value.

9. The storage means stores a second trie structure generated based on the search key, The second trie structure has each node associated with an attribute value of a different type from that of the first trie structure, 2. The information processing apparatus according to claim 1, wherein a node representing the same common prefix in the first trie structure and the second trie structure is configured as one node.

10. An information processing method executed by an information processing device, a retaining step of retaining the first trie structure generated based on the uniquely identifiable search key; a search step of searching for an attribute value corresponding to the search key from the search key by referring to the first trie structure stored in the storage; the first trie structure has each node associated with the attribute value; An information processing method characterized in that, in the search step, a search is performed on the first trie structure based on the search key, and the attribute value associated with the deepest reached node is set as a search result.

11. The computer of the information processing device a retaining step of retaining a first trie structure generated based on a uniquely identifiable search key; a search step of searching for an attribute value corresponding to the search key from the search key by referring to the first trie structure stored in the storage; the first trie structure has each node associated with the attribute value; In the search step, a search is performed on the first trie structure based on the search key, and the attribute value associated with the deepest reached node is set as a search result.

Images