Management method and database device

Abstract

To provide a management method and a database device with which it is possible to suppress the amount of data written to a storage device.SOLUTION: In a first vector database stored in a storage device, a group of first information pieces respectively indicating a plurality of first vectors are recorded. The plurality of first vectors correspond to a plurality of nodes constituting a first directed graph. The management method includes generating a second vector database in which a group of third information pieces respectively indicating two or more second vectors are recorded in a volatile memory, while generating a second directed graph including two or more nodes corresponding to two or more second vectors. The management method includes updating or setting one information piece among the group of first information pieces and the group of third information pieces so as to combine the first directed graph and the second directed graph together, and storing the second vector database in the storage device.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present embodiment relates to a management method and a database device. [Background technology]

[0002] There are vector databases that manage data in vector format. Vector databases enable search operations such as approximate nearest neighbor search (ANNS). One example of an ANNS algorithm is DiskANN (Disk-based Approximate Nearest Neighbor search).

[0003] According to DiskANN, a group of vector data is stored in a storage device, and index information defining a directed graph corresponding to the group of vector data is stored in volatile memory. Then, the vector data closest to the query is searched for along the directed graph defined by the index information. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Suhas Jayaram Subramanya, Devvrit, Rohan Kadekodi, Ravishankar Krishaswamy, and Harsha Vardhan Simhadri, “DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node”, [online], November 2019, NeurIPS, [retrieved on 2022-12-11], retrieved from the Internet:<URL: https: / / suhasjs.github.io / files / diskann_neurips19.pdf> Summary of the Invention [Problem to be solved by the invention]

[0005] An object of one embodiment is to provide a management method and a database device that can reduce the amount of data written to a storage device. [Means for solving the problem]

[0006] According to one embodiment, the management method includes storing a first vector database in a storage device and generating a second vector database in volatile memory. The first vector database records a group of first information pieces each representing a plurality of first vectors. The plurality of first vectors correspond to a plurality of nodes constituting a first directed graph. Each first information piece included in the group of first information pieces includes one of the plurality of first vectors and a second information piece representing a vector corresponding to an adjacent node. Generating the second vector database in volatile memory involves generating a second directed graph including two or more nodes corresponding to two or more second vectors, while also generating a second vector database in volatile memory, in which a group of third information pieces each representing two or more second vectors is recorded. Each third information piece included in the group of third information pieces includes one of two or more second vectors and a fourth information piece representing a vector corresponding to an adjacent node. The management method further includes combining the first directed graph and the second directed graph by updating or setting at least one piece of information from the group of first pieces of information and the group of third pieces of information, and storing the second vector database in a storage device. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a database device according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the configuration of an SSD included in the database device according to the first embodiment. [Figure 3] FIG. 2 is a diagram showing an example of the configuration of each memory die according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of a block configuration according to the first embodiment. [Figure 5] FIG. 2 is a diagram for explaining an example of a directed graph corresponding to a group of vector data managed by a vector database according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of the configuration of a vector database according to the first embodiment. [Figure 7] FIG. 3 is a diagram for explaining a method for adding a new node according to the first embodiment. [Figure 8] 4 is a diagram for explaining operations performed by a processor according to the first embodiment from generating a partial database to storing the partial database in an SSD. FIG. [Figure 9] FIG. 10 is a diagram for explaining a search operation when a partial directed graph is added to the upstream side of a directed graph. [Figure 10] 10A and 10B are diagrams for explaining a search operation when a partial directed graph is added downstream of a directed graph; [Figure 11] 10A and 10B are diagrams for explaining a search operation when a partial directed graph is added to the body part of a directed graph. [Figure 12] 5 is a flowchart showing an example of the operation of the database device according to the first embodiment. [Figure 13] 10 is a flowchart showing an example of an operation when a new node is added to the database device according to the second embodiment. [Figure 14] FIG. 10 is a diagram for explaining a search operation according to the second embodiment. [Figure 15] FIG. 10 is a diagram showing an example of how to use an SSD in a database device according to a first modification. [Figure 16] FIG. 10 is a diagram showing an example of how to use an SSD in a database device according to a second modification. DETAILED DESCRIPTION OF THE INVENTION

[0008] A management method and a database device according to embodiments will be described in detail below with reference to the accompanying drawings, although the present invention is not limited to these embodiments.

[0009] (First embodiment) FIG. 1 is a diagram illustrating an example of a configuration of a database device according to the first embodiment.

[0010] 1, the database device 1 includes a processor 11, an interface 12, a solid state drive (SSD) 13, a dynamic random access memory (DRAM) 14, and a bus 15. The processor 11, the interface 12, the SSD 13, and the DRAM 14 are electrically connected to one another via the bus 15.

[0011] The interface 12 is a device for inputting and outputting information to and from the database device 1. The interface 12 includes an interface that can communicate with an external device via a network, an interface to which a storage device can be connected, an interface to which an input device such as a keyboard can be connected, and the like. The database device 1 can accept an input of a query received from the outside via the interface 12. The database device 1 can also accept an input of new vector data received from the outside via the interface 12. The database device 1 may also accept an input of data in any format, such as text data or image data, via the interface 12, and generate new vector data based on the input data.

[0012] The SSD 13 is a large-capacity nonvolatile memory device that functions as a storage device in the database device 1. The SSD 13 includes a nonvolatile memory such as a NAND flash memory as a storage device.

[0013] The DRAM 14 is a volatile memory that operates faster than a storage device. The DRAM 14 functions as a cache area, a buffer area, a work area, etc. Note that the volatile memory applicable to the database device 1 is not limited to a DRAM.

[0014] The processor 11 is a computing device capable of executing a computer program and realizes functions defined by the computer program. The processor 11 is, for example, a CPU (Central Processing Unit). Note that one or more processors 11 may be provided depending on the functions to be realized. In the database device 1, the processor 11 manages the vector database based on the management program MPG. The management of the vector database includes an operation of adding new vector data to the vector database and a search operation in response to a query input. The management program MPG is stored, for example, in the SSD 13 or in a device external to the database device 1. The processor 11 loads the management program MPG from the SSD 13 or a device external to the database device 1 into the DRAM 14 under an environment provided by the operating system. The processor 11 then executes the management program MPG loaded into the DRAM 14. Note that in the example shown in FIG. 1, the management program MPG is stored in the SSD 13.

[0015] FIG. 2 is a diagram illustrating an example of the configuration of the SSD 13 included in the database device 1 according to the first embodiment.

[0016] The SSD 13 includes a memory controller MC and a plurality of memory packages PG, each of which includes a plurality of memory dies MD of NAND flash memory.

[0017] The memory controller MC and multiple memory packages PG are connected via one or more channels. As an example, the SSD 13 includes (m+1) channels CH0 to CHm, with two memory packages PG commonly connected to each channel. For example, memory packages PG0a and PG0b are connected to channel CH0, memory packages PG1a and PG1b are connected to channel CH1, and memory packages PGma and PGmb are connected to channel CHm. Each memory package PG includes two memory dies MD.

[0018] The number of channels included in the SSD 13, the number of memory packages PG included in the SSD 13, the number of memory dies MD included in each memory package PG, and the wiring between the memory controller MC and each memory die MD are not limited to this example.

[0019] The memory controller MC may be configured as a system-on-a-chip (SoC). Alternatively, the memory controller MC may be configured by multiple chips. Some or all of the memory controller MC may also be configured as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0020] The memory controller MC stores data input via the bus 15 in one of the memory dies MD, and outputs data read from one of the memory dies MD via the bus 15.

[0021] The memory controller MC also provides a logical address space to the outside of the SSD 13 (for example, the processor 11). The memory controller MC uses predetermined management information to manage the correspondence between logical addresses, which indicate locations within the logical address space, and physical addresses, which indicate locations within the storage areas of the group of memory dies MD. The management information is referred to as, for example, a logical-physical conversion table or a look-up table. The physical addresses indicate the memory package PG, the memory die MD, the blocks BLK, the pages P, and locations within the pages P. The blocks BLK and the pages P will be described later. The processor 11 uses the logical addresses to specify the location to write or read. The memory controller MC identifies the location to write or read by converting the logical addresses into physical addresses.

[0022] FIG. 3 is a diagram showing an example of the configuration of each memory die MD according to the first embodiment.

[0023] The memory die MD includes an access circuit 201 and a memory cell array 202 .

[0024] The memory cell array 202 includes a plurality of blocks BLK. In the example shown in Fig. 3, the memory cell array 202 includes (n+1) blocks BLK0 to BLKn as the plurality of blocks BLK.

[0025] Each block BLK has a plurality of memory cell transistors, each of which is associated with a row and a column, and each of which is capable of storing data.

[0026] The access circuit 201 includes, for example, a signal processing circuit, a row decoder, a column decoder, a sense amplifier, a latch circuit, and a voltage generating circuit. The access circuit 201 executes access to the memory cell array 202 in response to a command received from the memory controller MC. The access includes writing data, reading data, and erasing data.

[0027] Data is erased in units of blocks BLK. In other words, all data stored in one block BLK is erased at once. Data is written and rewritten in units of pages P, which are smaller than blocks BLK.

[0028] FIG. 4 is a diagram illustrating an example of the configuration of a block BLK according to the first embodiment.

[0029] 4, the block BLK includes a plurality of pages P. In the example shown in FIG. 4, the block BLK includes (k+1) pages P0 to Pk as the plurality of pages P. The access circuit 201 can write data to and read data from the memory cell array 202 in units of pages P. In other words, a page P is a storage area in the SSD 13 as a unit for writing and reading data.

[0030] In the database device 1, a group of vector data constituting a directed graph is managed as a vector database DB. A directed graph is a graph formed by regarding each vector data included in a group of vector data as a node and connecting the nodes with directed edges.

[0031] FIG. 5 is a diagram illustrating an example of a directed graph GF corresponding to a group of vector data managed by the vector database DB according to the first embodiment.

[0032] Each vector data included in the group of vector data is regarded as a node ND. Information about each node ND (hereinafter referred to as node information) includes corresponding vector data and adjacent node information. The adjacent node information is information indicating adjacent nodes, more specifically, out-neighbor nodes.

[0033] For example, a certain node ND0 is connected to each of nodes ND1 and ND2. Nodes ND1 and ND2 are out-adjacent nodes of node ND0. Therefore, the node information for node ND0 includes, in addition to vector data VD0 considered to be node ND0, adjacent node information NBR0-0 indicating the connection with node ND1 and adjacent node information NBR0-1 indicating the connection with node ND2.

[0034] When describing a directed graph GF, the node ND located at the most upstream side of the directed graph is referred to as the head node, and the node ND located at the most downstream side of the directed graph is referred to as the tail node. The section in the directed graph from immediately after the head node to immediately before the tail node is referred to as the body. For example, node ND0 corresponds to the head node of the directed graph GF, and node NDz corresponds to the tail node of the directed graph GF. The section from immediately after node ND0 to immediately before node NDz corresponds to the body.

[0035] In the example shown in FIG. 5, the directed graph GF has a tree-like shape, and the out-degree of each node ND is 2. The shape of the directed graph GF is not limited to a tree. Furthermore, multiple nodes ND do not need to have the same out-degree. In the directed graph GF, most nodes ND have an out-degree of 1 or more, but there may be a node with an out-degree of 0.

[0036] The directed graph GF is used in a search operation. In the search operation, the processor 11 performs a calculation to search for a node ND that is closest to the query based on an arbitrary search algorithm according to the directed graph GF. Any algorithm, including greedy search and beam search, may be used as the calculation algorithm for the search. To briefly explain an example, the processor 11 sequentially switches search target nodes, which are candidates for the node ND that is closest to the query, among multiple nodes ND along the directed graph GF. Each time the search target node is switched, the processor 11 calculates the distance between each adjacent node of the search target node and the query. Then, the processor 11 sets the node that is closest to the query among one or more adjacent nodes of one or more search target nodes that are currently close to the query as the next new search target node. The processor 11 sequentially switches search target nodes along the directed graph GF until it reaches the node ND that is estimated to be closest to the query. This process of switching search target nodes along the directed graph GF is also called a "hop." In other words, the directed graph GF can be considered to represent a path along which hops are possible.

[0037] In this specification, distance is a measure of similarity between data (including vector data ND and a query). Mathematically, distance is, for example, Euclidean distance. The mathematical definition of distance is not limited to Euclidean distance. In addition, the index used to evaluate distance is not limited to Euclidean distance, and any index corresponding to distance can be used.

[0038] A technique (comparison example) to be compared with the first embodiment will now be described. According to the comparison example, a search operation is performed according to DiskANN. That is, index information in which adjacent node information of each node is recorded for all nodes constituting a directed graph is stored in DRAM, and a group of vector data is stored in a storage device. Hops are performed based on the index information.

[0039] However, the size of the index information increases in accordance with the size of the vector data group, and therefore, according to the comparative example, the required DRAM capacity also increases in accordance with the size of the vector data group.

[0040] In the first embodiment, the vector database DB records, for each node, node information including vector data VD and adjacent node information NBR related to the vector data VD for each vector data VD (in other words, for each node ND). Then, for each hop, node information related to the hop destination node ND is acquired from the vector database DB, and the vector data VD included in the acquired node information is used to calculate the distance. Then, the hop is performed using the adjacent node information NBR included in the acquired node information. This eliminates the need to store adjacent node information for all nodes ND in the DRAM 14, and the storage capacity required for the DRAM 14 is reduced compared to the comparative example.

[0041] 6 is a diagram illustrating an example of the configuration of the vector database DB according to the first embodiment. As shown in this diagram, the vector database DB has a configuration in which node information is recorded for each node ND. Each piece of node information is an information piece that is an element of the vector database DB.

[0042] All node information has a common structure. As a representative of all node information, the structure of the node information of a certain node NDi will be explained.

[0043] The node information of the node NDi includes the vector data VDi of the node NDi and adjacent node information NBR for all adjacent nodes of the node NDi. Here, the node information of the node NDi includes adjacent node information NBRi-0 and adjacent node information NBRi-1.

[0044] All node information constituting the vector database DB is stored individually in one page P. Since reading from the memory cell array 202 is performed in units of page P, each piece of node information is acquired by reading from the memory cell array 202 once. Therefore, it is possible to minimize the time required to acquire each piece of node information.

[0045] Furthermore, the adjacent node information NBR included in the node information indicates an adjacent node. Indicating an adjacent node means indicating the location where the node information of the adjacent node is stored, or information from which the location where the node information of the adjacent node is stored can be derived. For example, the adjacent node information NBR may be a logical address indicating the location where the node information of the adjacent node is stored.

[0046] Hereinafter, the node ND and the vector data VD are treated as synonymous. Because the vector database DB includes a group of adjacent node information NBR, which is information that defines the structure of the directed graph GF, the vector database DB and the directed graph GF are treated as synonymous.

[0047] Each node information includes adjacent node information NBR indicating an adjacent node. Therefore, when a new node ND is added to the vector database DB, it may be necessary to update any of the existing node information stored in the vector database DB. In other words, when a new node ND is added as an adjacent node of an existing node ND, it is necessary to update the adjacent node information NBR of the existing node ND.

[0048] In NAND flash memory used in SSDs and other devices, memory cell transistors wear out depending on the number of write and erase cycles performed, reducing the reliability of the data stored in those memory cell transistors. Therefore, it is necessary to minimize the amount of data written to NAND flash memory.

[0049] When a new node ND is added to the vector database DB, a new write to one page P occurs to update the node information of the existing node ND. More specifically, the node information containing the updated adjacent node information NBR is written to an empty page P. In other words, in addition to writing the node information for the node ND to be added, a write to update the existing node information is required.

[0050] According to the first embodiment, in order to suppress the ratio R of the amount of writes for updating existing node information to the amount of writes for the node information of the node ND to be added, the database device 1 is set up so that two or more nodes ND are added per addition.

[0051] When X nodes ND are added per addition, (X+1) pieces of node information are written. Therefore, the ratio R is (X+1) / X. For example, when one node ND is added per addition, the ratio R is 2. Therefore, when one node ND is repeatedly added per addition, the amount of data written to the SSD 13 and further to each memory die MD increases. In the first embodiment, by configuring two or more nodes ND to be added per addition, the ratio R can be suppressed to 1.5 or less. In other words, the amount of data written to each memory die MD can be suppressed compared to when one node ND is added per addition.

[0052] 7 is a diagram for explaining a method for adding a new node ND according to the first embodiment. Here, a case will be described in which a node ND5 corresponding to vector data VD5 and a node ND6 corresponding to vector data VD6 are added to the vector database DB that configures the directed graph GF shown in FIG.

[0053] The processor 11 generates one directed graph (referred to as a partial directed graph pGF) including the added vector data VD5 and vector data VD6 as nodes ND. That is, the processor 11 generates a database (referred to as a partial database pDB) corresponding to the partial directed graph pGF in the DRAM 14.

[0054] The partial database pDB includes node information for each vector data VD, similar to the vector database DB corresponding to the directed graph GF. A node corresponding to a vector data VD is connected to a node corresponding to another vector data VD by adjacent node information NBR included in the node information.

[0055] In other words, generating a partial database pDB includes generating a partial directed graph pGF corresponding to the group of vector data VD to be added and recording node information for each node ND included in the partial directed graph pGF.

[0056] 7, the partial directed graph pGF includes a node ND5 that is vector data VD5 and a node ND6 that is vector data VD6. The node ND5 is connected to the node ND6, and the node ND6 is an adjacent node of the node ND5.

[0057] In the partial database pDB, the node information for node ND5 includes vector data VD5, adjacent node information NBR5-0, and adjacent node information NBR5-1. The node information for node ND6 includes vector data VD6, adjacent node information NBR6-0, and adjacent node information NBR6-1. The adjacent node information NBR5-0 indicates node ND6.

[0058] The processor 11 combines the partial directed graph pGF, which now includes two or more nodes ND, into the directed graph GF. The processor 11 sets or updates the adjacent node information of the upstream node ND of the two nodes ND sandwiching the boundary between the partial directed graph pGF and the directed graph GF, so as to indicate the downstream node ND of the two nodes ND.

[0059] 7, the partial directed graph pGF is added between the node ND1 and the node ND2 of the directed graph GF. Therefore, a first boundary between the node ND0 and the node ND5 and a second boundary between the node ND6 and the node ND2 are generated as boundaries between the partial directed graph pGF and the directed graph GF.

[0060] With respect to the first boundary, of the two nodes ND0 and ND5 that sandwich the first boundary, node ND0 corresponds to the upstream node ND. The content of adjacent node information NBR0-1 included in the node information of node ND0 is updated to content that indicates node ND5. In FIG. 7, the updated adjacent node information NBR0-1 is represented as adjacent node information NBR0-1'. The processor 11 generates new node information for node ND0 that includes adjacent node information NBR0-1' in place of the adjacent node information NBR0-1, and writes the generated node information to an empty page P, thereby updating the adjacent node information NBR0-1.

[0061] Regarding the second boundary, of the two nodes ND6 and ND2 on either side of the second boundary, node ND6 corresponds to the upstream node ND. The processor 11 sets the content indicating node ND2 as adjacent node information NBR6-1 included in the node information of node ND6.

[0062] The position where the partial digraph pGF is added is not limited to the above example. For example, the partial digraph pGF may be added upstream of the digraph GF, i.e., upstream of node ND0. Alternatively, the partial digraph pGF may be added downstream of the digraph GF, i.e., downstream of node NDz, for example.

[0063] When a partial directed graph pGF is added to the upstream side of the directed graph GF, the processor 11 does not update the node information of an existing node ND included in the directed graph GF. The processor 11 sets adjacent node information NBR included in the node information of the node ND included in the partial directed graph pGF, of the two nodes ND on either side of the join boundary.

[0064] When a partial directed graph pGF is added downstream of a directed graph GF, the processor 11 updates the adjacent node information NBR included in the node information of the node ND included in the directed graph GF, out of the two nodes ND on either side of the join boundary. The processor 11 does not update the node information for the partial directed graph pGF.

[0065] Of the two nodes ND on either side of the boundary between the partial directed graph pGF and the directed graph GF, the upstream node ND is referred to as the upstream boundary node, and the downstream node ND is referred to as the downstream boundary node. Also, the operation of setting or updating the adjacent node information of the node information related to the upstream boundary node to indicate the downstream boundary node is referred to as the join operation.

[0066] After the join operation, the processor 11 stores the partial database pDB in the SSD 13 as is.

[0067] FIG. 8 is a diagram for explaining the operation performed by the processor 11 according to the first embodiment from the generation of the partial database pDB to the storage of the partial database pDB in the SSD 13. In FIG.

[0068] Part (A) of Figure 8 shows the state of each memory (DRAM 14 and SSD 13) while the partial database pDB is being generated. The SSD 13 stores a vector database DB1, which is the vector database DB before the partial database pDB (more specifically, a partial database pDB1 described later) is stored in the SSD 13. When new vector data VD is input, the processor 11 generates a partial database pDB1 in the DRAM 14, which includes a node ND corresponding to the input vector data VD. Then, every time new vector data VD is input, the processor 11 adds a new node ND to the partial database pDB1.

[0069] The processor 11 performs a join operation to join the partial database pDB1 to the vector database DB1. After the join operation, the processor 11 stores the partial database pDB1 in the SSD 13, as shown in part (B) of Fig. 8. The processor 11 manages the vector database DB2, which is the vector database DB1 plus the partial database pDB1, as a new vector database DB.

[0070] After storing the partial database pDB in the SSD 13, new vector data VD may be input to the processor 11. When the new vector data VD is input, the processor 11 generates a partial database pDB2 in the DRAM 14, which includes a node ND corresponding to the input vector data VD. Then, the processor 11 performs a join operation to join the partial database pDB2 to the vector database DB2, and stores the partial database pDB2 in the SSD 13.

[0071] In this way, the processor 11 repeats the operation of generating a partial database pDB including two or more nodes ND in the DRAM 14 and the operation of storing the partial database pDB in the SSD 13. By repeating these operations, the database device 1 can grow the vector database DB.

[0072] The processor 11 can perform a search operation even while the partial database pDB is being generated.

[0073] 9 to 11 are schematic diagrams for explaining a search operation according to the first embodiment. In FIGS. 9 to 11, arrows with diagonal hatching indicate the direction of search. The direction of search is the direction of hops. In other words, the direction of search is from the upstream side to the downstream side of the directed graph.

[0074] FIG. 9 is a diagram for explaining a search operation when a partial directed graph pGF is added to the upstream side of a directed graph GF.

[0075] At the start of generating the partial database pDB, the processor 11 executes a join operation to join the partial database pDB to the upstream side of the vector database DB, and then executes addition of a node ND to the partial database pDB while maintaining the join between the partial database pDB and the vector database DB.

[0076] Since the partial database pDB is kept connected upstream of the vector database DB, when a query is input, the processor 11 can hop in the order of the group of nodes ND included in the partial database pDB and the group of nodes ND included in the vector database DB.

[0077] FIG. 10 is a diagram for explaining a search operation when a partial directed graph pGF is added downstream of a directed graph GF.

[0078] At the start of generation of the partial database pDB, the processor 11 executes a join operation to join the partial database pDB to the downstream side of the vector database DB, and then executes addition of a node ND to the partial database pDB while maintaining the join between the partial database pDB and the vector database DB.

[0079] Therefore, when a query is input, the processor 11 can hop in the order of the group of nodes ND included in the vector database DB1 and the group of nodes ND included in the partial database pDB.

[0080] FIG. 11 is a diagram for explaining a search operation when a partial directed graph pGF is added to the body part of a directed graph GF.

[0081] At the start of generating the partial database pDB, the processor 11 executes a join operation to join the partial database pDB to the body part of the vector database DB1, and then executes addition of the node ND to the partial database pDB while maintaining the join between the partial database pDB and the vector database DB1.

[0082] Therefore, when a query is input, the processor 11 can hop in the following order: the group of nodes ND included in the partial database pDB, the group of nodes ND included in the vector database DB1, and the group of nodes ND included in the partial database pDB.

[0083] Next, the operation of the database device 1 according to the first embodiment will be described.

[0084] 12 is a flowchart showing an example of the operation of the database device 1 according to the first embodiment. Here, the operation will be described in a state where the vector database DB has already been stored in the SSD 13. The vector database DB is stored in the SSD 13 by, for example, the processor 11.

[0085] The processor 11 determines whether the destination of the new node is on the upstream side of the vector database DB (S101). The destination of the new node can be set, for example, by the user of the database device 1. Alternatively, the processor 11 may autonomously determine the destination of the new node by some method.

[0086] If the destination of the new node is the upstream side of the vector database DB (S101: Yes), the processor 11 starts generating a partial database pDB including the tail node connected to the head node of the vector database DB (S102). That is, the processor 11 starts generating the partial database pDB that is connected to the vector database DB. The processor 11 generates the partial database pDB in the DRAM 14.

[0087] If the destination of the new node is not on the upstream side of the vector database DB (S101: No), the processor 11 determines whether the destination of the new node is on the downstream side of the vector database DB (S103).

[0088] If the destination of the new node is downstream of the vector database DB (S103: Yes), the processor 11 updates the adjacent node information BNR of the end node of the vector database DB to indicate a new node ND (referred to as the first set node) (S104). The processor 11 generates new node information for the end node of the vector database DB including adjacent node information NBR indicating the first set node, and writes the newly generated node information to a free page P of the SSD 13.

[0089] The processor 11 starts generating a partial database pDB that includes the first setting node as the leading node (S105). Through S104 and S105, the processor 11 generates in the DRAM 14 the partial database pDB that is combined with the vector database DB.

[0090] If the destination of the new node is not downstream of the vector database DB (S103: No), that is, if the destination of the new node is the body part of the vector database DB, the processor 11 updates the adjacent node information NBR of the upstream boundary node of the vector database DB to indicate a certain new node ND (referred to as the second set node) (S106). The processor 11 generates new node information of the upstream boundary node including the adjacent node information NBR indicating the first set node, and writes the newly generated node information to a free page P of the SSD 13.

[0091] The processor 11 starts generating a partial database pDB that includes the second set node as a leading node and a trailing node connected to the downstream boundary node of the vector database DB (S107). Through S106 and S107, the processor 11 generates in the DRAM 14 the partial database pDB that is connected to the vector database DB.

[0092] After S102, S105, or S107, the processor 11 determines whether or not the timing has come to make the partial database pDB non-volatile (S108). The method for determining the timing to make the partial database pDB non-volatile is not limited to a specific method. The timing to make the partial database pDB non-volatile may be the timing when the size of the partial database pDB reaches a predetermined value. Alternatively, the timing to make the partial database pDB non-volatile may be the timing when a user of the database device 1 instructs non-volatilization. However, the timing to make the partial database pDB non-volatile is set to be after the number of nodes ND included in the partial database pDB becomes two or more.

[0093] If the timing to make the partial database pDB nonvolatile has not yet arrived (S108: No), the processor 11 waits for the timing to make the partial database pDB nonvolatile.

[0094] When the time has come to make the partial database pDB nonvolatile (S108: Yes), the processor 11 stores the partial database pDB in the DRAM 14 in the SSD 13 (S109). Then, control shifts to S101.

[0095] 6, one piece of node information is stored per page P. Therefore, in S109, the processor 11 stores each of the multiple pieces of node information included in the partial database pDB in a different page P.

[0096] As described above, according to the first embodiment, the processor 11 generates in the DRAM 14 a partial database pDB that includes two or more new nodes ND and is combined with the vector database DB in the SSD 13. Then, the processor 11 stores the partial database pDB in the SSD 13.

[0097] Therefore, the amount of data written to the SSD 13 can be reduced compared to when one node ND is added to the vector database DB at a time.

[0098] Note that merging the partial database pDB to the vector database DB includes updating or setting the node information of the upstream boundary node of two nodes ND adjacent to each other across the boundary between the partial directed graph pGF and the directed graph GF (see, for example, S102, S104-S105, and S106-S107 in Figures 7 and 12).

[0099] Also, in the first embodiment, when a query is input when the partial database pDB is stored in the DRAM 14, for example, as shown in Figures 9 to 11, the processor 11 searches for the node ND closest to the query using both the partial database pDB and the vector database DB.

[0100] Therefore, even before the partial database pDB is stored in the SSD 13, a search can be performed that includes the partial database pDB in the search range.

[0101] Furthermore, in the first embodiment, the processor 11 stores one piece of node information per page P. For example, when storing the partial database pDB in the SSD 13, the processor 11 also stores one piece of node information per page P for the partial database pDB.

[0102] Therefore, it is possible to reduce the time required to read node information during a search operation.

[0103] In the above example, one piece of node information is stored per page P. The number of pieces of node information stored per page P is not limited to one, as long as one piece of node information is not stored across two or more pages P. An integer number of pieces of node information may be stored per page P.

[0104] Furthermore, the database device 1 has been described as including an SSD 13 as a storage device. Storage devices applicable to the database device 1 are not limited to SSDs. The database device 1 may also include a magnetic disk device as a storage device. An example of a magnetic disk device is an HDD (Hard Disk Drive).

[0105] (Second embodiment) In the first embodiment, a partial database pDB that is combined with the vector database DB is generated in the DRAM 14. A partial database pDB that is not combined with the vector database DB may be generated in the DRAM 14, and then a combining operation may be performed to combine the partial database pDB with the vector database DB. In the second embodiment, an example will be described in which the combining operation is performed immediately before the partial database pDB is made non-volatile. Below, differences from the first embodiment will be described. The same matters as in the first embodiment will be explained briefly or omitted.

[0106] FIG. 13 is a flowchart showing an example of the operation of the database device 1 according to the second embodiment when a new node is added.

[0107] The processor 11 starts generating a partial database pDB (S201). In S201, the processor 11 generates a partial database pDB that is not connected to the vector database DB in the DRAM 14, and sequentially adds new nodes ND to the partial database pDB in the DRAM 14.

[0108] Then, the processor 11 determines whether or not the timing has come to make the partial database pDB nonvolatile (S202). The processor 11 determines whether or not the timing has come to make the partial database pDB nonvolatile, for example, by using the same method as in S108.

[0109] If the timing to make the partial database pDB nonvolatile has not yet arrived (S202: No), the processor 11 waits for the timing to make the partial database pDB nonvolatile.

[0110] If the time has come to make the partial database pDB nonvolatile (S202: Yes), the processor 11 determines whether the destination of the new node is on the upstream side of the vector database DB (S203).

[0111] If the destination of the new node is on the upstream side of the vector database DB (S203: Yes), the processor 11 sets the content indicating the head node of the vector database DB as the adjacent node information BNR of the end node of the partial database pDB (S204).

[0112] If the destination of the new node is not on the upstream side of the vector database DB (S203: No), the processor 11 determines whether the destination of the new node is on the downstream side of the vector database DB (S205).

[0113] If the destination of the new node is downstream of the vector database DB (S205: Yes), the processor 11 updates the adjacent node information BNR of the end node of the vector database DB to indicate the start node of the partial database pDB (S206). The processor 11 generates new node information for the end node of the vector database DB including adjacent node information NBR indicating the start node of the partial database pDB, and writes the newly generated node information to a free page P of the SSD 13.

[0114] If the destination of the new node is not downstream of the vector database DB (S205: No), that is, if the destination of the new node is the body part of the vector database DB, the processor 11 sets the adjacent node information BNR of the end node of the partial database pDB to indicate the downstream boundary node of the vector database DB (S207). Furthermore, the processor 11 updates the adjacent node information NBR of the upstream boundary node of the vector database DB to indicate the start node of the partial database pDB (S208). The processor 11 newly generates node information of the upstream boundary node including the adjacent node information NBR indicating the start node of the partial database pDB, and writes the newly generated node information to a free page P of the SSD 13.

[0115] After S204, S206, or S208, the processor 11 stores the partial database pDB in the DRAM 14 in the SSD 13 (S209). Then, control transitions to S201.

[0116] In this way, as long as the partial database pDB is stored in the SSD 13 in a state where it is combined with the vector database DB, the timing of the combining operation of combining the partial database pDB with the vector database DB is not limited to a specific timing. As shown in Fig. 13 , the combining operation may be performed immediately before the partial database pDB is stored in the SSD 13.

[0117] FIG. 14 is a diagram for explaining a search operation according to the second embodiment.

[0118] In the second embodiment, the partial database pDB is combined with the vector database DB immediately before being stored in the SSD 13. When a query is input before the partial database pDB is combined with the vector database DB, the processor 11 separately executes a first search operation in which the vector database DB in the SSD 13 is the search range, and a second search operation in which the partial database pDB in the DRAM 14 is the search range.

[0119] Then, the processor 11 identifies the vector data VD that is closest to the query among the vector data VD that is estimated to be closest to the query obtained by a search operation with the vector database DB in SSD13 as the search range and the vector data VD that is estimated to be closest to the query obtained by a search operation with the partial database pDB in DRAM14 as the vector data VD that is closest to the query among the vector data VD stored in the vector database DB and the partial database pDB.

[0120] In this way, before the join operation is performed, the processor 11 performs a first search operation based on the vector database DB and a second search operation based on the partial database pDB, and identifies the node ND closest to the query based on the results of the search based on the vector database DB and the results of the search based on the partial database pDB.

[0121] Therefore, similarly to the first embodiment, even before the partial database pDB is stored in the SSD 13, a search including the partial database pDB in the search range is possible.

[0122] [Variations] Below, several modifications that can be applied to the first and second embodiments will be described.

[0123] (Variation 1) 15 is a diagram showing an example of how to use the SSD 13 in the database device 1 according to Modification 1. Note that in this diagram, the memory package PG is omitted from illustration in order to simplify the drawing.

[0124] In the first modification, a plurality of mutually independent vector databases DB are stored in the SSD 13. Each of the plurality of vector databases DB corresponds to independent directional data. Each of the plurality of vector databases DB is an independent vector database that is not linked to any other vector database DB among the plurality of vector databases DB.

[0125] Each vector database DB is not distributed across two or more memory dies MD, but is stored in one memory die MD. In the example shown in Fig. 15, vector database DBa, vector database DBb, vector database DBc, and vector database DBd are each stored in a different memory die MD.

[0126] The memory controller MC can access multiple memory dies MD connected to different memory channels CH in parallel in time. Also, the memory controller MC can access multiple memory dies MD connected to the same channel CH in parallel in time by interleaving. In other words, the multiple memory dies MD included in the SSD 13 can be considered as elements that can be accessed in parallel.

[0127] In a search operation using a vector database DB, node information is acquired from the vector database DB for each hop. That is, node information is continuously read from one vector database DB. When a search operation is simultaneously performed for each of multiple vector database DBs, node information is continuously read from each of the multiple vector database DBs.

[0128] In Modification 1, the multiple vector databases DB are stored in separate memory dies MD. Therefore, the memory controller MC can read the node information from each of the multiple vector databases DB in parallel, thereby improving the efficiency of acquiring the node information.

[0129] (Variation 2) FIG. 16 is a diagram showing an example of a method of using the SSD 13 in the database device 1 according to the second modification.

[0130] In the second modification, a plurality of vector databases DB may be stored in the SSD 13. In part (A) of Fig. 16, as an example of a plurality of vector databases DB, a vector database DBr and a vector database DBs are stored in the SSD 13. The processor 11 separately executes a search operation in which the vector database DBr is the search range and a search operation in which the vector database DBs is the search range.

[0131] The processor 11 can execute a join operation to join the vector database DBr and the vector database DBs. In the example shown in part (B) of Fig. 16, the vector database DBr is joined upstream of the vector database DBs. Therefore, when a query is input after this join operation, the processor 11 identifies the node ND closest to the query by hopping in the order of the group of nodes ND included in the vector database DBr and the group of nodes ND included in the vector database DBs.

[0132] [Note] According to the first and second embodiments, the following additional aspects are provided.

[0133] (Appendix 1) storing a first vector database in a storage device in which a group of first information pieces indicating each of a plurality of first vectors is recorded, the plurality of first vectors corresponding to a plurality of nodes constituting a first directed graph, and each of the first information pieces included in the group of first information pieces includes one of the plurality of first vectors and a second information piece indicating a vector corresponding to an adjacent node; While generating a second directed graph including two or more nodes corresponding to two or more second vectors, a second vector database is generated in a volatile memory in which a group of third information pieces indicating each of the two or more second vectors is recorded, and each of the third information pieces included in the group of third information pieces includes one of the two or more second vectors and a fourth information piece indicating a vector corresponding to an adjacent node; Combining the first and second directed graphs by updating or setting at least one piece of information from the first set of pieces of information and the third set of pieces of information; storing the second vector database on the storage device; A management method is provided, including:

[0134] The vector database DB in SSD13 (vector database DB1 in the example of FIG. 8) is an example of a first vector database. Each piece of node information recorded in the vector database DB is an example of a first information piece. The vector data VD included in the node information recorded in the vector database DB is an example of a first vector. The directed graph GF corresponding to the vector database DB is an example of a first directed graph. The adjacent node information NBR included in each piece of node information recorded in the vector database DB is an example of a second information piece.

[0135] Furthermore, the partial database pDB (partial database pDB1 in the example of FIG. 8) generated in DRAM 14 is an example of a second vector database. Each piece of node information recorded in the partial database pDB is an example of a third information piece. The vector data VD included in the node information recorded in the partial database pDB is an example of a second vector. The partial directed graph pGF corresponding to the partial database pDB is an example of a second directed graph. The adjacent node information NBR included in each piece of node information recorded in the partial database pDB is an example of a fourth information piece.

[0136] (Appendix 2) Combining the first directed graph and the second directed graph includes updating or setting an information piece corresponding to an upstream node of two nodes adjacent to each other across a boundary between the first directed graph and the second directed graph, among the first group of information pieces and the third group of information pieces. A method of administration as described in Appendix 1 is provided.

[0137] (Appendix 3) generating the second vector database includes combining the first directed graph and the second directed graph; When a query is input while the second vector database is stored in the volatile memory, searching for a vector that is closest to the query based on the first vector database in the storage device and the second vector database in the volatile memory; further comprising: The administration method described in Appendix 1 or Appendix 2 is provided.

[0138] (Appendix 4) When a query is input while the second vector database is stored in the volatile memory, performing a lookup of a vector closest to the query based on the first vector database in the storage device and a lookup of a vector closest to the query based on the second vector database in the volatile memory; Identifying a vector that is closest to the query based on a lookup result based on the first vector database in the storage device and a lookup result based on the second vector database in the volatile memory; further comprising: The administration method described in Appendix 1 or Appendix 2 is provided.

[0139] (Appendix 5) the storage device comprises a plurality of unit storage areas, each of the plurality of unit storage areas corresponding to a unit of data write and read in the storage device; storing the first vector database in a storage device includes storing an integer number of first information pieces included in the group of first information pieces in each of the plurality of unit storage areas; storing the second vector database in the storage device includes storing an integer number of third information pieces included in the group of third information pieces in each of the plurality of unit storage areas; Management method described in Appendix 1.

[0140] (Appendix 6) a storage device storing a first vector database in which a group of first information pieces indicating each of a plurality of first vectors is recorded, the plurality of first vectors corresponding to a plurality of nodes constituting a first directed graph, and each of the first information pieces included in the group of first information pieces includes one of the plurality of first vectors and a second information piece indicating a vector corresponding to an adjacent node; a volatile memory; While generating a second directed graph including two or more nodes corresponding to two or more second vectors, a second vector database is generated in the volatile memory, in which a group of third information pieces indicating each of the two or more second vectors is recorded, and each of the third information pieces included in the group of third information pieces includes one of the two or more second vectors and a fourth information piece indicating a vector corresponding to an adjacent node; Combining the first and second directed graphs by updating or setting at least one piece of information from the first set of pieces of information and the third set of pieces of information; storing the second vector database on the storage device; a processor configured to execute A database device is provided, comprising:

[0141] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0142] 1 Database device, 11 Processor, 12 Interface, 13 SSD, 14 DRAM, 15 Bus, 201 Access circuit, 202 Memory cell array, DB, DB1, DB2, DBa, DBb, DBd, DBd, DBr, DBs Vector database, BNR Adjacent node information, GF Directed graph, pGF Partial directed graph, pDB Partial database, MPG Management program.

Claims

1. storing a first vector database in a storage device in which a group of first information pieces indicating each of a plurality of first vectors is recorded, the plurality of first vectors corresponding to a plurality of nodes constituting a first directed graph, and each first information piece included in the group of first information pieces includes one of the plurality of first vectors and a second information piece indicating a vector corresponding to an adjacent node; While generating a second directed graph including two or more nodes corresponding to two or more second vectors, a second vector database is generated in a volatile memory in which a group of third information pieces indicating each of the two or more second vectors is recorded, and each of the third information pieces included in the group of third information pieces includes one of the two or more second vectors and a fourth information piece indicating a vector corresponding to an adjacent node; Combining the first and second directed graphs by updating or setting at least one piece of information from the first set of pieces of information and the third set of pieces of information; storing the second vector database on the storage device; Management methods including.

2. Combining the first directed graph and the second directed graph includes updating or setting an information piece corresponding to an upstream node of two nodes adjacent to each other across a boundary between the first directed graph and the second directed graph, among the group of first information pieces and the group of third information pieces. The management method according to claim 1 .

3. generating the second vector database includes combining the first directed graph and the second directed graph; When a query is input while the second vector database is stored in the volatile memory, searching for a vector that is closest to the query based on the first vector database in the storage device and the second vector database in the volatile memory; further comprising: The management method according to claim 1 or 2.

4. When a query is input while the second vector database is stored in the volatile memory, performing a search for a vector closest to the query based on the first vector database in the storage device and a search for a vector closest to the query based on the second vector database in the volatile memory; Identifying a vector that is closest to the query based on a search result based on the first vector database in the storage device and a search result based on the second vector database in the volatile memory; further comprising: The management method according to claim 1 or 2.

5. the storage device comprises a plurality of unit storage areas, each of the plurality of unit storage areas corresponding to a unit of data write and read in the storage device; storing the first vector database in a storage device includes storing an integer number of first information pieces included in the group of first information pieces in each of the plurality of unit storage areas; storing the second vector database in the storage device includes storing an integer number of third information pieces included in the group of third information pieces in each of the plurality of unit storage areas; The management method according to claim 1 .

6. a storage device storing a first vector database in which a group of first information pieces indicating each of a plurality of first vectors is recorded, the plurality of first vectors corresponding to a plurality of nodes constituting a first directed graph, and each first information piece included in the group of first information pieces includes one of the plurality of first vectors and a second information piece indicating a vector corresponding to an adjacent node; a volatile memory; While generating a second directed graph including two or more nodes corresponding to two or more second vectors, a second vector database is generated in the volatile memory, in which a group of third information pieces indicating each of the two or more second vectors is recorded, and each of the third information pieces included in the group of third information pieces includes one of the two or more second vectors and a fourth information piece indicating a vector corresponding to an adjacent node; Combining the first and second directed graphs by updating or setting at least one piece of information from the first set of pieces of information and the third set of pieces of information; storing the second vector database on the storage device; a processor configured to execute A database device comprising:

Images