Data management method and multimode database

Abstract

A data management method and a multimode database, the method comprising: receiving an object creation request initiated by a service requester, the object creation request being used for requesting to create a target logic object defined by a preset target identifier; according to table structure description information defined for the target identifier in advance, a target data table is created, index information used for supporting mixed search of data in the target data table is configured, the table name of the target data table is determined based on the object name of the target logic object, and the target data table is used for managing service data of the service requester.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of computer technology, and in particular relate to a data management method and a multi-model database. Background Technology

[0002] A database that supports multiple data models can be called a multi-model database. Because multi-model databases natively support various data models such as scalars, unstructured text, dense vectors, sparse vectors, and semi-structured documents, they have become one of the key solutions for artificial intelligence (AI) applications.

[0003] Compared to single-model databases such as ElasticSearch and Milvus, multi-model databases typically require more complex table structures, such as multiple fields involving various data models, and index information to support mixed searches of data in the tables. This requires relevant developers, such as AI application developers, to have a complete understanding of the table structure design rules of multi-model databases. Summary of the Invention

[0004] The purpose of this invention is to provide a data management method and a multi-model database.

[0005] Firstly, a data management method is provided for use in a multi-model database. The method includes: receiving an object creation request initiated by a business requester, the object creation request being used to request the creation of a target logical object defined by a preset target identifier; creating a target data table based on table structure description information predefined for the target identifier; and configuring index information to support mixed searches of data in the target data table, wherein the table name of the target data table is determined based on the object name of the target logical object, and the target data table is used to manage the business data of the business requester.

[0006] Secondly, a multi-model database is provided, comprising: a request receiving unit configured to receive an object creation request initiated by a business requester, the object creation request being used to request the creation of a target logical object defined by a preset target identifier; and a data table creation unit configured to create a target data table based on table structure description information predefined for the target identifier, and to configure index information for supporting mixed searches of data in the target data table, the table name of the target data table being determined based on the object name of the target logical object, and the target data table being used to manage the business data of the business requester.

[0007] Thirdly, a computing device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0008] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, wherein when the computer program is executed in a computing device, the computing device performs the method described in the first aspect.

[0009] In the technical solutions provided in the embodiments of this specification, developers do not need to learn or pay much attention to the table structure design rules of multi-model databases. Users of multi-model databases can trigger the multi-model database to create data tables and related indexes as needed without constructing SQL statements, which greatly reduces the threshold for using multi-model databases, is more friendly to relevant developers, and has better ease of use. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is one of the flowcharts for a data management method provided in the embodiments of this specification;

[0012] Figure 2 This is an example illustration of creating a data table in a multi-model database.

[0013] Figure 3 This is a second flowchart of a data management method provided in the embodiments of this specification;

[0014] Figure 4 This is the third flowchart of a data management method provided in the embodiments of this specification;

[0015] Figure 5 This is a schematic diagram of the structure of a multi-model database provided in the embodiments of this specification. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0017] As mentioned earlier, for users of multi-model databases, such as various AI applications, it is usually required that the relevant developers fully understand the table structure design rules of multi-model databases. Users need to predefine the table structure of the multi-model database tables they will use to ensure that users can use the predefined table structure to construct Structured Query Language (SQL) statements as needed. This allows the multi-model database to execute SQL statements to perform related transactions, such as creating data tables, updating data in data tables, and querying data in data tables. Therefore, the threshold for using multi-model databases is relatively high.

[0018] The applicant's research revealed that for users of multimodal data, such as most AI applications, the design of data table structures often exhibits a high degree of similarity. For example, AI applications where the business data is primarily unstructured text typically require data tables to have a primary key field for storing the primary key, a text content field for storing the text content, and an embedding vector field for storing the dense vectors / embedded vectors corresponding to the text content. Similarly, AI applications where the business data is primarily unstructured images typically require data tables to have a primary key field for storing the primary key, an address information field for storing the image's address information, a text content field for storing the text content (e.g., text content extracted from images through optical character recognition), and an embedding vector field for storing the dense vectors / embedded vectors corresponding to the text content, and so on.

[0019] Based on the above issues and related findings, the applicant proposes adding at least one logical object, distinct from data tables, defined by at least one pre-defined target identifier, to the multi-model database. Simultaneously, the applicant pre-defines table structure description information corresponding to each of the at least one target identifier within the multi-model database. When a user of the multi-model database needs to create a data table, they can directly request the database to create a logical object defined by a specific target identifier, triggering the database to create the data table and related index information based on the pre-defined table structure description information for that target identifier.

[0020] In this way, developers do not need to learn or pay too much attention to the table structure design rules of multi-model databases. Users of multi-model databases can trigger the multi-model database to create data tables and related indexes as needed without constructing SQL statements, which greatly reduces the threshold for using multi-model databases, making them more developer-friendly and more user-friendly.

[0021] For ease of description, the following text will use “Collection” as an example.

[0022] To further improve the usability of multi-model databases, additional Data Definition Language (DDL) statements related to Collections can be added and executed by the relevant storage engine. For example, the following can be added: "createcollection XX" for creating logical objects defined by a Collection, "drop collectionXX" for deleting logical objects defined by a Collection, and "show collections" for displaying the names of all logical objects created by the user and defined by a Collection, etc. In the aforementioned DDL statements, the placeholder "XX" represents the object name of the logical object.

[0023] The logical objects defined through Collections and their associated DDL statements do not hide the definition and implementation of data tables in the multi-model database. This means that the multi-model database still supports direct manipulation of data tables through SQL statements related to the tables. In this way, while maintaining the ease of use of the multi-model database, it does not affect its other inherent functionalities.

[0024] Correspondingly, multi-model databases can provide users and relevant developers with development interfaces of both software development kit (SDK) and SQL difficulty.

[0025] For beginners, the SDK-level development interface can be used. There is no need to understand the table structure design rules of multi-model databases or learn SQL statements. This allows them to efficiently build multi-model database applications such as most AI applications.

[0026] More specifically, the client SDK for the multi-model database provides standard interfaces related to Collection, including object creation, data storage, and data query interfaces. The object creation interface allows users to initiate object creation requests to the multi-model database, requesting the database to create the target logical object defined by the Collection. The data storage interface allows users to initiate data storage requests for this target logical object. The data query interface allows users to initiate data query requests for this target logical object. It should be noted that the aforementioned object creation, data storage, and data query requests can be DDL statements or equivalent protocol content.

[0027] For developers with strong technical skills, namely those familiar with multi-model databases and SQL statements, they can also develop their own user-friendly SQL statement generation capabilities. The SQL statements mentioned here include DDL statements related to Collections as well as other SQL statements that can directly manipulate data tables, supporting flexible operation and definition of table structures according to actual business needs.

[0028] The following section first provides an example of how to create a target data table in a multi-model database.

[0029] Figure 1 This is one of the flowcharts for a data management method provided in the embodiments of this specification. The method can be executed by a multi-model database, and more specifically, by the storage engine of the multi-model database.

[0030] Reference Figure 1 As shown, the method may include, but is not limited to, the following steps S101 and S103.

[0031] Step S101: Receive an object creation request initiated by the business requester. The object creation request is used to request the creation of a target logical object defined by a preset target identifier.

[0032] The business requester mentioned here is the user of the multi-model database, such as the server of an AI application.

[0033] In one possible implementation, the business requester can generate the object creation request through the object creation interface provided by the client SDK corresponding to the multi-model database. This object creation request can be a DDL statement or equivalent protocol content.

[0034] For example, refer to Figure 2As shown, the source code written by the developer for the business request side may include the code statement "collection = client.get_or_create_collection(name="my_collection")". The business logic implemented by this code statement is to call the object creation interface "get_or_create_collection()" provided by the client SDK to query whether a Collection named my_collection exists in the multi-model database. If it exists, it returns it; otherwise, it generates an object creation request to request the multi-model database to create a logical object named "my_collection" defined by the Collection. For example, this object creation request can be the Collection-related DDL statement "create collection my_collection".

[0035] It needs to be understood. Figure 2 The example source code is only used to assist in describing the technical solutions provided in the embodiments of this specification. The content or technical implementation logic that the source code may contain is essentially pseudocode used to describe the source code, for example, through... Figure 2 The object name my_collection and its other parameter values ​​in the source code should actually be replaced with the parameter names in the source code.

[0036] When the developers of the business requesting party have relatively strong technical capabilities, they may not need to generate object creation requests by calling the client SDK corresponding to the multi-model database. Instead, they can generate object creation requests directly from the source code written by the developers, such as the DDL statement "create collection my_collection" in the previous example.

[0037] As mentioned earlier, an object creation request can be a DDL statement or an equivalent protocol. For example, an object creation request can be protocol content that organizes the target identifier "Collection", operation type "create", and object name "my_collection" according to preset data organization rules. Furthermore, it should be noted that in some technical scenarios, it may not be necessary to specify the object name of the target logical object to be created; instead, the multi-model database may assign the object name to the target logical object.

[0038] Step S103: Based on the table structure description information predefined for the target identifier, create a target data table and configure index information to support mixed searches of data in the target data table. The table name of the target data table is determined based on the object name of the target logical object. The target data table is used to manage the business data of the business requester.

[0039] The format of the business data described herein may include, but is not limited to, unstructured text or unstructured images.

[0040] The table structure description information described here may describe one or more of the following key information: table name generation logic, field definitions, constraint information, and index definitions. Field definitions include one or more of the following: field names, data types, default values, whether NULL values ​​are allowed, and field comments for each field in the data table. Constraint information includes at least primary key constraints (i.e., specifying the field name as the primary key), and may also include other constraints such as foreign key constraints. Index definitions may include the primary key index and field-level indexes corresponding to one or more other fields. Furthermore, the table structure description information may also include other optional information such as partitioning strategies (e.g., dividing the table into a preset number of partitions based on the hash value of a field value under a specified partition key) and replication configuration (e.g., specifying 3 replicas per partition).

[0041] As mentioned earlier, a multi-model database can add at least one logical object, different from the data table, defined by at least one preset target identifier. In specific technical scenarios, different table structure description information can be predefined for each target identifier.

[0042] Continuing with the previous example, for "create collection my_collection", the target data table may include a primary key field (different from the text content field and the embedded vector field), a text content field, and an embedded vector field. The index information of the target data table may include the full-text index corresponding to the text content field and the embedded vector index corresponding to the embedded vector field. The full-text index supports multiple matching modes such as keywords, phrases, and Boolean expressions, and can use ranking algorithms including BM25 to retrieve the text content under the text content field. The embedded vector index supports various types of vector distance calculations, such as Manhattan distance, Euclidean distance, inner product distance, and pre-distance, and retrieves the embedded vectors under the embedded vector field based on the vector distance.

[0043] The target data table may also include metadata fields for storing document data containing one or more key-value pairs; the indexing information of the target data table also includes the generalized inverted (GIN) index corresponding to the metadata fields.

[0044] Continuing with the previous example, for the object creation request "create collection my_collection", the target data table corresponding to this object creation request in a multi-model database can have the table structure described by the following pseudocode:

[0045] / / * Table name generation logic, c$version number (default initial version is v1)$+object name

[0046] create table c$v1$ my_collection(

[0047] / / Field name _id; data type is a 512-byte string; _id is the primary key (i.e., _id is the primary key field); field values ​​under _id are not allowed to be null; the default value is the string obtained by converting the next value in the data sequence.

[0048] _id varbinary(512) PRIMARY KEY NOT NULL to_char(seq.next),

[0049] / / * Field name is document; data type is string

[0050] document string,

[0051] / / * Field name: embedding (i.e., embedding vector field); data type: vector(?), placeholder "?" represents vector dimension, its value is determined by the embedding model default_model; if no field value is directly provided for the embedding field when inserting records into the data table, the embedding function ai_embed() calls the embedding model default_model to perform embedding calculations on the text content under the document field (i.e., text content field), and the obtained embedding vector is used as the default value.

[0052] embedding vector(?) DEFAULT ai_embed(default_model, document),

[0053] / / * Field name: metadata (i.e., metadata field); data type: json

[0054] metadata json,

[0055] FULLTEXT INDEX idx1(document), / / Create a full-text index for the document field.

[0056] VECTOR INDEX idx2 (embedding), / / Create an embedded vector index for the embedding field.

[0057] GIN INDEX idx3 (metadata), / / Create a GIN index for the metadata field.

[0058] ).

[0059] It is understood that the table structure of the target data table in the aforementioned example is merely illustrative and does not constitute a limitation of this application. For example, if the data type of the metadata field is JSON, the index information of the target data table may also include the search index corresponding to the metadata field. Here, the GIN index is typically used to support exact match queries for key-value pairs, while the search index is typically used to support querying keys containing specific keywords and their corresponding values ​​within a JSON document.

[0060] After receiving an object creation request, the multi-model database can, based on predefined table structure description information, sequentially generate and execute SQL statements for creating the target data table, and SQL statements for creating / configuring the corresponding index information for the target data table, ultimately completing the creation of the target data table and the configuration of the corresponding index information. Alternatively, the multi-model database may not generate SQL statements for the object creation request, but instead use a dedicated interface to directly create the target data table and configure the corresponding index information for the target data table based on predefined table structure description information for the target identifier.

[0061] For developers with relatively strong technical skills, constraint information can be added as needed in the object creation request. Multi-model databases can comprehensively consider constraint information and pre-defined table structure descriptions for the target identifier to create the target data table and related index information. In this way, the table structure of the target data table can be dynamically adjusted according to the specific needs of different application scenarios through the constraint information carried in the object creation request, achieving advanced customization while maintaining the usability of multi-model databases.

[0062] The aforementioned constraint information may include, but is not limited to, sparse vector enabling information, field replacement information, and partition configuration information.

[0063] Full-text indexing typically employs the BM25 algorithm and inverted indexes such as those used in Elasticsearch. By converting text content into sparse vectors using SPLADE, ColBERT, or other embedding models, and combining the corresponding sparse vector indexes with dense / embedded vector indexes, a hybrid search can be performed on the data in the target table. Compared to a hybrid search using only full-text and embedded vector indexes, this approach offers better similarity recall. Therefore, when a business requester chooses to use sparse vector indexes instead of full-text indexes, they can include sparse vector enabling information in the object creation request. The corresponding target table in the multi-model database can then include sparse vector fields. Correspondingly, the indexing information of the target table can include the sparse vector indexes corresponding to the sparse vector fields but not the full-text indexes corresponding to the content fields.

[0064] For example, the source code written by the developer for the business request side may include the following code statement: "collection = client.create_collection( name="my_collection", use_sparse_vector=True". The business logic implemented by this code statement is to call the object creation interface "client.get_or_create_collection()" provided by the client SDK to query whether a Collection named my_collection exists in the multi-model database. If it exists, it returns it; otherwise, it generates an object creation request to request the multi-model database to create a logical object named "my_collection" defined by the Collection. In addition to specifying the object name "my_collection", the object creation request also carries the sparse vector enabling information "use_sparse_vector=True", indicating that the target data table needs to include the sparse vector field "sparse_vector".

[0065] Some text content or images may be relatively large in size, making it unsuitable to store them entirely in the target data table due to storage cost or performance considerations. The specific text content / images might be stored in object storage such as S3 or OSS. The multi-model database can store only the address information of the text content / images without directly storing the text content / images. In this case, the business requester can include field replacement information in the object creation request through appropriate parameter options. The target data table created in the multi-model database can then include an address information field. Alternatively, the target data table can exclude the aforementioned text content field; this address information field is used to store the address information of the relevant text content or image in other storage systems such as S3 or OSS.

[0066] For example, the source code written by the developer for the business requester might include the following code statement: "collection = client.create_collection( name="my_collection", store_document_externally=True". The business logic implemented by this code statement is to call the object creation interface "client.get_or_create_collection()" provided by the client SDK to query whether a Collection named my_collection exists in the multi-model database. If it exists, it returns it; otherwise, it generates a new object creation request to request the multi-model database to create a logical object named "my_collection" defined by the Collection. In addition to specifying the object name "my_collection", the object creation request also carries the field replacement information "store_document_externally=True", indicating that the target data table needs to include the address information field "store_document_externally". Optionally, the target data table does not directly include text content fields.

[0067] In large-scale distributed scenarios, such as those with data volumes reaching terabyte levels or involving high-concurrency writes, more granular data sharding strategies are often required. In such cases, the business requester can include partitioning configuration information in the object creation request via appropriate parameter options. Correspondingly, when creating a target data table, a multi-model database can configure multiple partitions corresponding to the target data table based on the partitioning configuration information. For example, the partitioning configuration information may include the aforementioned partitioning strategy.

[0068] The following example describes the process of updating business data in a target data table that has already been created.

[0069] Figure 3 This is a second flowchart of a data management method provided in the embodiments of this specification. This method can be executed by a multi-model database, and more specifically, by the storage engine of the multi-model database.

[0070] Reference Figure 3 As shown, the method may include, but is not limited to, some or all of the following steps S301 to S309.

[0071] Step S301: Receive a data storage request initiated by the business requester for the target logical object. The data storage request includes at least the first business data corresponding to the first target field in the target data table.

[0072] The first business data mentioned here can typically be text content, or it can be text content or image address information. That is, the first target field can typically be a text content field or an address information field.

[0073] In some embodiments, the data storage request may also include a first primary key corresponding to the first business data.

[0074] For example, refer to Figure 2 As shown, the source code written by the developer for the business request side may include code statements for calling the data storage interface "collection.upsert()" provided by the client SDK. These statements pass two primary business data items to `collection.upsert()` through corresponding assignment operations: "This is a document about pineapple" and "This is a document about oranges". They may also pass the primary keys "id1" and "id2" corresponding to these two primary business data items. Then, `collection.upsert()` sends a data storage request containing the two primary business data items and their primary keys to the multi-model database. Furthermore, this data storage request may also include, for example, the object name of the target logical object, such as "my_collection".

[0075] In some embodiments, the data storage request may also include metadata corresponding to the first business data.

[0076] Step S303: When the data storage request includes the first primary key corresponding to the first business data, determine whether there is a field value with the same as the first primary key under the primary key field in the target data table.

[0077] Step S305: If the data storage request does not include the first primary key corresponding to the first business data, generate the first primary key corresponding to the first business data.

[0078] Referring to the previous text, the first primary key can be a string obtained by converting the next value in a data sequence.

[0079] Step S307: Update the first record in the target data table with the first primary key as the primary key.

[0080] When it is determined through step S303 that there is a field value in the primary key field of the target data table that is the same as the first primary key, the update described here is to update the first record already stored in the target data table with the first primary key as the primary key, for example, to replace the field value in the first record corresponding to the first target field with the first business data.

[0081] When it is determined through step S303 that there is no field value under the primary key field in the target data table that is the same as the first primary key, or when the first primary key corresponding to the first business data is generated through step S305, the update described here is to add a new first record to the target data table. The field value under the primary key field in the first record is the first primary key, and the field value under the first target field is the first business data. The content under other fields can be determined based on the first business data or other preset rules.

[0082] Step S309: Update the index information according to the updated first record.

[0083] Continuing with the previous example, for instance, the full-text index and the embedded vector index of the target data table can be updated accordingly.

[0084] The following example describes the process of querying business data in a target data table that has already been created.

[0085] Figure 4 This is the third flowchart of a data management method provided in the embodiments of this specification. This method can be executed by a multi-model database, and more specifically, by the storage engine of the multi-model database.

[0086] Reference Figure 4 As shown, the method may include, but is not limited to, the following steps S401 and S403.

[0087] Step S401: Receive a data query request initiated by the business requester for the target logical object. The data query request includes the second business data corresponding to the second target field in the target data table.

[0088] The second business data mentioned here can typically be text content or an image.

[0089] In some embodiments, the data query request may also include a second primary key corresponding to the second business data.

[0090] For example, refer to Figure 2As shown, the source code written by the developer for the business request side may include code statements for calling the data query interface "collection.query()" provided by the SDK. These code statements may pass second business data, such as "This is a query document about Florida," to `collection.query()` through corresponding assignment operations. Additionally, they may pass the recall count of similar data, i.e., the value of `n_results`, to `collection.query()`. Then, `collection.query()` sends a data query request to the multi-model database containing the aforementioned second business data and the recall count.

[0091] In addition, the data request may include, for example, the object name of the target logical object, such as "my_collection".

[0092] Data query requests can also include other filtering information. As mentioned earlier, the document data under the metadata field may contain one or more key-value pairs, and the data query request can also include filtering conditions for these key-value pairs.

[0093] Step S403: Perform a mixed search on the data in the target data table based on the index information of the target data table to obtain target business data similar to the second business data, and return the target business data to the business requester.

[0094] Multi-model databases can execute multi-path recall strategies related to secondary business data based on the index information of the target data table.

[0095] Continuing with the previous example, the multi-model database can convert the second business data "This is a query document about Florida" into embedding vectors and / or sparse vectors. Next, based on the second business data and the full-text index / sparse vector index, the multi-model database can query multiple pieces of business data similar to the second business data from the target data table; and based on the embedding vector corresponding to the second business data and the embedding vector index, it can query multiple embedding vectors similar to the embedding vector of the second business data from the target data table; finally, based on the multiple pieces of business data and multiple embedding vectors obtained from the query, and the recall count, it can determine one or more target business data similar to the second business data from the target data table and return them to the business requester.

[0096] It is understood that the foregoing examples are only used to assist in describing the technical solutions of this application and do not constitute a limitation on this application.

[0097] As the scale of business data from business requesters grows, there is a need to provide a smooth way to improve the system performance and cost-effectiveness of multi-model databases. Therefore, product features that optimize the system performance of multi-model databases through strong typing can be implemented as needed.

[0098] In one possible implementation, a multi-model database can automatically extract structured information from metadata fields and optimize storage space using this structured information. For example, among multiple key-value pairs (i.e., key-value pairs) included in the same document data under the metadata field, some keys may have a common prefix. The multi-model database can compress and store keys with common prefixes using appropriate tree structures or other methods, such as using relatively short identifiers in the document data to represent the larger common prefixes. As another example, multiple documents under the metadata field may include multiple identical common keys. By sorting these multiple common keys, instead of storing them as key-value pairs, only the values ​​of the multiple common keys in a portion of the document data need to be stored sequentially.

[0099] In another possible implementation, the requesting party can explicitly expand the common public keys included in different document data under the metadata field into fields in the target data table. More specifically, the requesting party can initiate a field expansion request to the multi-model database for a target logical object. This field expansion request includes the object name of the target logical object and the name of the target field to be expanded. The name of the target field belongs to the common key included in the document data under the metadata field. Correspondingly, the multi-model database can add the target field to the target data table, add the corresponding inverted index to the target field, and update the GIN index corresponding to the metadata field and / or the document data under the metadata field.

[0100] In the various embodiments illustrated above, during the processes of creating a target data table, updating business data in the target data table, querying business data from the target business data table, and expanding fields in the target data table, the relevant requests initiated by the business requesting party to the multi-model database target logical objects defined in the multi-model database using target identifiers, rather than specific target data tables. The multi-model database needs to further determine the name of the target data table to be operated on based on the object name carried in the relevant request or the object name determined based on relevant rules before performing relevant operations on the target data table. This ensures a good user experience for the client SDK and avoids implementation differences caused by different users / business requesters using different high-level languages. Furthermore, if the table structure description information is optimized, it can be directly optimized in the multi-model database, avoiding compatibility issues caused by designing different client SDKs for different high-level languages.

[0101] However, based on the aforementioned embodiments, it is conceivable that in some embodiments, the predefined table structure description information for the target identifier may be directly defined in the client SDK rather than in the multi-model database. The multi-model database itself is unaware of the logical objects defined by the target identifier. The client SDK corresponding to the multi-model database may directly provide SQL statement generation capabilities. For example, the object creation interface provided by the client SDK may directly construct SQL statements that can be executed by the multi-model database to create the target data table and configure index information based on the relevant table structure description information. The multi-model database completes the creation of the target data table and configures the corresponding index information for the target data table by directly executing the SQL statement.

[0102] Based on the same concept as the aforementioned method embodiments, this specification also provides a multi-model database 500.

[0103] Reference Figure 5 As shown, the multi-model database 500 includes: a request receiving unit 501, configured to receive an object creation request initiated by a business requester, the object creation request being used to request the creation of a target logical object defined by a preset target identifier; and a data table creation unit 503, configured to create a target data table based on table structure description information predefined for the target identifier, and to configure index information for supporting mixed searches of data in the target data table, the table name of the target data table being determined based on the object name of the target logical object, and the target data table being used to manage the business data of the business requester.

[0104] This specification also provides a computer-readable storage medium storing a computer program / instruction, which, when executed in a computer, causes the computer to perform a data management method provided in the foregoing embodiments.

[0105] This specification also provides a computing device in the embodiments, including a memory and a processor. The memory stores computer programs / instructions, and when the processor executes the computer programs / instructions, it implements a data management method provided in the foregoing embodiments.

[0106] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0107] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0108] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0109] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0110] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0111] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0112] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0114] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0115] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0116] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0117] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0119] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0120] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A data management method, the method being applied to a multi-model database, the method comprising: Receive an object creation request initiated by a business requester, the object creation request being used to request the creation of a target logical object defined by a preset target identifier; Based on the table structure description information predefined for the target identifier, a target data table is created, and index information for supporting mixed searches of data in the target data table is configured. The name of the target data table is determined based on the object name of the target logical object. The target data table is used to manage the business data of the business requester.

2. The method according to claim 1, wherein the object creation request is a Data Definition Language (DDL) statement.

3. The method according to claim 1, wherein the object creation request is generated by the business requester through the object creation interface provided by the client software development kit (SDK) corresponding to the multi-model database.

4. The method according to claim 1, wherein the target data table includes a primary key field, a text content field, and an embedding vector field; the index information includes a full-text index corresponding to the text content field and an embedding vector index corresponding to the embedding vector field.

5. The method according to claim 4, wherein the target data table further includes a metadata field for storing document data containing at least one key-value pair; the index information further includes a general inverted GIN index and / or a search index corresponding to the metadata field.

6. The method according to claim 5, further comprising: Receive a field expansion request initiated by the business requester for the target logical object. The field expansion request includes the field name of the target field to be expanded. The field name belongs to the public key included in the document data under the meta information field. Add an inverted index corresponding to the target field to the index information, and update the GIN index, search index and / or document data under the metadata field accordingly.

7. The method according to claim 4, wherein when the object creation request includes sparse vector enabling information, the target data table further includes a sparse vector field, and the index information includes the sparse vector index corresponding to the sparse vector field but does not include the full-text index corresponding to the text content field.

8. The method according to claim 4, wherein when the object creation request includes field replacement information, the target data table includes an address information field but does not include the text content field, and the address information field is used to store the address information of the business data.

9. The method according to claim 1, further comprising: When the object creation request includes partition configuration information, multiple partitions corresponding to the target data table are configured according to the partition configuration information.

10. The method according to any one of claims 1-9, further comprising: Receive the data storage request initiated by the business requester for the target logical object, the data storage request including a first primary key and the first business data corresponding to the first target field in the target data table; According to the data storage request, update the first record in the target data table with the first primary key as the primary key; The index information is updated accordingly based on the updated first record.

11. The method according to any one of claims 1-9, further comprising: Receive a data query request initiated by the business requester for the target logical object, wherein the data query request includes second business data corresponding to the second target field in the target data table; A mixed search is performed on the data in the target data table based on the index information to obtain target business data similar to the second business data, and the target business data is returned to the business requester.

12. A multi-model database, comprising: The request receiving unit is configured to receive an object creation request initiated by a business requester, wherein the object creation request is used to request the creation of a target logical object defined by a preset target identifier; The data table creation unit is configured to create a target data table based on the table structure description information predefined for the target identifier, and to configure index information to support mixed searches of data in the target data table. The table name of the target data table is determined based on the object name of the target logical object. The target data table is used to manage the business data of the business requester.

13. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the method of any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computing device, the computing device performs the method of any one of claims 1-11.

Images