Multi-modal data storage method and device, equipment, medium and product

Abstract

The invention discloses a multi-modal data storage method, device, equipment, medium and product, which are applied to the technical field of storage, and the method comprises the following steps: obtaining to-be-written data, the to-be-written data comprising multi-modal data; writing the to-be-written data into target data storage, and synchronously writing target metadata corresponding to the to-be-written data into a metadata storage partition in distributed key value storage; generating a write log corresponding to the target metadata in the distributed key value storage; and by subscribing an index engine of the write log, updating index data corresponding to the to-be-written data in an index storage partition in the distributed key value storage based on the target metadata and the to-be-written data. In this way, complex processing caused by cross-system transaction coordination can be avoided, and synchronous updating of the index data and the metadata can be efficiently and reliably guaranteed.

Description

Technical Field

[0001] This invention relates to the field of storage technology, and in particular to a multimodal data storage method, apparatus, device, medium and product. Background Technology

[0002] In the field of multimodal data storage, existing technical solutions typically employ a hybrid architecture: data is stored in object storage or distributed file systems, metadata is stored in the object storage or distributed file system's own private metadata organization structure, or in a separate database; and indexing relies on external search engines or vector database systems. This disperses different data across different components, making cross-system transaction coordination complex and hindering the synchronous updating of index data and metadata.

[0003] It is evident that how to avoid the complex processing caused by cross-system transaction coordination, and how to efficiently and reliably ensure the synchronous update of index data and metadata, are problems that need to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a multimodal data storage method, apparatus, device, medium, and product that can avoid complex processing caused by cross-system transaction coordination and efficiently and reliably ensure the synchronous update of index data and metadata.

[0005] In a first aspect, the present invention provides a multimodal data storage method, comprising: acquiring data to be written, wherein the data to be written includes data of multiple modalities; writing the data to be written into a target data storage, and synchronously writing the target metadata corresponding to the data to be written into a metadata storage partition in a distributed key-value store; generating a write log corresponding to the target metadata in the distributed key-value store; and updating the index data corresponding to the data to be written in the index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the index engine of the write log.

[0006] Secondly, the present invention provides a multimodal data storage device, comprising: a data acquisition module for acquiring data to be written, wherein the data to be written includes data of multiple modalities; a data storage module for writing the data to be written to a target data storage and synchronously writing the target metadata corresponding to the data to be written to a metadata storage partition in a distributed key-value store; a log generation module for generating a write log corresponding to the target metadata in the distributed key-value store; and an index synchronization module for updating the index data corresponding to the data to be written in the index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the index engine of the write log.

[0007] Thirdly, the present invention provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the aforementioned multimodal data storage method.

[0008] Fourthly, the present invention provides a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the aforementioned multimodal data storage method.

[0009] Fifthly, the present invention provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned multimodal data storage method.

[0010] As can be seen from the above scheme, the present invention provides a multimodal data storage method, including: acquiring data to be written, wherein the data to be written includes data of multiple modalities; writing the data to be written into a target data storage, and synchronously writing the target metadata corresponding to the data to be written into the metadata storage partition of the distributed key-value storage; generating a write log corresponding to the target metadata in the distributed key-value storage; and updating the index data corresponding to the data to be written in the index storage partition of the distributed key-value storage based on the target metadata and the data to be written by subscribing to the index engine of the write log.

[0011] As can be seen, this invention writes the multimodal data to be written to the target data storage, and simultaneously writes the corresponding metadata to the metadata storage partition in the distributed key-value store, and generates a write log in the metadata storage partition. The index engine is a subscriber to the metadata write log, and synchronously updates the corresponding index data in the index storage partition in the distributed key-value store. In this way, a unified distributed key-value storage platform is provided for storing metadata and index data. The index does not need to rely on an external system, which can avoid the complex processing caused by cross-system transaction coordination, and efficiently and reliably ensure the synchronous update of index data and metadata.

[0012] Accordingly, the multimodal data storage device, equipment, medium and product provided by the present invention also have the above-mentioned technical effects. Attached Figure Description

[0013] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1A flowchart of a multimodal data storage method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a multimodal data storage system architecture provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a multimodal data storage device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an electronic device structure provided in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0016] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0017] With the development of artificial intelligence technology, multimodal large-scale models have made significant progress in fields such as natural language understanding, computer vision, and speech recognition. These models need to process massive amounts of heterogeneous data, including text, images, audio, and video, but existing data storage and management technologies face many challenges. Traditional data storage architectures are mainly divided into three categories: relational databases (such as MySQL and Oracle), NoSQL (Not Only SQL) databases (such as MongoDB and Cassandra), and distributed file / object systems (such as HDFS (Hadoop Distributed File System) and Ceph). Relational databases use structured tables to store data, which is suitable for processing structured queries, but has limited support for unstructured data; NoSQL databases can handle semi-structured data, but their indexing capabilities are weak, making it difficult to meet the growing demand for multimodal data retrieval; distributed file / object systems are good at storing large-scale files, but lack efficient metadata management and indexing mechanisms.

[0018] In the field of multimodal data processing, existing technical solutions typically employ a hybrid architecture: unstructured data is stored in object storage or distributed file systems; metadata is stored in the object storage or the distributed file system's own private metadata organization structure, or in a separate NoSQL database; and indexes rely on external search engines (such as Elasticsearch) or vector database systems. For example, an image retrieval system might store image files in S3, image tags in MySQL, and visual feature indexes in Elasticsearch. This architecture suffers from the following problems: 1. Data fragmentation: Data, metadata, and indexes are scattered across multiple systems, leading to complex processing flows and requiring multiple cross-system interactions. For example, modifying image tags requires simultaneous updates to MySQL and Elasticsearch, involving multi-system transaction coordination. 2. Performance bottlenecks: Cross-system synchronization and data migration between external indexing systems and storage systems incur significant overhead, especially in large-scale data scenarios. For example, the cross-system synchronization latency between Elasticsearch and S3 typically exceeds 200ms, failing to meet real-time retrieval requirements. 3. Difficulty in metadata management: The increasing demand for annotation and alignment information for multimodal data leads to an explosive growth in metadata, making it difficult for traditional databases to handle high-concurrency write and query demands. For example, the metadata volume of an image-text dataset may exceed 10 times that of the original data. 4. Low efficiency of multimodal collaborative retrieval: Due to the lack of a unified metadata and indexing foundation, data updates and multimodal retrieval require coordination across multiple systems, resulting in high response latency. For example, cross-modal text-image retrieval requires first querying the Elasticsearch keyword index and then calling the vector database for similarity calculation, with an overall latency exceeding 500ms. Existing solutions all suffer from the problem of coordinating heterogeneous data storage, metadata management, and cross-modal retrieval. Metadata and indexes are still scattered across different components, cross-system transaction coordination is complex, vector indexes rely on external systems (such as Milvus), cannot achieve atomic updates with metadata, and storage costs remain high.

[0019] This invention aims to address the following core issues in existing multimodal data storage and management technologies: Unified storage and management of heterogeneous data: Multimodal large-scale models need to process various types of data such as text, images, audio, and video, with significant differences in structure, scale, and access patterns. Designing a unified storage architecture that can efficiently store heterogeneous data while supporting multimodal querying and analysis is a pressing issue. In traditional hybrid architectures, data, metadata, and indexes are stored separately, leading to high costs for maintaining data consistency and low efficiency in cross-modal retrieval. Explosive growth of metadata: The amount of metadata, such as annotation and alignment information, in multimodal data far exceeds the original data itself. For example, in an image-text pair dataset, each image may correspond to multiple text descriptions, and information such as alignment scores and annotation sources needs to be recorded, resulting in a more than tenfold increase in metadata volume. Traditional distributed storage systems struggle to handle such large-scale metadata storage and high-concurrency access requirements; for example, HBase's write throughput is only in the thousands per second when processing millions of metadata entries. Low efficiency in multimodal collaborative retrieval: Existing solutions based on external indexing systems suffer from metadata and index redundancy, multiple accesses, and other issues, leading to low retrieval efficiency. For example, text-image joint retrieval requires filtering candidate sets through Elasticsearch first, then calling a vector database for similarity calculation, resulting in an overall latency exceeding 800ms, which cannot meet the needs of scenarios such as real-time recommendations. System scalability and performance issues: With the continuous growth of multimodal data scale, storage systems need to have good horizontal scalability. At the same time, the additional overhead of directory tree traversal and permission checks in traditional storage architectures limits the overall performance of the system. For example, the single-point bottleneck of HDFS's NameNode (i.e., metadata node) makes it difficult for the cluster size to exceed 10,000 nodes, and the file permission check latency exceeds 50ms.

[0020] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] Next, a multimodal data storage method provided by an embodiment of the present invention will be described in detail. Figure 1 This is a flowchart of a multimodal data storage method provided in an embodiment of the present invention. The multimodal data storage method includes the following steps.

[0022] Step S11: Obtain the data to be written, wherein the data to be written includes data in multiple modalities.

[0023] The data to be written can be data written to the storage system by a client carrying a large model, including add writes or update writes. Multiple modalities can include text, images, video, audio, etc. This invention implements multimodal data storage. Multimodal data refers to a data set containing two or more different types of information carriers, including but not limited to text, images, audio, video, and sensor data. These data have heterogeneous characteristics; for example, text is a sequence structure, images are pixel matrices, audio is a waveform signal, and video is a composite of image sequences and audio. Multimodal data is generated in large quantities in fields such as the Internet, healthcare, transportation, and industrial monitoring, and its fusion analysis can provide more comprehensive information than single-modal data. This invention provides a multimodal data storage solution that can meet the needs of large multimodal models, as well as traditional scenarios such as high-performance computing, graph computing, and big data that require storage and analysis of multimodal data.

[0024] Step S12: Write the data to be written to the target data storage, and synchronously write the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value storage.

[0025] The target data storage includes distributed object storage and distributed file storage. Writing the data to be written to the target data storage may include: if the target object to which the data to be written is to be written is the distributed object storage, then the data to be written is written to the distributed object storage; if the target object to be written is the distributed file storage, then the data to be written is written to the distributed file storage.

[0026] In addition, embodiments of the present invention can also obtain data read requests. If the data read request is a request corresponding to distributed object storage, then the corresponding data is read from the distributed object storage. If the data read request is a request corresponding to distributed file storage, then the corresponding data is read from the distributed file storage.

[0027] In other words, this embodiment supports both object storage and distributed file systems for data storage. Distributed file systems have standard file operation interfaces and support random read / write access to data. Distributed object storage features high scalability and low cost. An object storage system (Object Storage System, an object-oriented data storage architecture) organizes data as objects. Compared to file systems, object storage does not rely on a directory tree structure, supports large-scale data storage and efficient retrieval, and is suitable for storing unstructured data such as images and videos. A distributed file storage system is a file-level data storage management system that distributes file data across multiple independent nodes via a network and provides a unified namespace and access interface through a collaborative mechanism, possessing high scalability, high availability, and fault tolerance. Unstructured data refers to data without a predefined data model or organizational structure. Unstructured data in multimodal scenarios includes raw images, audio waveforms, video streams, etc.

[0028] Metadata, in this context, refers to data that describes other data. In multimodal scenarios, this includes data source, creation time, format, content description, annotation information, and alignment relationships between different modalities. Examples include image-text matching annotations and time synchronization information between video frames and audio clips. As the scale of multimodal data grows, the amount of metadata expands exponentially. This embodiment stores metadata using distributed key-value storage, a data storage system based on key-value pairs. The key serves as a unique identifier, and the value represents the associated data. Its distributed nature allows data to be distributed across multiple nodes, achieving load balancing and high availability through consistent hashing, sharding, and other technologies. Databases such as Redis Cluster or Apache Cassandra can be used, offering higher read / write performance and horizontal scalability to address the problem of exponential metadata growth. This embodiment allows writing metadata IDs and metadata itself when writing metadata.

[0029] Furthermore, synchronously writing the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value store can include: synchronously writing the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value store through a metadata engine. Here, the metadata engine is a component used to manage metadata. Metadata is written synchronously when writing data. For metadata management, file system tree-structured namespaces, object storage flat address spaces, and multimodal descriptive information, such as text embedding vectors and visual feature descriptions, can be stored.

[0030] Step S13: Generate the write log corresponding to the target metadata in the distributed key-value store.

[0031] This invention can generate write logs corresponding to target metadata and add them to a message queue. In an optional implementation, generating the write logs corresponding to the target metadata in the distributed key-value store can include: generating the write logs corresponding to the target metadata through the distributed key-value store and adding the write logs to a message queue in the distributed key-value store. Correspondingly, the write logs are read from the message queue by subscribing to the index engine of the write logs. That is, the distributed key-value store can generate the write logs corresponding to the target metadata and add them to the message queue; the distributed key-value store is the log producer, and the index engine is the log consumer. Using a unified distributed key-value store to store metadata and index data makes the coordination between the metadata engine and the index engine more convenient.

[0032] In another optional implementation, generating the write log corresponding to the target metadata in the distributed key-value store may include: after synchronously writing the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value store through the metadata engine, generating the write log corresponding to the target metadata, and adding the write log to the message queue in the distributed key-value store. That is, the metadata engine can also add the log to the message queue.

[0033] Step S14: By subscribing to the index engine of the write log, update the index data corresponding to the data to be written in the index storage partition of the distributed key-value store based on the target metadata and the data to be written.

[0034] In this embodiment, the indexing engine includes a keyword indexing engine and a vector indexing engine. Correspondingly, updating the index data corresponding to the data to be written in the index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the indexing engine of the write log includes: updating the keyword index data corresponding to the data to be written in the keyword index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the keyword indexing engine of the write log; and updating the vector index data corresponding to the data to be written in the vector index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the vector indexing engine of the write log.

[0035] The keyword index data may include keyword information and an index. The keyword information may include a keyword ID (identifier) ​​and a keyword. The index is generated based on the keyword information. The vector index data may include vector information and an index. The vector information may include a vector ID (identifier) ​​and a vector. The index is generated based on the vector information. The keyword index data may adopt an inverted index structure, and the vector index data may adopt a graph structure.

[0036] Furthermore, after the index engine updates the index data corresponding to the data to be written in the index storage partition of the distributed key-value store, the write log is deleted from the message queue. That is, in this embodiment of the invention, the write log can be deleted promptly, and the data, metadata, and index data are synchronously updated. The keyword index engine is a component built based on keyword indexes, and the vector index engine is a component built based on vector indexes.

[0037] In an optional implementation, deleting the write log from the message queue may include: detecting a log status identifier corresponding to the write log, wherein the log status identifier represents the consumption status of the write log; when the log status identifier represents that the write log is in a consumption completed state, the write log is deleted from the message queue. That is, embodiments of the present invention can determine whether the write log has been consumed by identifying a ten-day status identifier.

[0038] Furthermore, embodiments of the present invention may further include: after adding the write log to the message queue in the distributed key-value store, setting the log status identifier to a first preset value to indicate that the write log is in an unconsumed state; when the vector index engine subscribing to the write log begins to consume the write log, setting the log status identifier to a second preset value to indicate that the write log is in a consumed state; and when the index engine completes updating the index data, setting the log status identifier to a third preset value to indicate that the write log is in a consumed state. Using three different preset values ​​to mark the consumption state of the write log ensures the consistency of metadata and index data.

[0039] In addition, embodiments of the present invention can also incorporate a transaction mechanism of distributed key-value storage, where the transaction is completed when both metadata and index data are updated.

[0040] Furthermore, embodiments of the present invention may also include: responding to a retrieval request through a target index engine, wherein the target index engine is a keyword index engine or a vector index engine; sending a multimodal collaborative retrieval request to a metadata engine and / or a non-target index engine in the index engine through the distributed key-value storage; and preloading the metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine into a cache through the metadata engine and / or the non-target index engine, so as to respond to the query request corresponding to the associated metadata and the associated index data.

[0041] In other words, if the target indexing engine is a vector indexing engine, then the non-target indexing engine is a keyword indexing engine; if the target indexing engine is a keyword indexing engine, then the non-target indexing engine is a vector indexing engine. Metadata associated with the response data of the retrieval request and / or the index data of the non-target indexing engine can be pre-loaded into the cache based on pre-recorded associations. Because of this pre-loading, if further retrieval of data based on the non-target indexing engine or reading data based on metadata is required, there is no need to retrieve the corresponding index data or metadata again, thereby improving retrieval efficiency. The association can be a mapping relationship between keyword ID, vector ID, metadata ID, and the global ID of the data. For example, the mapping relationship between the global ID of a file and its keyword ID, vector ID, and metadata ID. For instance, first, vector indexing data is retrieved through the vector indexing engine; the associated metadata ID and keyword ID are determined through the vector ID and association relationship; then, the metadata engine is triggered to retrieve metadata based on the metadata ID, and the keyword engine is triggered to retrieve keywords based on the keyword ID. This data is then placed in the cache. If further queries targeting the file itself occur, the file is read based on the metadata in the cache; if further keyword retrieval is required, keyword retrieval can be performed based on the keyword index data in the cache.

[0042] In an optional implementation, preloading the metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine into the cache via the metadata engine and / or the non-target index engine may include: preloading the metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine into the cache based on the data access frequency via the metadata engine and / or the non-target index engine.

[0043] Here, data access frequency characterizes how frequently data is accessed. In this embodiment of the invention, a threshold can be set, and the metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine, whose data access frequency exceeds the preset threshold, can be pre-loaded into the cache. This ensures that data is likely to be retrieved, improving retrieval efficiency while avoiding unnecessary loading.

[0044] Embodiments of the present invention may further include: recording the data access frequency of data in the target data storage, metadata in the metadata storage partition, and index data in the index storage partition.

[0045] This invention can record the access time of data, metadata, and index data, as well as the data access frequency. In an optional implementation, a metadata engine can be used to record the access time of data, metadata, and index data, and the data access frequency.

[0046] Furthermore, embodiments of the present invention can also perform hierarchical storage of data in the target data storage, metadata in the metadata storage partition, and index data in the index storage partition based on the data access frequency.

[0047] In other words, embodiments of the present invention can perform data tiering, which involves classifying data according to access frequency, value, or performance requirements and storing it on different media (such as SSDs, HDDs, and magnetic tapes) to optimize storage costs and access efficiency. Typically, hot data is stored in a high-speed tier, and cold data in a low-cost tier.

[0048] In an optional implementation, based on the data access frequency, the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition are stored in a hierarchical manner, including: classifying the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition into cold data, warm data, and hot data based on the data access frequency, wherein the data access frequency of cold data is less than that of warm data, and the data access frequency of warm data is less than that of hot data; storing the cold data, warm data, and hot data in different levels of storage space. The response efficiency of the storage space corresponding to cold data, warm data, and hot data ranges from low to high.

[0049] As can be seen, this embodiment of the invention writes the multimodal data to be written to the target data storage, and simultaneously writes the corresponding metadata to the metadata storage partition in the distributed key-value store, and generates a write log in the metadata storage partition. The index engine is a subscriber to the metadata write log, and synchronously updates the corresponding index data in the index storage partition in the distributed key-value store. In this way, a unified distributed key-value storage platform is provided for storing metadata and index data. The index does not need to rely on an external system, which can avoid the complex processing caused by cross-system transaction coordination, and efficiently and reliably ensure the synchronous update of index data and metadata.

[0050] Further, see Figure 2 As shown in the diagram, this invention provides a schematic of a multimodal data storage system architecture. The proposed multimodal data storage system adopts a unified foundation, layered decoupling, and collaborative optimization architecture design. It constructs a unified metadata and index foundation through distributed key-value storage (KV), solving the problems of heterogeneity, metadata explosion, and multimodal retrieval efficiency in multimodal data storage. The system decomposes complex data management requirements into independent functional layers, each focusing on optimization in a specific domain, while achieving collaboration through a unified scheduling mechanism. It mainly includes the following components.

[0051] 1) Access Interface: Provides a unified access point for multimodal data, supporting the following interfaces: Data storage interface: Compatible with traditional protocols such as POSIX (Portable Operating System Interface), S3 (Simple Storage Service), and HDFS, reducing application migration costs; Metadata interface: Provides key-value operation interface, supporting batch writing and transactional updates; Index interface: Provides mixed query capabilities for keyword retrieval (such as Lucene syntax) and vector retrieval (such as cosine similarity).

[0052] 2) Data Storage: Supports both object storage and distributed file systems (such as HDFS). Distributed file storage: Features a standard file operation interface, supporting random read and write access to data. Distributed object storage: Offers high scalability and low cost.

[0053] 3) Distributed KV Foundation: Relying on lightweight interfaces, low latency, and distributed friendliness, it breaks through the bottleneck of traditional metadata storage. As the core of the entire system, it provides lightweight, high-performance key-value storage services, supporting horizontal scaling and high availability.

[0054] 4) Metadata and Indexing: A multimodal data management hub is built upon a distributed key-value (KV) architecture. Metadata Management: This includes a tree-structured namespace for the storage file system, a flat address space for object storage, and multimodal descriptive information (such as text embedding vectors and visual features). Keyword Indexing Engine: Built upon a distributed KV architecture, this engine extends retrieval interfaces and functionalities. Optional features include an inverted index structure supporting millisecond-level candidate set filtering. Vector Indexing Engine: Built upon a distributed KV architecture, this engine extends vector retrieval interfaces and functionalities. Optional features include a graph structure supporting cosine similarity calculation and TOP-K (top K largest / smallest elements) sorting. Collaborative Incremental Updates: Index updates are triggered in real-time by subscribing to KV logs, avoiding the high latency issues associated with cross-system message notifications.

[0055] 5) Unified and Collaborative Scheduling: Achieve global resource optimization and intelligent scheduling. Access Frequency Tagging: Record the access time and frequency of data, metadata, and indexes as a basis for lifecycle management; Cache Collaboration: Preload high-frequency data based on access correlation to improve memory cache hit rate; Lifecycle Management: Dynamically adjust the storage location of data to balance performance and cost.

[0056] In this embodiment of the invention, the following features are provided: A unified storage platform: Integrated storage of metadata and indexes is built based on distributed key-value pairs (KV) to solve the problem of heterogeneous data fragmentation; Collaborative incremental updates: Real-time index updates are achieved by subscribing to metadata and KV logs, eliminating cross-system synchronization delays; Lifecycle management: Hot / warm / cold tiered storage based on the access frequency of data, metadata, and indexes optimizes storage resource configuration; Cache collaboration: Multi-level caching linkage between metadata, indexes, and original data is achieved based on data access correlation, improving response speed in high-frequency access scenarios; Hybrid index architecture: Semantic retrieval interfaces are extended on the distributed KV platform, enabling collaborative retrieval of keywords and vector indexes, supporting multimodal queries. This decouples data storage from metadata and indexes, reusing the distributed KV platform as a unified platform for multimodal data management, supporting independent evolution or replacement; It allows for elastic scaling, with the distributed KV sharding architecture supporting horizontal scaling, capable of handling trillions of metadata levels; Collaborative optimization: The unified platform enables cross-component optimization, such as adjusting data distribution based on index access patterns to improve overall query performance.

[0057] This invention provides a unified metadata and index storage mechanism for edge storage systems designed for novel computing applications. Based on a distributed key-value (KV) system, a storage layer for metadata and indexes is constructed, storing not only file metadata but also multimodal descriptions, annotation data, alignment relationships, etc. On top of the distributed KV, a metadata layer (i.e., a metadata engine), a keyword index engine, and a vector index engine are built to enable various service extensions. This integrated storage foundation for metadata and indexes, built on a distributed key-value (KV) storage system, completely breaks down the separation between data and indexes in traditional architectures, achieving one-time writing and global availability of metadata and indexes. Furthermore, a collaborative incremental update mechanism for metadata and indexes is provided: relying on the unified storage foundation built on distributed KV, the system triggers the index module to synchronously update the keyword index and vector index in real time by subscribing to KV logs when updating unstructured data. Based on the transactional characteristics of distributed KV, when updating image metadata, consistent updates of descriptive text, labels, and visual feature indexes can be completed atomically, ensuring strong consistency between data and indexes. A single transaction can be defined, and all three updates are completed within the transaction. A lifecycle management mechanism is provided: each access to metadata and indexes is marked with access time and frequency information. The system automatically tiers storage based on access frequency: hot data (access interval < 1 hour): resident in memory cache to reduce access latency; warm data (access interval 1-24 hours): stored on SSD nodes to improve throughput; cold data (access interval > 7 days): migrated to HDD or archive storage to reduce storage costs. A cache coordination mechanism is also provided: based on the access time and frequency of metadata and indexes, access correlations are identified, triggering cache linkage between metadata and index layers and data storage. For example, when retrieving an image, the system automatically preloads the aligned text description and related visual feature vectors, reducing subsequent access latency.

[0058] During data read and write operations, the client reads and writes multimodal data (text, images, audio, etc.) through file or object interfaces. The metadata engine synchronously generates data changes within the namespace. The keyword retrieval engine synchronously extracts keywords such as file name, type, size, and access time, and generates an index for storage. The vector engine synchronously generates multimodal features such as text embedding vectors, visual feature vectors, and audio fingerprints, while constructing a metadata structure containing data descriptions, annotation information, and modality alignment relationships. The distributed KV storage platform performs transactional operations, atomically writing metadata, keyword indexes, and vector indexes. The metadata engine automatically records the access time, frequency, and correlation of the data, metadata, and indexes, and uniformly schedules lifecycle management based on a heat algorithm and triggers cache coordination mechanisms. For incremental index update processes, when unstructured data is modified, metadata and data are updated synchronously. The distributed KV storage platform synchronously updates metadata fields and associated indexes through a transaction mechanism. The index module consumes transaction logs, performing index reconstruction only on the changed parts to avoid the overhead of a full rebuild. After the update is complete, the system automatically refreshes the corresponding entries in the memory cache and deletes the logs to ensure data consistency in subsequent accesses.

[0059] Multimodal collaborative retrieval process: The client submits a single-modal retrieval request, such as a vector retrieval. The vector retrieval engine executes the request and filters out vector information. Simultaneously, the vector retrieval engine's retrieval request and results are recorded by the distributed KV storage platform, triggering the platform to send multimodal collaborative retrieval requests to the metadata engine and keyword retrieval engine, executing related metadata queries and keyword searches. If the client application then performs keyword searches or data read / write operations based on the vector retrieval results: keyword information is obtained instantly without delay; metadata information has been pre-acquired, eliminating one round trip / time of a metadata query request.

[0060] In this invention, a multimodal data storage system includes a distributed key-value (KV) storage base for storing metadata and indexes of multimodal data, wherein the indexes include keyword indexes and vector indexes; an unstructured data storage layer for storing unstructured data such as text, images, and videos, whose metadata is carried by distributed KV; a metadata engine built on distributed KV to manage metadata in a file system tree-like namespace or an object system flat address space; a keyword index engine built on distributed KV to support attribute information retrieval of unstructured data; and a vector index engine built on distributed KV to store embedded vectors and other information such as text, audio, video, and images. The distributed KV base achieves atomic updates of metadata and indexes through transactional operations, ensuring consistency during data modification. Incremental index updates are achieved by subscribing to metadata KV logs, reducing index maintenance latency. Tiered storage based on access frequency is supported: hot data resides in memory cache, warm data is stored on SSD nodes, and cold data is archived to low-cost storage media. Furthermore, a cache coordination module is provided to link the caching strategies of metadata, indexes, and original data based on data access correlation, improving memory cache hit rate. By defining a combined interface for traditional keyword retrieval and vector retrieval, multimodal retrieval supporting keyword filtering and vector similarity calculation can be achieved. This technical solution can also be applied to centralized storage systems.

[0061] The solution provided by this invention can efficiently handle the explosive growth of metadata. The metadata foundation built on a distributed key-value (KV) architecture boasts extremely high write performance and horizontal scalability, easily handling the challenge of metadata volumes increasing more than tenfold in multimodal data. It improves multimodal retrieval performance; the unified distributed KV foundation eliminates cross-system interaction bottlenecks in traditional architectures, and the composite index is directly built in the KV storage, supporting atomic operations for keyword filtering and vector retrieval. It achieves overall system optimization; the unified storage foundation for metadata and indexes facilitates cross-component joint optimization. For example, the metadata structure can be dynamically adjusted according to query patterns, or metadata information can be used to optimize data and index distribution (i.e., adjusting to different storage layers). Lifecycle management and cache coordination mechanisms effectively reduce system costs. It simplifies the system architecture; compared to traditional architectures combining multiple systems, this invention organically integrates metadata management, index construction, and data storage, reducing repetitive issues in system component development and improving maintainability. It enhances architectural flexibility and scalability; the key-value operation interface of the distributed KV architecture has excellent scalability, allowing for the construction of richer upper-layer semantic interfaces and application-layer services. Data consistency and reliability are guaranteed. Based on a unified distributed key-value base, metadata and indexes are stored. Transactional operations can ensure strong consistency among data, metadata and indexes, solving the industry pain point of index lag after data updates in traditional hybrid architectures.

[0062] See Figure 3 This is a schematic diagram of the structure of a multimodal data storage device provided in an embodiment of the present invention. The multimodal data storage device may include the following modules.

[0063] The data acquisition module 31 is used to acquire data to be written, wherein the data to be written includes data in multiple modalities.

[0064] Data storage module 32 is used to write the data to be written to the target data storage and synchronously write the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value storage.

[0065] The log generation module 33 is used to generate write logs corresponding to the target metadata in the distributed key-value store.

[0066] The index synchronization module 34 is used to update the index data corresponding to the data to be written in the index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the index engine of the write log.

[0067] The indexing engine includes a keyword indexing engine and a vector indexing engine.

[0068] Correspondingly, the index synchronization module 34 can be specifically used to: update the keyword index data corresponding to the data to be written in the keyword index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the keyword index engine of the write log; and update the vector index data corresponding to the data to be written in the vector index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the vector index engine of the write log.

[0069] In an optional implementation, the log generation module 33 is specifically used to: generate a write log corresponding to the target metadata through the distributed key-value storage, and add the write log to the message queue in the distributed key-value storage.

[0070] The index synchronization module 34 is further configured to: read the write log from the message queue by subscribing to the index engine of the write log.

[0071] The index synchronization module 34 is further configured to: delete the write log from the message queue after the index engine updates the index data corresponding to the data to be written in the index storage partition of the distributed key-value store.

[0072] The index synchronization module 34 can also be specifically used to: detect the log status identifier corresponding to the written log, wherein the log status identifier represents the consumption status of the written log; when the log status identifier represents that the written log is in a consumption completed state, the written log is deleted from the message queue.

[0073] Furthermore, the device also includes a log status identifier modification module, configured to: after adding the write log to the message queue in the distributed key-value store, set the log status identifier to a first preset value, indicating that the write log is in an unconsumed state; when the vector index engine subscribing to the write log starts consuming the write log, set the log status identifier to a second preset value, indicating that the write log is in a consumed state; and when the index engine completes updating the index data, set the log status identifier to a third preset value, indicating that the write log is in a consumed state.

[0074] Data storage module 32 includes a metadata storage submodule, which is used to synchronously write the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value storage through the metadata engine.

[0075] In an optional implementation, the log generation module 33 is specifically used to: generate a write log corresponding to the target metadata after synchronously writing the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value storage through the metadata engine, and add the write log to the message queue in the distributed key-value storage.

[0076] The apparatus further includes a retrieval module, comprising: a response submodule, configured to respond to a retrieval request through a target index engine, wherein the target index engine is a keyword index engine or a vector index engine; and a collaborative retrieval submodule, configured to send a multimodal collaborative retrieval request to a metadata engine and / or a non-target index engine within the index engine through the distributed key-value storage; and to pre-load metadata associated with the response data of the retrieval request and / or index data of the non-target index engine into a cache through the metadata engine and / or the non-target index engine, so as to respond to query requests corresponding to the associated metadata and associated index data.

[0077] Specifically, the collaborative retrieval submodule is used to preload the metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine into the cache based on the data access frequency through the metadata engine and / or the non-target index engine.

[0078] The device further includes a recording module for recording the data access frequency of data in the target data storage, metadata in the metadata storage partition, and index data in the index storage partition.

[0079] The device further includes a hierarchical storage module, used to: hierarchically store the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition based on the data access frequency.

[0080] The hierarchical storage module can be specifically used to: classify the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition into cold data, warm data, and hot data based on the data access frequency, wherein the data access frequency of the cold data is less than that of the warm data, and the data access frequency of the warm data is less than that of the hot data; and store the cold data, the warm data, and the hot data in different levels of storage space.

[0081] In an optional implementation, the keyword index data adopts an inverted index structure, and the vector index data adopts a graph structure.

[0082] In an optional implementation, the target data storage includes distributed object storage and distributed file storage. The data storage module 32 may include a data storage submodule, specifically used for: if the target object to which the data to be written is the distributed object storage, then writing the data to be written to the distributed object storage; if the target object to be written is the distributed file storage, then writing the data to be written to the distributed file storage.

[0083] As can be seen, this embodiment of the invention writes the multimodal data to be written to the target data storage, and simultaneously writes the corresponding metadata to the metadata storage partition in the distributed key-value store, and generates a write log in the metadata storage partition. The index engine is a subscriber to the metadata write log, and synchronously updates the corresponding index data in the index storage partition in the distributed key-value store. In this way, a unified distributed key-value storage platform is provided for storing metadata and index data. The index does not need to rely on an external system, which can avoid the complex processing caused by cross-system transaction coordination, and efficiently and reliably ensure the synchronous update of index data and metadata.

[0084] Figure 3 For a description of the features in the corresponding embodiments, please refer to Figure 1 The relevant descriptions of the corresponding embodiments will not be repeated here.

[0085] Figure 4A structural diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 4 As shown, the electronic device includes: a memory 40 for storing a computer program; and a processor 41 for executing the computer program to implement the steps of the multimodal data storage method as described in the above embodiments.

[0086] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: acquiring data to be written, wherein the data to be written includes data of multiple modalities; writing the data to be written to the target data storage, and synchronously writing the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value storage; generating a write log corresponding to the target metadata in the distributed key-value storage; and updating the index data corresponding to the data to be written in the index storage partition of the distributed key-value storage based on the target metadata and the data to be written by subscribing to the index engine of the write log.

[0087] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: by subscribing to the keyword index engine of the write log, updating the keyword index data corresponding to the data to be written in the keyword index storage partition of the distributed key-value store based on the target metadata and the data to be written; by subscribing to the vector index engine of the write log, updating the vector index data corresponding to the data to be written in the vector index storage partition of the distributed key-value store based on the target metadata and the data to be written.

[0088] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: generate a write log corresponding to the target metadata through the distributed key-value storage, and add the write log to the message queue in the distributed key-value storage.

[0089] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: reading the write log from the message queue by subscribing to the index engine of the write log.

[0090] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: after the index engine updates the index data corresponding to the data to be written in the index storage partition of the distributed key-value store, the write log is deleted from the message queue.

[0091] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: detecting the log status identifier corresponding to the written log, wherein the log status identifier represents the consumption status of the written log; when the log status identifier represents that the written log is in a consumption completed state, the written log is deleted from the message queue.

[0092] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: after adding the write log to the message queue in the distributed key-value store, the log status identifier is set to a first preset value, indicating that the write log is in an unconsumed state; when the vector index engine subscribing to the write log starts consuming the write log, the log status identifier is set to a second preset value, indicating that the write log is in a consumed state; when the index engine completes updating the index data, the log status identifier is set to a third preset value, indicating that the write log is in a consumed state.

[0093] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: synchronously write the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value storage through the metadata engine.

[0094] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: after synchronously writing the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value storage through the metadata engine, a write log corresponding to the target metadata is generated, and the write log is added to the message queue in the distributed key-value storage.

[0095] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: responding to a retrieval request through a target index engine, wherein the target index engine is a keyword index engine or a vector index engine; sending a multimodal collaborative retrieval request to the metadata engine and / or a non-target index engine in the index engine through the distributed key-value storage; and preloading the metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine into a cache through the metadata engine and / or the non-target index engine, so as to respond to the query request corresponding to the associated metadata and the associated index data.

[0096] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: based on the data access frequency, the metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine are preloaded into the cache through the metadata engine and / or the non-target index engine.

[0097] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: recording the data access frequency of the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition.

[0098] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: based on the data access frequency, hierarchically store the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition.

[0099] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: based on the data access frequency, classify the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition into cold data, warm data, and hot data, wherein the data access frequency of the cold data is less than that of the warm data, and the data access frequency of the warm data is less than that of the hot data; store the cold data, the warm data, and the hot data in different levels of storage space.

[0100] As can be seen, this embodiment of the invention writes the multimodal data to be written to the target data storage, and simultaneously writes the corresponding metadata to the metadata storage partition in the distributed key-value store, and generates a write log in the metadata storage partition. The index engine is a subscriber to the metadata write log, and synchronously updates the corresponding index data in the index storage partition in the distributed key-value store. In this way, a unified distributed key-value storage platform is provided for storing metadata and index data. The index does not need to rely on an external system, which can avoid the complex processing caused by cross-system transaction coordination, and efficiently and reliably ensure the synchronous update of index data and metadata.

[0101] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: if the target object to which the data to be written is to be written is the distributed object storage, then the data to be written is written to the distributed object storage; if the target object to be written is the distributed file storage, then the data to be written is written to the distributed file storage.

[0102] The processor 41 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 41 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 41 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 41 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 41 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0103] The memory 40 may include one or more computer-readable storage media, which may be non-transitory. The memory 40 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 40 is used to store at least the following computer program 401, which, after being loaded and executed by the processor 41, is capable of implementing the relevant steps of the multimodal data storage method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 40 may also include an operating system 402 and data 403, and the storage method may be temporary or permanent storage. The operating system 402 may include Windows, Unix, Linux, etc. The data 403 may include, but is not limited to, multimodal data.

[0104] In some embodiments, the electronic device may further include a display screen 42, an input / output interface 43, a communication interface 44, a power supply 45, and a communication bus 46.

[0105] Those skilled in the art will understand that Figure 4 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0106] It is understood that if the multimodal data storage method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk, or optical disk, and other media capable of storing program code.

[0107] Based on this, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the multimodal data storage method described above.

[0108] The following describes a computer program product provided by an embodiment of the present invention. The computer program product described below can be referred to in conjunction with other embodiments described herein.

[0109] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed multimodal data storage method.

[0110] The foregoing has provided a detailed description of a multimodal data storage method, apparatus, device, medium, and product provided by embodiments of the present invention. The various embodiments are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0111] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0112] The foregoing has provided a detailed description of a multimodal data storage method, apparatus, device, medium, and product provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A multimodal data storage method, characterized in that, include: Acquire the data to be written, wherein the data to be written includes data in multiple modalities; The data to be written is written to the target data storage, and the target metadata corresponding to the data to be written is synchronously written to the metadata storage partition in the distributed key-value storage. In the distributed key-value store, a write log corresponding to the target metadata is generated; By subscribing to the index engine of the write log, the index data corresponding to the data to be written in the index storage partition of the distributed key-value store is updated based on the target metadata and the data to be written.

2. The multimodal data storage method according to claim 1, characterized in that, The indexing engine includes a keyword indexing engine and a vector indexing engine; Accordingly, by subscribing to the index engine of the write log, the index data corresponding to the data to be written in the index storage partition of the distributed key-value store is updated based on the target metadata and the data to be written, including: By subscribing to the keyword index engine of the write log, the keyword index data corresponding to the data to be written in the keyword index storage partition of the distributed key-value store is updated based on the target metadata and the data to be written; By subscribing to the vector index engine of the write log, the vector index data corresponding to the data to be written in the vector index storage partition of the distributed key-value store is updated based on the target metadata and the data to be written.

3. The multimodal data storage method according to claim 2, characterized in that, The write log corresponding to the target metadata is generated in the distributed key-value store, including: The write log corresponding to the target metadata is generated through the distributed key-value storage, and the write log is added to the message queue in the distributed key-value storage.

4. The multimodal data storage method according to claim 3, characterized in that, Also includes: The write log is read from the message queue by subscribing to the index engine of the write log.

5. The multimodal data storage method according to claim 4, characterized in that, Also includes: After the index engine updates the index data corresponding to the data to be written in the index storage partition of the distributed key-value store, the write log is deleted from the message queue.

6. The multimodal data storage method according to claim 5, characterized in that, Deleting the write log from the message queue includes: Detect the log status identifier corresponding to the written log, wherein the log status identifier represents the consumption status of the written log; When the log status identifier indicates that the write log has been consumed, the write log is deleted from the message queue.

7. The multimodal data storage method according to claim 6, characterized in that, Also includes: After the write log is added to the message queue in the distributed key-value store, the log status identifier is set to a first preset value, indicating that the write log is in an unconsumed state. When the vector index engine that subscribes to the write log starts consuming the write log, the log status identifier is set to the second preset value, indicating that the write log is in a consumption state; Once the indexing engine has finished updating the index data, it sets the log status identifier to a third preset value, indicating that the log writing is in a consumption completed state.

8. The multimodal data storage method according to claim 2, characterized in that, Synchronously writing the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value store includes: The target metadata corresponding to the data to be written is synchronously written to the metadata storage partition in the distributed key-value store through the metadata engine.

9. The multimodal data storage method according to claim 8, characterized in that, The write log corresponding to the target metadata is generated in the distributed key-value store, including: After the target metadata corresponding to the data to be written is synchronously written to the metadata storage partition in the distributed key-value store through the metadata engine, a write log corresponding to the target metadata is generated, and the write log is added to the message queue in the distributed key-value store.

10. The multimodal data storage method according to claim 8, characterized in that, Also includes: The retrieval request is responded to through the target index engine, wherein the target index engine is a keyword index engine or a vector index engine; Multimodal collaborative retrieval requests are sent to the metadata engine and / or non-target index engines in the index engine through the distributed key-value store; The metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine are pre-loaded into the cache by the metadata engine and / or the non-target index engine, so as to respond to the query request corresponding to the associated metadata and the associated index data.

11. The multimodal data storage method according to claim 10, characterized in that, The metadata associated with the response data of the retrieval request and / or the index data of the non-target index engine are pre-loaded into the cache through the metadata engine and / or the non-target index engine, including: Metadata associated with the response data of the retrieval request and / or index data of the non-target index engine are pre-loaded into the cache based on the data access frequency through the metadata engine and / or the non-target index engine.

12. The multimodal data storage method according to claim 11, characterized in that, Also includes: Record the data access frequency of the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition.

13. The multimodal data storage method according to claim 12, characterized in that, Also includes: Based on the data access frequency, the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition are stored in a hierarchical manner.

14. The multimodal data storage method according to claim 13, characterized in that, Based on the data access frequency, the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition are stored in a hierarchical manner, including: Based on the data access frequency, the data in the target data storage, the metadata in the metadata storage partition, and the index data in the index storage partition are classified into cold data, warm data, and hot data, wherein the data access frequency of cold data is less than that of warm data, and the data access frequency of warm data is less than that of hot data. The cold data, the warm data, and the hot data are stored in different levels of storage space.

15. The multimodal data storage method according to claim 2, characterized in that, The keyword index data adopts an inverted index structure, and the vector index data adopts a graph structure.

16. The multimodal data storage method according to any one of claims 1 to 15, characterized in that, The target data storage includes distributed object storage and distributed file storage. Writing the data to be written to the target data storage includes: If the target object for writing the data to be written is the distributed object storage, then the data to be written is written to the distributed object storage; if the target object for writing is the distributed file storage, then the data to be written is written to the distributed file storage.

17. A multimodal data storage device, characterized in that, include: A data acquisition module is used to acquire data to be written, wherein the data to be written includes data in multiple modalities; The data storage module is used to write the data to be written to the target data storage and synchronously write the target metadata corresponding to the data to be written to the metadata storage partition in the distributed key-value storage. The log generation module is used to generate write logs corresponding to the target metadata in the distributed key-value store; The index synchronization module is used to update the index data corresponding to the data to be written in the index storage partition of the distributed key-value store based on the target metadata and the data to be written by subscribing to the index engine of the write log.

18. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the multimodal data storage method as described in any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the multimodal data storage method as described in any one of claims 1 to 16.

20. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the multimodal data storage method according to any one of claims 1 to 16.

Images