Vector database collaborative optimization method, device and equipment with large language model

Abstract

The application discloses a vector database and large language model collaborative optimization method, device and equipment, which comprises the following steps: firstly, extracting data segments of target unstructured data; then, constructing a graph neural network by using the data segments, and optimizing a large language model by using semantic information and mutual dependency of each node in the graph neural network; then, encoding each data segment by using the optimized large language model, and constructing or updating vector values of the data segments in a vector database by using obtained coding vectors; then, updating the semantic information and mutual dependency of each node in the graph neural network according to the similarity between the vector values of each data segment in the database; then, re-executing the optimization of the large language model and the subsequent steps by using the updated semantic information and mutual dependency of each node in the graph neural network, and sequentially repeating the above steps until a preset stop optimization condition is met, so as to obtain the optimized graph neural network, vector database and large language model.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method, apparatus and device for collaborative optimization of vector database and large language model. Background Technology

[0002] Vector databases, which transform and store unstructured data into semantically meaningful vector data that supports fast vector similarity searches, have become a crucial data storage and retrieval tool. Furthermore, with the continuous growth of data volume and application demands, as well as the development of large language model (LLM) technology, researchers are increasingly focusing on research and applications related to vector databases and LLM. Among these, effectively updating and optimizing vector databases and LLM to fully leverage the value of large amounts of unstructured data is particularly important.

[0003] Currently, existing methods for updating and optimizing vector databases typically include data slicing-based optimization, vector encoder-based optimization, and vector retrieval-based optimization, but the results of these methods are not ideal. Existing methods for updating and optimizing large language models (LLMs) typically include model architecture-based methods, training or fine-tuning-based methods, compression and quantization-based methods, and methods based on external plugins or knowledge bases, but the results of these methods are also not ideal. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, and device for collaborative optimization of vector database and large language model, which can achieve iterative optimization of large language model LLM and vector database based on graph neural network to obtain vector database and large language model containing rich semantic information and complex interdependencies.

[0005] This application provides a method for collaborative optimization of vector databases and large language models, including:

[0006] Acquire the target unstructured data; and extract data fragments from the target unstructured data;

[0007] A graph neural network is constructed using data fragments of the target unstructured data; and a large language model (LLM) is optimized using the semantic information and interdependencies of each node in the graph neural network.

[0008] The optimized Large Language Model (LLM) is used to encode each data segment, and the resulting encoded vectors are used to construct a vector database or update the vector values ​​of the corresponding data segments in the vector database.

[0009] Based on the similarity between the vector values ​​of each data segment in the updated vector database, the semantic information and interdependencies of each node in the graph neural network are updated.

[0010] By utilizing the semantic information and interdependencies of each node in the updated graph neural network, the optimization of the large language model LLM and subsequent steps are re-executed. This process is repeated iteratively and collaboratively to optimize the graph neural network, the large language model LLM, and the vector database until the preset stopping optimization condition is met, resulting in the optimized graph neural network, the large language model LLM, and the vector database.

[0011] In one possible implementation, the target unstructured data includes text data; the extraction of data fragments from the target unstructured data includes:

[0012] The text data is segmented using a word segmenter to obtain the characters, words, and phrases contained in the text data.

[0013] In one possible implementation, the target unstructured data includes image data; the extraction of data segments from the target unstructured data includes:

[0014] The image data is segmented using an image segmentation model to obtain image blocks contained in the image data.

[0015] In one possible implementation, the target unstructured data includes audio data and / or video data; the extraction of data segments from the target unstructured data includes:

[0016] The audio data and / or video data are segmented into single frames or N frames to obtain audio segments of the audio data and / or video segments of the video data; where N is a positive integer greater than 1.

[0017] In one possible implementation, constructing a graph neural network using data fragments of the target unstructured data includes:

[0018] The co-occurrence frequency of each data segment in all target unstructured data corpora is counted, and the co-occurrence frequency is normalized as the weight value of the edge between each data segment in the graph neural network;

[0019] Initial vectors containing semantic information for each data segment are generated based on the vector embedding method, and these vectors serve as nodes in the graph neural network.

[0020] In one possible implementation, updating the semantic information and interdependencies of each node in the graph neural network based on the similarity between the vector values ​​of each data segment in the updated vector database includes:

[0021] Calculate the vector values ​​of each data segment in the updated vector database, and calculate the similarity between the vector values ​​of each data segment;

[0022] Based on the similarity between the vector values ​​of the various data segments, the weight values ​​of the edges between the nodes in the graph neural network are updated, thereby updating the semantic information and interdependencies of the nodes in the graph neural network.

[0023] In one possible implementation, the preset stopping optimization condition includes at least one of the following: reaching a preset number of iterations, the loss function value of the task corresponding to the large language model converging, and the update rate of the edges between nodes in the graph neural network being lower than a preset percentage.

[0024] This application also provides a vector database and large language model collaborative optimization device, including:

[0025] An acquisition unit is used to acquire target unstructured data and extract data fragments from the target unstructured data.

[0026] The first optimization unit is used to construct a graph neural network using data fragments of the target unstructured data; and to optimize the large language model LLM using the semantic information and interdependencies of each node in the graph neural network.

[0027] The first update unit is used to encode each data segment using the optimized Large Language Model (LLM) and to construct a vector database or update the vector values ​​of the corresponding data segments in the vector database using the obtained encoded vectors.

[0028] The second update unit is used to update the semantic information and interdependencies of each node in the graph neural network based on the similarity between the vector values ​​of each data segment in the updated vector database.

[0029] The second optimization unit is used to utilize the semantic information and interdependencies of each node in the updated graph neural network to re-execute the first optimization unit, the first update unit, and the second update unit, and so on, to iteratively and collaboratively optimize the graph neural network, the large language model LLM, and the vector database until the preset stopping optimization condition is met, thus obtaining the optimized graph neural network, the large language model LLM, and the vector database.

[0030] In one possible implementation, the target unstructured data includes text data; the acquisition unit is specifically used for:

[0031] The text data is segmented using a word segmenter to obtain the characters, words, and phrases contained in the text data.

[0032] In one possible implementation, the target unstructured data includes image data; the acquisition unit is specifically used for:

[0033] The image data is segmented using an image segmentation model to obtain image blocks contained in the image data.

[0034] In one possible implementation, the target unstructured data includes audio data and / or video data; the acquisition unit is specifically used for:

[0035] The audio data and / or video data are segmented into single frames or N frames to obtain audio segments of the audio data and / or video segments of the video data; where N is a positive integer greater than 1.

[0036] In one possible implementation, the first optimization unit includes:

[0037] The statistical subunit is used to count the co-occurrence frequency of each data segment in all target unstructured data corpora, and normalize the co-occurrence frequency as the weight value of the edge between each data segment in the graph neural network.

[0038] The generation subunit is used to generate initial vectors containing semantic information for each data segment based on the vector embedding method, which serve as nodes in the graph neural network.

[0039] In one possible implementation, the second update unit includes:

[0040] The computational subunit is used to calculate the vector values ​​of each data segment in the updated vector database and to calculate the similarity between the vector values ​​of each data segment.

[0041] The update subunit is used to update the weight values ​​of the edges between nodes in the graph neural network based on the similarity between the vector values ​​of the data segments, thereby updating the semantic information and interdependencies of each node in the graph neural network.

[0042] In one possible implementation, the preset stopping optimization condition includes at least one of the following: reaching a preset number of iterations, the loss function value of the task corresponding to the large language model converging, and the update rate of the edges between nodes in the graph neural network being lower than a preset percentage.

[0043] This application embodiment also provides a vector database and large language model collaborative optimization device, including: a processor, a memory, and a system bus;

[0044] The processor and the memory are connected via the system bus;

[0045] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described implementations of the vector database and large language model co-optimization method.

[0046] This application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described methods for co-optimization of vector databases and large language models.

[0047] This application also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any one of the above-described methods for co-optimization of vector database and large language model.

[0048] This application provides a method, apparatus, and device for collaborative optimization of a vector database and a large language model. First, target unstructured data is acquired; data fragments of the target unstructured data are extracted; then, a graph neural network is constructed using the data fragments of the target unstructured data; and the semantic information and interdependencies of each node in the graph neural network are used to optimize the large language model (LLM). Next, the optimized large language model (LLM) is used to encode each data fragment, and the obtained encoded vectors are used to construct a vector database or update the vector values ​​of the corresponding data fragments in the vector database. Then, based on the similarity between the vector values ​​of each data fragment in the updated vector database, the semantic information and interdependencies of each node in the graph neural network are updated. Furthermore, the semantic information and interdependencies of each node in the updated graph neural network can be used to re-execute the optimization of the large language model (LLM) and subsequent steps, and so on, iteratively co-optimizing the graph neural network, the large language model (LLM), and the vector database until a preset stopping optimization condition is met, resulting in an optimized graph neural network, a vector database containing rich semantic information and complex interdependencies, and the large language model (LLM). Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart illustrating a method for collaborative optimization of a vector database and a large language model, provided in an embodiment of this application;

[0051] Figure 2 An example diagram illustrating the overall process of collaborative optimization of vector database and large language model provided in this application embodiment;

[0052] Figure 3 This is a schematic diagram illustrating the composition of a vector database and large language model collaborative optimization device provided in an embodiment of this application. Detailed Implementation

[0053] With the continuous development and application of multimedia and internet technologies, various types of unstructured data, such as text, images, voice, video, documents, protein structures, and sensor data, are experiencing explosive growth. In the era of data empowerment, effectively utilizing the value of massive amounts of unstructured data has become a key aspect of the intelligent development of various industries.

[0054] Vector databases overcome the limitations of traditional databases, which can only store structured data. They transform and store unstructured data into vector data with semantic information and support for fast vector similarity search. This allows for applications in various machine learning applications such as retrieval, recommendation, and question answering, as well as in scenarios involving the training, inference, real-time or professional knowledge bases, and long-term memory of large language models. Furthermore, with the continuous growth of data volume and application demands, and the development of Large Language Model (LLM) technology, researchers are increasingly focusing on research and applications related to vector databases and large language models. Among these, effectively updating and optimizing vector databases and large language models to fully leverage the value of large amounts of unstructured data is particularly important.

[0055] Existing methods for updating and optimizing vector databases typically include: optimization based on data slicing, optimization based on vector encoders, and optimization based on vector retrieval. Existing methods for updating and optimizing large language models (LLMs) typically include: methods based on model architecture, methods based on training or fine-tuning, methods based on compression and quantization, and methods based on external plugins or knowledge bases.

[0056] Among the existing vector database update optimization methods, data slicing-based methods can optimize the size and semantic integrity of data fragments, but may not balance global or target performance; vector encoder-based optimization methods can optimize the vector representation of each target object, but general machine learning or deep learning model vector encoders have difficulty encoding large amounts of multimodal unstructured data into vector representations; vector retrieval-based optimization methods can optimize the accuracy or speed of vector search, but the effectiveness of vector retrieval often depends on the performance of the vector encoder.

[0057] Among the optimization methods for large language models, model architecture-based methods can optimize model size, encoding / decoding capabilities, attention mechanisms, and context length, but require specialized experience and extensive validation to design an excellent model architecture. Training or fine-tuning-based methods can optimize the model's basic capabilities and downstream task performance, but require large amounts of data or computational resources. Compression and quantization-based methods can optimize the model's memory and computational resource usage and accelerate model training and inference, but compression and quantization often lead to a decrease in model performance or require additional network models or resources to compensate for the performance loss. Methods based on external plugins or knowledge bases can enhance or expand the model's capabilities or knowledge boundaries, but external plugins, third-party APIs, and knowledge bases are often difficult to update or to coordinate with the model for optimization.

[0058] It is evident that the existing methods for updating and optimizing vector databases and large language models do not achieve ideal optimization results.

[0059] To address the aforementioned shortcomings, this application provides a method for the collaborative optimization of a vector database and a large language model. First, target unstructured data is acquired; then, data fragments of the target unstructured data are extracted; next, a graph neural network is constructed using these data fragments; and the semantic information and interdependencies of each node in the graph neural network are used to optimize the large language model (LLM). Then, the optimized LLM is used to encode each data fragment, and the resulting encoded vectors are used to construct a vector database or update the vector values ​​of the corresponding data fragments in the vector database. Based on the similarity between the vector values ​​of each data fragment in the updated vector database, the semantic information and interdependencies of each node in the graph neural network are updated. Furthermore, using the updated semantic information and interdependencies of each node in the graph neural network, the optimization of the large language model (LLM) and subsequent steps can be re-executed. This process is repeated iteratively to collaboratively optimize the graph neural network, the large language model (LLM), and the vector database until a preset stopping condition is met, resulting in an optimized graph neural network, a vector database containing rich semantic information and complex interdependencies, and the large language model (LLM).

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] First Embodiment

[0062] See Figure 1 This is a flowchart illustrating a method for collaborative optimization of a vector database and a large language model, as provided in this embodiment. The method includes the following steps:

[0063] S101: Acquire the target unstructured data; and extract data fragments from the target unstructured data.

[0064] In this embodiment, the data used for collaborative optimization of the vector database and the Large Language Model (LLM) is defined as the target unstructured data, and a corpus containing all the target unstructured data is constructed. It should be noted that this embodiment does not limit the language type of the target unstructured data; for example, the target unstructured data can be Chinese data or English data. This embodiment also does not limit the length of the target unstructured data; that is, the target unstructured data can be sentence data (i.e., a collection of words) or passage data (i.e., a collection of sentences), etc. Furthermore, this embodiment does not limit the content and type of the target unstructured data; for example, the target unstructured data can be, but is not limited to, text, images, speech, video, documents, protein structures, sensor data, etc.

[0065] Large Language Models (LLMs) are deep learning-based language models that can generate new language expressions, such as text, sentences, paragraphs, and even articles, based on input text content. LLMs utilize large-scale language datasets and are trained on language rules and patterns through autoregressive generation. They can simulate human commands to generate language expressions (such as text data). Specifically, when generating new text data, LLMs predict the probability of the next language unit based on previously generated content until complete text data is generated.

[0066] Furthermore, after processing the target unstructured data, existing or future data processing algorithms can be used to process the target unstructured data, such as... Figure 2 As shown, data fragments of the target unstructured data are proposed and represented using tokens. That is, tokens can be used to represent data fragments such as characters, sub-words, words, and phrases with semantic information, in order to execute subsequent steps S102-S105.

[0067] Specifically, one possible implementation is that the target unstructured data may include text data; then the process of extracting data fragments from the target unstructured data may include: using a tokenization tool to segment the text data to obtain tokens corresponding to the characters, words, and phrases contained in the text data.

[0068] The specific structure of the word segmenter is not limited and can be set according to the actual situation and experience. For example, the word segmenter can be set to Byte Pair Encoding (BPE).

[0069] Another optional implementation is that the target unstructured data may include image data; then the process of extracting data fragments from the target unstructured data may include: using an image segmentation model to segment the image data to obtain the token corresponding to the image patch contained in the image data.

[0070] The specific composition and structure of the image segmentation model are not limited and can be set according to the actual situation and empirical values. For example, the image segmentation model can be set as Deformable Patch.

[0071] Another optional implementation is that the target unstructured data may include audio data and / or video data; then the process of extracting data segments from the target unstructured data may include: segmenting the audio data and / or video data into single frames or N (positive integers greater than 1) frames to obtain the tokens corresponding to the audio segments of the audio data and / or the video segments of the video data.

[0072] S102: Construct a graph neural network using data fragments of the target unstructured data; and optimize the large language model LLM by utilizing the semantic information and interdependencies of each node in the graph neural network.

[0073] In this embodiment, after extracting the data segment tokens of the target unstructured data in step S101, the co-occurrence frequency of each data segment token in the entire target unstructured data corpus can be counted and normalized as the weight value of the edges between the data segments in the graph neural network. Then, based on the vector embedding method (the specific content is not limited, such as the global vector word embedding GloVe), the initial vector containing the semantic information of each data segment token is generated as the node of the graph neural network to construct the graph neural network (such as a continuous multi-layer graph convolutional neural network).

[0074] Furthermore, the constructed graph neural network (such as a multi-layered graph convolutional neural network) can be concatenated and spliced ​​before the output layer (without specific structural limitations, such as a linear layer or a multi-layer perceptron, of a large language model (LLM) – the specific structure of which is not limited, such as a plain text model like LLaMA or OPT, or a multimodal model like LLaVA or NExT-GPT) – to leverage the semantic information and interdependencies of each node in the graph neural network to optimize the token vector. This allows for the use of various large language modeling tasks (such as language understanding and generation, semantic reasoning, question answering, information retrieval, and multimodal understanding and generation) on corpus data to train and optimize the large language model and the graph neural network.

[0075] For example: Figure 2 As shown, taking the large language model task as a knowledge question answering example, the user inputs a question into the large language model (Which of the wolf, dog, and squid belongs to invertebrates?). The input goes through several modules including tokenization (segmenting into tokens such as wolf, dog, squid, and invertebrate), vectorization, language modeling optimization and knowledge integration, relationship modeling optimization (modeling and updating the relationships between wolf, dog, squid, and invertebrate, and adding tokens such as cephalopods, vertebrates, and mammals and their relationships), and language decoding. The resulting answer from the large language model is: Squid is an invertebrate because it belongs to the class Cephalopoda (molluscs). Wolves and dogs belong to the class Mammalia (mammals) within the class Vertebrates.

[0076] S103: Encode each data segment using the optimized Large Language Model (LLM), and use the obtained encoded vectors to construct a vector database or update the vector values ​​of the corresponding data segments in the vector database.

[0077] In this embodiment, after obtaining the optimized large language model LLM through step S102, the optimized large language model LLM can be used as a vector encoder to encode each token to obtain vectors suitable for various large language model tasks (such as language understanding and generation, semantic reasoning, knowledge question answering, information retrieval, multimodal understanding and generation, etc.) to construct an initialized vector database or update the vector values ​​of the corresponding tokens in the vector database.

[0078] For example: Based on the above examples, such as Figure 2As shown, taking the large language model task as knowledge question answering as an example, each token is encoded into a vector with semantic information after passing through a tokenizer, text encoder, large language model, and graph neural network. These vectors (vectors corresponding to wolves, dogs, squid, and invertebrates) can then be used to construct a vector database. After training and optimization for the knowledge question answering task, the vectors output by the graph neural network can also be updated accordingly (e.g., updating the values ​​of the vectors corresponding to wolves, dogs, squid, and invertebrates, and adding vectors for cephalopods, vertebrates, mammals, etc.).

[0079] S104: Based on the similarity between the vector values ​​of each data segment in the updated vector database, update the semantic information and interdependencies of each node in the graph neural network.

[0080] In this embodiment, after obtaining the vector values ​​of each data segment token in the updated vector database through step S103, existing or future data vectorization methods can be used to first calculate the vector values ​​of each data segment in the updated vector database, and then calculate the similarity between the vector values ​​of each data segment (the specific content is not limited, such as cosine similarity, Euclidean distance, etc.). Then, based on the similarity between the vector values ​​of each data segment, the weight values ​​of the edges between each node in the graph neural network are updated, thereby updating the semantic information and interdependencies of each node in the graph neural network, so that the graph neural network can better model the semantic information of each node vector and the interrelationships between nodes.

[0081] For example: Based on the above examples, such as Figure 2 As shown, taking the large language model task as knowledge question answering as an example, after the vector database is updated (the values ​​of each token vector have changed, and new token vectors may be added), nodes corresponding to the newly added tokens (such as nodes corresponding to tokens of Cephalopoda, Vertebrates, and Mammals) can be added based on the similarity between the calculated token vectors, and the edges between tokens in the graph neural network can be updated (such as deleting and adding edges between nodes corresponding to tokens of Wolf, Dog, Squid, Invertebrates, Cephalopoda, Vertebrates, and Mammals).

[0082] S105: Utilize the semantic information and interdependencies of each node in the updated graph neural network to re-execute the optimization of the large language model LLM and subsequent steps, and so on, iteratively and collaboratively optimizing the graph neural network, the large language model LLM and the vector database until the preset stopping optimization condition is met, and obtain the optimized graph neural network, the large language model LLM and the vector database.

[0083] In this embodiment, after updating the semantic information and interdependencies of each node in the graph neural network based on the similarity between the vector values ​​of each data segment in the updated vector database in step S104, the process of optimizing the large language model LLM in step S102 and subsequent steps S103-S104 can be re-executed using the updated semantic information and interdependencies of each node in the graph neural network. This process is repeated iteratively and collaboratively optimized in the form of graph neural network, large language model LLM, and vector database until the preset stopping optimization condition is met, resulting in an optimized graph neural network, a vector database containing rich semantic information and complex interdependencies, and a large language model LLM.

[0084] The specific value of the preset stopping optimization condition is not limited and can be set according to the actual situation and experience. One possible implementation is that the preset stopping optimization condition includes at least one of the following: reaching a preset number of iterations (the specific number is not limited, such as 5 times), the loss function value of the large language model corresponding to the task (the specific content is not limited, such as language understanding and generation, semantic reasoning, knowledge question answering, information retrieval, multimodal understanding and generation, etc.) converges (such as the value of the loss function is small enough to remain basically unchanged), and the update rate of the edges between nodes in the graph neural network is lower than a preset percentage (the specific value is not limited, such as 2%).

[0085] In this way, by performing the above steps S101-S105, we can fully utilize and mutually promote the advantages of graph neural networks in semantic information and relation modeling, high-quality vector encoding with strong generalization of large language models, and vector similarity retrieval calculation of vector databases. This results in graph neural networks that are more conducive to relation modeling and vector retrieval, vector databases with strong generalization, and improved semantic reasoning and semantic relation modeling capabilities of large language models.

[0086] In summary, this embodiment provides a method for collaborative optimization of a vector database and a large language model. First, target unstructured data is acquired; then, data fragments of the target unstructured data are extracted; next, a graph neural network is constructed using these data fragments; and the semantic information and interdependencies of each node in the graph neural network are used to optimize the large language model (LLM). Then, the optimized LLM is used to encode each data fragment, and the resulting encoded vectors are used to construct a vector database or update the vector values ​​of the corresponding data fragments in the vector database. Based on the similarity between the vector values ​​of each data fragment in the updated vector database, the semantic information and interdependencies of each node in the graph neural network are updated. Furthermore, the updated semantic information and interdependencies of each node in the graph neural network are used to re-execute the optimization of the large language model (LLM) and subsequent steps, iteratively optimizing the graph neural network, the large language model (LLM), and the vector database until a preset stopping condition is met, resulting in an optimized graph neural network, a vector database containing rich semantic information and complex interdependencies, and the large language model (LLM).

[0087] Second Embodiment

[0088] This embodiment will introduce a device for the collaborative optimization of vector database and large language model. For related content, please refer to the above method embodiment.

[0089] See Figure 3 This is a schematic diagram of the composition of a vector database and large language model collaborative optimization device provided in this embodiment. The device 300 includes:

[0090] Acquisition unit 301 is used to acquire target unstructured data and extract data fragments from the target unstructured data;

[0091] The first optimization unit 302 is used to construct a graph neural network using data fragments of the target unstructured data; and to optimize the large language model LLM by utilizing the semantic information and interdependencies of each node in the graph neural network.

[0092] The first update unit 303 is used to encode each data segment using the optimized large language model LLM, and to construct a vector database or update the vector value of the corresponding data segment in the vector database using the obtained encoding vectors.

[0093] The second updating unit 304 is used to update the semantic information and interdependencies of each node in the graph neural network based on the similarity between the vector values ​​of each data segment in the updated vector database.

[0094] The second optimization unit 305 is used to utilize the semantic information and interdependencies of each node in the updated graph neural network to re-invoke the first optimization unit, the first update unit, and the second update unit, and so on, to iteratively and collaboratively optimize the graph neural network, the large language model LLM, and the vector database until the preset stopping optimization condition is met, thereby obtaining the optimized graph neural network, the large language model LLM, and the vector database.

[0095] In one implementation of this embodiment, the target unstructured data includes text data; the acquisition unit 301 is specifically used for:

[0096] The text data is segmented using a word segmenter to obtain the characters, words, and phrases contained in the text data.

[0097] In one implementation of this embodiment, the target unstructured data includes image data; the acquisition unit 301 is specifically used for:

[0098] The image data is segmented using an image segmentation model to obtain image blocks contained in the image data.

[0099] In one implementation of this embodiment, the target unstructured data includes audio data and / or video data; the acquisition unit 301 is specifically used for:

[0100] The audio data and / or video data are segmented into single frames or N frames to obtain audio segments of the audio data and / or video segments of the video data; where N is a positive integer greater than 1.

[0101] In one implementation of this embodiment, the first optimization unit 302 includes:

[0102] The statistical subunit is used to count the co-occurrence frequency of each data segment in all target unstructured data corpora, and normalize the co-occurrence frequency as the weight value of the edge between each data segment in the graph neural network.

[0103] The generation subunit is used to generate initial vectors containing semantic information for each data segment based on the vector embedding method, which serve as nodes in the graph neural network.

[0104] In one implementation of this embodiment, the second update unit 304 includes:

[0105] The computational subunit is used to calculate the vector values ​​of each data segment in the updated vector database and to calculate the similarity between the vector values ​​of each data segment.

[0106] The update subunit is used to update the weight values ​​of the edges between nodes in the graph neural network based on the similarity between the vector values ​​of the data segments, thereby updating the semantic information and interdependencies of each node in the graph neural network.

[0107] In one implementation of this embodiment, the preset stopping optimization condition includes at least one of the following: reaching a preset number of iterations, the loss function value of the task corresponding to the large language model converging, and the update rate of the edges between nodes in the graph neural network being lower than a preset percentage.

[0108] Furthermore, embodiments of this application also provide a device for collaborative optimization of a vector database and a large language model, including: a processor, a memory, and a system bus;

[0109] The processor and the memory are connected via the system bus;

[0110] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described implementations of the vector database and large language model collaborative optimization method.

[0111] Furthermore, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to execute any of the above-described methods for the collaborative optimization of the vector database and the large language model.

[0112] Furthermore, this application embodiment also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described methods for the collaborative optimization of vector database and large language model.

[0113] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0114] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0115] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0116] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for collaborative optimization of vector databases and large language models, characterized in that, include: Acquire the target unstructured data; And extract data fragments from the target unstructured data; A graph neural network is constructed using data fragments of the target unstructured data; Furthermore, the semantic information and interdependencies of each node in the graph neural network are utilized to optimize the large language model LLM; The optimized Large Language Model (LLM) is used to encode each data segment, and the resulting encoded vectors are used to construct a vector database or update the vector values ​​of the corresponding data segments in the vector database. Based on the similarity between the vector values ​​of each data segment in the updated vector database, the semantic information and interdependencies of each node in the graph neural network are updated. By utilizing the semantic information and interdependencies of each node in the updated graph neural network, the optimization of the large language model LLM and subsequent steps are re-executed. This process is repeated iteratively and collaboratively to optimize the graph neural network, the large language model LLM, and the vector database until the preset stopping optimization condition is met, resulting in the optimized graph neural network, the large language model LLM, and the vector database.

2. The method according to claim 1, characterized in that, The target unstructured data includes text data; the extraction of data fragments from the target unstructured data includes: The text data is segmented using a word segmenter to obtain the characters, words, and phrases contained in the text data.

3. The method according to claim 1, characterized in that, The target unstructured data includes image data; the extraction of data segments from the target unstructured data includes: The image data is segmented using an image segmentation model to obtain image blocks contained in the image data.

4. The method according to claim 1, characterized in that, The target unstructured data includes audio data and / or video data; the extraction of data segments from the target unstructured data includes: The audio data and / or video data are segmented into single frames or N frames to obtain audio segments of the audio data and / or video segments of the video data; where N is a positive integer greater than 1.

5. The method according to claim 1, characterized in that, The step of constructing a graph neural network using data fragments of the target unstructured data includes: The co-occurrence frequency of each data segment in all target unstructured data corpora is counted, and the co-occurrence frequency is normalized and used as the weight value of the edge between each data segment in the graph neural network. Initial vectors containing semantic information for each data segment are generated based on the vector embedding method, and these vectors serve as nodes in the graph neural network.

6. The method according to claim 1, characterized in that, The step of updating the semantic information and interdependencies of each node in the graph neural network based on the similarity between the vector values ​​of each data segment in the updated vector database includes: Calculate the vector values ​​of each data segment in the updated vector database, and calculate the similarity between the vector values ​​of each data segment; Based on the similarity between the vector values ​​of the various data segments, the weight values ​​of the edges between the nodes in the graph neural network are updated, thereby updating the semantic information and interdependencies of the nodes in the graph neural network.

7. The method according to any one of claims 1-6, characterized in that, The preset stopping optimization conditions include at least one of the following: reaching a preset number of iterations, the loss function value of the task corresponding to the large language model converging, and the update rate of the edges between nodes in the graph neural network being lower than a preset percentage.

8. A device for collaborative optimization of vector database and large language model, characterized in that, include: Acquisition unit, used to acquire target unstructured data; And extract data fragments from the target unstructured data; The first optimization unit is used to construct a graph neural network using data fragments of the target unstructured data; and to optimize the large language model LLM using the semantic information and interdependencies of each node in the graph neural network. The first update unit is used to encode each data segment using the optimized Large Language Model (LLM) and to construct a vector database or update the vector values ​​of the corresponding data segments in the vector database using the obtained encoded vectors. The second update unit is used to update the semantic information and interdependencies of each node in the graph neural network based on the similarity between the vector values ​​of each data segment in the updated vector database. The second optimization unit is used to utilize the semantic information and interdependencies of each node in the updated graph neural network to re-execute the first optimization unit, the first update unit, and the second update unit, and so on, to iteratively and collaboratively optimize the graph neural network, the large language model LLM, and the vector database until the preset stopping optimization condition is met, thus obtaining the optimized graph neural network, the large language model LLM, and the vector database.

9. A device for collaborative optimization of vector databases and large language models, characterized in that, include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method described in any one of claims 1-7.

Images