A search method and system based on heterogeneous data sources

Abstract

The application relates to the technical field of data search, and provides a search method and system based on a heterogeneous data source, which comprises the following steps: collecting a heterogeneous data source, preprocessing the collected heterogeneous data source, and obtaining heterogeneous data; pretraining a Transformer model by using the heterogeneous data, and obtaining a heterogeneous data source model; converting the heterogeneous data obtained by preprocessing into original vector data by using the obtained heterogeneous data source model; inserting the original vector data into a vector database, and constructing an index for the inserted original vector data; and matching corresponding original vector data in the vector database for the query input of the heterogeneous data source model through the index. The search method and system based on the heterogeneous data source can realize multi-data source fusion and mutual assistance data retrieval between different data sources.

Description

Technical Field

[0001] This invention relates to the field of data search technology, and in particular to a search method and system based on heterogeneous data sources. Background Technology

[0002] VisualBERT is a Transformer-based multimodal model that combines visual and linguistic information to achieve joint understanding and processing of images and text. It leverages the self-attention mechanism of the Transformer model to process both image features and text sequences, enabling inference and generation for multimodal tasks. In VisualBERT, image features and text sequences are input into a visual encoder and a language encoder, respectively. The visual encoder transforms image features into a series of visual vectors, while the language encoder transforms text sequences into a series of semantic vectors. These vectors represent high-level feature representations of the input visual and linguistic information.

[0003] However, VisualBERT is limited to processing text and images. In practical applications, existing retrieval systems perform isolated searches for different data sources, at most only integrating text and images—for example, retrieving text from text and images from images, or vice versa, but not retrieving text from images. Because real-world data sources are diverse, VisualBERT cannot effectively handle other complex data sources such as unstructured data like speech, tables, and graphs. This prevents data sources like speech, tables, and graphs (composed of nodes and edges) from being integrated into existing retrieval systems.

[0004] Therefore, how to provide a method for integrating multiple data sources and retrieving data between different data sources has become an urgent technical problem to be solved. Summary of the Invention

[0005] In view of this, in order to overcome the shortcomings of the prior art, the present invention aims to provide a search method and system based on heterogeneous data sources.

[0006] According to a first aspect of the present invention, a search method based on heterogeneous data sources is provided, comprising:

[0007] Collect heterogeneous data sources, preprocess the collected heterogeneous data sources, and obtain heterogeneous data;

[0008] The Transformer model is pre-trained using heterogeneous data to obtain a heterogeneous data source model;

[0009] The obtained heterogeneous data source model is used to convert the preprocessed heterogeneous data into raw vector data.

[0010] Insert the original vector data into the vector database and build an index for the inserted original vector data;

[0011] The query input for the heterogeneous data source model is indexed to match the corresponding original vector data in the vector database.

[0012] Preferably, in the search method based on heterogeneous data sources of the present invention, heterogeneous data sources are collected, and the collected heterogeneous data sources are preprocessed to obtain heterogeneous data, including: collecting text data sources, table data sources, graph data sources, audio data sources and image data sources, and preprocessing the collected text data sources, table data sources, graph data sources, audio data sources and image data sources respectively to obtain the corresponding heterogeneous data.

[0013] Preferably, in the search method based on heterogeneous data sources of the present invention, heterogeneous data sources are collected, and the collected heterogeneous data sources are preprocessed to obtain heterogeneous data, including:

[0014] Collect text data sources, refine the granular processing of the collected text data sources to obtain semantic information of the text data, and use the obtained semantic information and the collected text data sources as text data;

[0015] Collect tabular data sources, use the table headers as keys and the table content as values, perform key-value pair transformation on the collected tabular data sources, and use the obtained tabular key-value pair data as tabular data.

[0016] Collect graph data sources, convert the collected graph data sources into node sequences, and use the converted node sequences as graph data;

[0017] The process involves collecting audio data sources, converting the format of the collected audio data sources, extracting features from the converted audio data sources, and using the converted audio data sources and the extracted audio features as audio data.

[0018] The process involves collecting image data sources, converting the format of the collected image data sources, extracting features from the converted image data sources, and using the converted image data sources and the extracted image features as image data.

[0019] Preferably, in the search method based on heterogeneous data sources of the present invention, the refined granular processing includes part-of-speech tagging, named entity recognition, and dependency parsing.

[0020] Preferably, in the search method based on heterogeneous data sources of the present invention, the process of collecting heterogeneous data sources, preprocessing the collected heterogeneous data sources to obtain heterogeneous data further includes: performing privacy data filtering and deduplication processing on the collected heterogeneous data sources.

[0021] Preferably, in the search method based on heterogeneous data sources of the present invention, the Transformer model is pre-trained using heterogeneous data to obtain a heterogeneous data source model, including:

[0022] The Transformer model was pre-trained using text data to obtain the first pre-trained model.

[0023] The first pre-trained model is pre-trained using image data to obtain the second pre-trained model;

[0024] The second pre-trained model is pre-trained using tabular and graphical data. Text data is then added during the pre-training process of the second pre-trained model using tabular and graphical data to obtain the third pre-trained model.

[0025] The third pre-trained model is pre-trained using audio data. Text data is added during the pre-training process of the third pre-trained model using audio data to obtain a heterogeneous data source model.

[0026] Preferably, in the search method based on heterogeneous data sources of the present invention, the preprocessed heterogeneous data is converted into original vector data through the obtained heterogeneous data source model, including: converting the preprocessed heterogeneous data into original vector data in the same vector space through the heterogeneous data source model.

[0027] Preferably, in the search method based on heterogeneous data sources of the present invention, inserting the original vector data into a vector database and constructing an index for the inserted original vector data includes:

[0028] Insert the raw vector data into the vector database, configure the index type for the inserted raw vector data, and configure the corresponding index parameters for the configured index type;

[0029] Based on the configured index type and index parameters, an index is built on the inserted original vector data;

[0030] Store the data structures and metadata required for the constructed index, and create a loading interface for the constructed index.

[0031] Preferably, in the search method based on heterogeneous data sources of the present invention, the query input of the heterogeneous data source model is matched with the corresponding original vector data in the vector database through indexing, including:

[0032] By inputting query data into a heterogeneous data source model, the input query data is converted into query vector data;

[0033] By indexing the original vector data, the transformed query vector data is compared with the original vector data in the vector database in terms of similarity.

[0034] When the similarity between the queried vector data and the original vector data in the vector database is higher than a preset value, the original vector data is output as the query result.

[0035] According to a second aspect of the present invention, a search system based on heterogeneous data sources is provided. The system includes a search server for collecting heterogeneous data sources, preprocessing the collected heterogeneous data sources to obtain heterogeneous data; pre-training a Transformer model using the heterogeneous data to obtain a heterogeneous data source model; converting the preprocessed heterogeneous data into raw vector data using the obtained heterogeneous data source model; inserting the raw vector data into a vector database and constructing an index for the inserted raw vector data; and matching the corresponding raw vector data in the vector database with the query input of the heterogeneous data source model using the index.

[0036] According to a third aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect of the present invention.

[0037] This invention relates to a search method and system based on heterogeneous data sources. By preprocessing the collected heterogeneous data sources, the Transformer model is pre-trained using the preprocessed heterogeneous data to obtain a heterogeneous data source model. The obtained heterogeneous data source model is then used to convert the preprocessed heterogeneous data into raw vector data, and an index is built for the raw vector data. The index is then used as the query input for the heterogeneous data source model to match the corresponding raw vector data in the vector database. This enables data retrieval that integrates multiple data sources and allows for mutual assistance between different data sources. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of a system for a search method based on heterogeneous data sources applicable to embodiments of the present invention;

[0040] Figure 2 This is a flowchart illustrating the steps of a search method based on heterogeneous data sources according to an embodiment of the present invention.

[0041] Figure 3 This is an example diagram illustrating the execution of a search method based on heterogeneous data sources according to an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of the structure of the device provided by the present invention. Detailed Implementation

[0043] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0044] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0045] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0046] Figure 1 An exemplary system for a search method based on heterogeneous data sources, applicable to embodiments of the present invention, is shown. For example... Figure 1 As shown, the system may include a search server 101, a communication network 102, and / or one or more search clients 103. Figure 1 The example in the text is multiple search clients 103.

[0047] Search server 101 can be any suitable server used to store information, data, programs, and / or any other suitable type of content. In some embodiments, search server 101 can perform appropriate functions. For example, in some embodiments, search server 101 can be used to perform data searches based on heterogeneous data sources. As an optional example, in some embodiments, search server 101 can be used to implement data search queries by constructing a heterogeneous data source model. For example, search server 101 can be used to collect heterogeneous data sources, preprocess the collected heterogeneous data sources to obtain heterogeneous data; pre-train a Transformer model using the heterogeneous data to obtain a heterogeneous data source model; convert the preprocessed heterogeneous data into raw vector data using the obtained heterogeneous data source model; insert the raw vector data into a vector database and build an index for the inserted raw vector data; and match the query input of the heterogeneous data source model with the corresponding raw vector data in the vector database using the index.

[0048] As another example, in some embodiments, the search server 101 may send a search method based on a heterogeneous data source to the search client 103 for the user to use, according to a request from the search client 103.

[0049] As an optional example, in some embodiments, the search client 103 is used to provide a visual search interface for receiving a user's selection input operation based on a heterogeneous data source, and for obtaining and displaying the search interface corresponding to the option selected by the selection input operation from the search server 101 in response to the selection input operation. The search interface displays at least the information on the search based on the heterogeneous data source and the operation options for the information on the search based on the heterogeneous data source.

[0050] In some embodiments, communication network 102 may be any suitable combination of one or more wired and / or wireless networks. For example, communication network 102 may include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. Search client 103 may connect to communication network 102 via one or more communication links (e.g., communication link 104), which may be linked to search server 101 via one or more communication links (e.g., communication link 105). Communication links may be any communication link suitable for transmitting data between search client 103 and search server 101, such as network links, dial-up links, wireless links, hardwired links, any other suitable communication links, or any suitable combination of such links.

[0051] Search client 103 may include any one or more clients that present an interface related to searches based on heterogeneous data sources in an appropriate form for user use and operation. In some embodiments, search client 103 may include any suitable type of device. For example, in some embodiments, search client 103 may include mobile devices, tablets, laptops, desktop computers, and / or any other suitable type of client device.

[0052] Although search server 101 is illustrated as a single device, in some embodiments, any suitable number of devices may be used to perform the functions performed by search server 101. For example, in some embodiments, multiple devices may be used to implement the functions performed by search server 101. Alternatively, cloud services may be used to implement the functions of search server 101.

[0053] Based on the above system, this invention provides a search method based on heterogeneous data sources, which will be described in the following embodiments.

[0054] Figure 2 This is a flowchart illustrating the steps of a search method based on a heterogeneous data source according to an embodiment of the present invention. The search method based on a heterogeneous data source in this embodiment can be executed on a search server, such as... Figure 2 As shown, this search method based on heterogeneous data sources includes the following steps:

[0055] Step S201: Collect heterogeneous data sources, preprocess the collected heterogeneous data sources, and obtain heterogeneous data.

[0056] Figure 3 This is an example diagram illustrating the execution of a search method based on heterogeneous data sources according to an embodiment of the present invention. Figure 3 As shown, as an optional example, the method of this embodiment of the invention collects text data sources, table data sources, graph data sources, audio data sources and image data sources, and preprocesses the collected text data sources, table data sources, graph data sources, audio data sources and image data sources respectively to obtain corresponding heterogeneous data.

[0057] As an optional example, the method of this embodiment of the invention collects heterogeneous data sources in the following manner, preprocesses the collected heterogeneous data sources, and obtains heterogeneous data:

[0058] The method involves collecting text data sources and refining the granularity of these data to obtain semantic information. This semantic information, along with the collected text data sources, forms the final text data. It's important to note that this embodiment employs part-of-speech tagging, named entity recognition, and dependency parsing to refine the granularity of the collected text data sources. Those skilled in the art can apply other suitable granularity processing to the collected text data sources based on specific application scenarios when implementing this method; this embodiment does not impose any limitations on this approach.

[0059] The process involves collecting data from a tabular data source, using the table headers as keys and the table content as values, performing key-value pair transformations on the collected data, and then using the resulting key-value pair data as the table data.

[0060] The method involves collecting a graph data source, converting it into a node sequence, and then using this node sequence as graph data. A graph data source is a data source composed of edges and nodes. In this embodiment, the collected graph data source is converted into a node sequence using a machine learning algorithm for learning node representations in graph data, such as the DeepWalk algorithm. DeepWalk is a machine learning algorithm for learning node representations in graph data. It is based on the concept of random walks in graphs, treating nodes as words and random walk sequences as sentences, thereby converting graph data into a representation similar to word vectors in natural language processing. Those skilled in the art can choose other suitable methods or algorithms to convert the collected graph data source into a node sequence according to the actual scenario when implementing the method of this invention; this embodiment does not impose any limitations on this.

[0061] The process involves acquiring audio data sources, converting the acquired audio data sources into their formats, extracting features from the converted audio data sources, and finally using the converted audio data sources and the extracted audio features as audio data.

[0062] The process involves collecting image data sources, converting the format of the collected image data sources, extracting features from the converted image data sources, and using the converted image data sources and the extracted image features as image data.

[0063] It should be noted that the method in this embodiment selects appropriate format conversion and feature extraction methods based on the actual application scenario. To ensure data security, privacy, and accuracy, this embodiment can also perform privacy data filtering and deduplication on the collected heterogeneous data sources.

[0064] Step S202: Pre-train the Transformer model using heterogeneous data to obtain a heterogeneous data source model.

[0065] As an optional example, the method of this embodiment of the invention uses text data to pre-train a Transformer model to obtain a first pre-trained model; uses image data to pre-train the first pre-trained model to obtain a second pre-trained model; uses tabular data and graph data to pre-train the second pre-trained model, and adds text data during the process of using tabular data and graph data to pre-train the second pre-trained model to obtain a third pre-trained model; uses audio data to pre-train the third pre-trained model, and adds text data during the process of using audio data to pre-train the third pre-trained model to obtain a heterogeneous data source model.

[0066] For example, the method in this embodiment first pre-trains the Transformer model using text data to obtain a first pre-trained model. To ensure the first pre-trained model converges quickly, this embodiment initializes the first pre-trained model with the model parameters obtained during the process of obtaining the first pre-trained model. Then, it pre-trains the first pre-trained model using image data to obtain a second pre-trained model. To avoid instability caused by a large data source span during training, this embodiment randomly adds text data to the tabular and graph data to initialize the second pre-trained model with the model parameters obtained during the process of obtaining the second pre-trained model. The second pre-trained model is then pre-trained to obtain a third pre-trained model. To avoid the adverse effects of audio data on model stability, this embodiment adds text data during the pre-training of the third pre-trained model using audio data to initialize the third pre-trained model with the model parameters obtained during the process of obtaining the third pre-trained model. The third pre-trained model is then pre-trained to obtain a final pre-trained model that integrates multiple heterogeneous data sources. It should be noted that in the method of this embodiment, the proportion of text data added to tabular and graphical data, as well as the proportion of text data added to audio data, can be selected according to the actual application scenario, and this embodiment does not impose any restrictions on this.

[0067] Step S203: Convert the preprocessed heterogeneous data into raw vector data using the obtained heterogeneous data source model.

[0068] As an optional example, the method of this embodiment converts preprocessed heterogeneous data into original vector data in the same vector space through a heterogeneous data source model. For example, this embodiment uses a self-supervised learning model to vectorize the data. In practical applications, in addition to self-supervised learning models, the method of this embodiment can also use supervised learning-based methods for data vectorization. This involves labeling the data and using supervised learning algorithms to train the model, thereby providing a more accurate vector representation.

[0069] Step S204: Insert the original vector data into the vector database and build an index for the inserted original vector data.

[0070] As an optional example, the method of this embodiment inserts raw vector data into a vector database, configures an index type for the inserted raw vector data, configures corresponding index parameters for the configured index type, constructs an index on the inserted raw vector data according to the configured index type and index parameters, stores the data structure and metadata required for the constructed index, and creates a loading interface for the constructed index.

[0071] For example, the method in this embodiment can use the Milvus vector database for index construction. Specifically, this embodiment inserts the original vector data into the Milvus vector database, and then configures the index type and index parameters for the inserted original vector data, including but not limited to Exact Index and Approximate Nearest Neighbor Index. Configuring the index type and index parameters can improve the efficiency of subsequent data searches. Next, this embodiment begins the index construction operation, building an index on the inserted original vector data according to the configured index type and index parameters, and calculating the data structure and metadata required for the index through the Milvus vector database, storing them in memory or on disk. After the index construction is completed, an index loading interface is created through the Milvus vector database, and the created index loading interface is loaded during subsequent searches using the index name. Those skilled in the art can choose a suitable vector database to build the index for the original vector data according to the actual application scenario when implementing the method of this invention; this embodiment does not impose any limitations on this.

[0072] It should be noted that the method of this embodiment can also create other index structures to accelerate data search and query, including but not limited to inverted indexes, B-trees, hash tables, and other index structures. Those skilled in the art can choose to construct a suitable index structure based on the data volume, search requirements, and performance requirements when implementing the method of this invention; this embodiment does not impose any limitations in this regard.

[0073] Step S205: Match the original vector data in the vector database with the query input of the heterogeneous data source model using the index.

[0074] As an optional example, the method of this embodiment of the invention inputs query data into a heterogeneous data source model, converts the input query data into query vector data, and compares the similarity of the converted query vector data with the original vector data in a vector database using the index of the original vector data. When the similarity between the query vector data and the original vector data in the vector database is higher than a preset value, the original vector data is output as the query result. For example, in this embodiment of the method, the user-input query data is input into a subsequent data source model to obtain the corresponding query vector data representation. Then, the obtained query vector data is compared with the original vector data in a vector database, such as the Milvus vector database, and the similarity is sorted. The original vector data with the highest similarity is output as the query result.

[0075] Furthermore, in addition to providing a unified search query method, the method of this invention can also provide targeted search query parsing indexes and interfaces for different types of data sources, thereby providing users with more refined query functions and a better user experience based on the characteristics of data types and query requirements.

[0076] This invention relates to a search method and system based on heterogeneous data sources. By preprocessing the collected heterogeneous data sources, the Transformer model is pre-trained using the preprocessed heterogeneous data to obtain a heterogeneous data source model. The obtained heterogeneous data source model is then used to convert the preprocessed heterogeneous data into raw vector data, and an index is built for the raw vector data. The index is then used as the query input for the heterogeneous data source model to match the corresponding raw vector data in the vector database. This enables data retrieval that integrates multiple data sources and allows for mutual assistance between different data sources.

[0077] like Figure 4 As shown, the present invention also provides a device including a processor 310, a communication interface 320, a memory 330 for storing a processor-executable computer program, and a communication bus 340. The processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 implements the aforementioned search method based on heterogeneous data sources by running the executable computer program.

[0078] The computer program in memory 330, when implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0079] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected based on actual needs to achieve the purpose of this embodiment. Those skilled in the art can understand and implement this without any creative effort.

[0080] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0081] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A search method based on heterogeneous data sources, characterized in that, The method includes: Collect heterogeneous data sources, preprocess the collected heterogeneous data sources, and obtain heterogeneous data; The Transformer model is pre-trained using heterogeneous data to obtain a heterogeneous data source model; The obtained heterogeneous data source model is used to convert the preprocessed heterogeneous data into raw vector data. Insert the original vector data into the vector database and build an index for the inserted original vector data; The query input for the heterogeneous data source model is indexed to match the corresponding original vector data in the vector database; The Transformer model is pre-trained using heterogeneous data to obtain heterogeneous data source models, including: The Transformer model was pre-trained using text data to obtain the first pre-trained model. The first pre-trained model is pre-trained using image data to obtain the second pre-trained model; The second pre-trained model is pre-trained using tabular and graphical data. Text data is then added during the pre-training process of the second pre-trained model using tabular and graphical data to obtain the third pre-trained model. The third pre-trained model is pre-trained using audio data. Text data is added during the pre-training process of the third pre-trained model using audio data to obtain a heterogeneous data source model.

2. The search method based on heterogeneous data sources according to claim 1, characterized in that, Collect heterogeneous data sources and preprocess them to obtain heterogeneous data. These include: collecting text data sources, table data sources, graph data sources, audio data sources, and image data sources. Preprocess the collected text data sources, table data sources, graph data sources, audio data sources, and image data sources respectively to obtain the corresponding heterogeneous data.

3. The search method based on heterogeneous data sources according to claim 1, characterized in that, Collect heterogeneous data sources, preprocess the collected heterogeneous data sources to obtain heterogeneous data, including: Collect text data sources, refine the granular processing of the collected text data sources to obtain semantic information of the text data, and use the obtained semantic information and the collected text data sources as text data; Collect tabular data sources, using the table headers as keys and the table content as values, perform key-value pair transformation on the collected tabular data sources, and use the obtained tabular key-value pair data as tabular data; Collect graph data sources, convert the collected graph data sources into node sequences, and use the converted node sequences as graph data; The process involves collecting audio data sources, converting the format of the collected audio data sources, extracting features from the converted audio data sources, and using the converted audio data sources and the extracted audio features as audio data. The process involves collecting image data sources, converting the format of the collected image data sources, extracting features from the converted image data sources, and using the converted image data sources and the extracted image features as image data.

4. The search method based on heterogeneous data sources according to claim 3, characterized in that, The refined granularity processing includes part-of-speech tagging, named entity recognition, and dependency parsing.

5. The search method based on heterogeneous data sources according to claim 3, characterized in that, The process includes collecting heterogeneous data sources, preprocessing the collected heterogeneous data sources to obtain heterogeneous data, and also performing privacy data filtering and deduplication on the collected heterogeneous data sources.

6. The search method based on heterogeneous data sources according to claim 1, characterized in that, The heterogeneous data obtained from preprocessing is converted into raw vector data through the obtained heterogeneous data source model, including: converting the heterogeneous data obtained from preprocessing into raw vector data in the same vector space through the heterogeneous data source model.

7. The search method based on heterogeneous data sources according to claim 1, characterized in that, Insert the raw vector data into the vector database and build an index for the inserted raw vector data, including: Insert the raw vector data into the vector database, configure the index type for the inserted raw vector data, and configure the corresponding index parameters for the configured index type; Based on the configured index type and index parameters, an index is built on the inserted original vector data; Store the data structures and metadata required for the constructed index, and create a loading interface for the constructed index.

8. The search method based on heterogeneous data sources according to claim 1, characterized in that, By indexing the query input of the heterogeneous data source model, the corresponding original vector data in the vector database is matched, including: By inputting query data into a heterogeneous data source model, the input query data is converted into query vector data; By indexing the original vector data, the transformed query vector data is compared with the original vector data in the vector database in terms of similarity. When the similarity between the queried vector data and the original vector data in the vector database is higher than a preset value, the original vector data is output as the query result.

9. A search system based on heterogeneous data sources, characterized in that, The system includes a search server, which is used to collect heterogeneous data sources, preprocess the collected heterogeneous data sources, and obtain heterogeneous data. The Transformer model is pre-trained using heterogeneous data to obtain a heterogeneous data source model; The obtained heterogeneous data source model converts the preprocessed heterogeneous data into raw vector data; the raw vector data is inserted into the vector database, and an index is built for the inserted raw vector data; the index is used to match the corresponding raw vector data in the vector database as the query input of the heterogeneous data source model; the heterogeneous data is used to pre-train the Transformer model to obtain the heterogeneous data source model, including: using text data to pre-train the Transformer model to obtain the first pre-trained model; The first pre-trained model is pre-trained using image data to obtain the second pre-trained model; the second pre-trained model is pre-trained using tabular and graph data, and text data is added during the pre-training process of the second pre-trained model using tabular and graph data to obtain the third pre-trained model; the third pre-trained model is pre-trained using audio data, and text data is added during the pre-training process of the third pre-trained model using audio data to obtain the heterogeneous data source model.

Images