HYBRID VECTOR DATABASE: COMBINED VISUAL AND TEXTUAL SEARCH METHOD

Abstract

The invention relates to a method for retrieving requests from users with high performance from a vector database via multiple searches. More specifically, the invention relates to a method for searching with inputs that identify the relevant content as both visual and textual, thereby ranking the most similar content in a hybrid vector database.

Description

1 TARIFF VISUAL AND TEXTUAL DATA FROM HYBRID VECTOR DATABASE COMBINED SEARCH METHOD Technical Area The invention handles requests from users by searching multiple times in a vector database. It relates to a method that proposes high-performance retrieval. The invention, more specifically, relates to the content in both visual and textual form. By searching with defining inputs, the most similar hybrid vector is found in the database. It relates to a method of sorting content. Previous Tech 10 In current applications, content containing visual and textual data is analyzed. When the study was conducted, only textual searches were possible. Content containing visual data was also searchable. The search was conducted using the texts provided. Field experts used similar methods. by looking at the formats of the content containing textual phrases, they can determine if they are similar. It used to be the decision-making process. However, thanks to the artificial intelligence technologies that have developed in recent years, 15 visual narrators who can produce textual expressions of the figures included in the content These visual narrators have been developed for searching texts in the content pool. It is used. However, this type of search is not fully automated either. A vector dataset containing embedded vectors of the images found in the content. Searching the database may not be effective in all cases. Because there are 20 visual forms. Since depth and color information is not available, feature extraction and the process of extracting these features are performed. Generating embedded vectors does not always guarantee a high success rate. The image is included in document CN116563859A, which is part of the known state of the art. a method using visual embedding vectors for searches It is mentioned. 25 2 In document number WO2020255912A1, which is included in the known state of the art. a method that uses a text embedding vector for textual searches It is mentioned. Current methods used in textual and visual searches have various shortcomings and It shows shortcomings. In shape-based search methods, the shapes are different 5 Drawing them at different angles or scales makes it difficult to capture the similarity. Additionally, feature extraction and embedding from shapes that do not contain depth and color information. Vector generation does not always guarantee a high success rate. This situation, making it difficult to accurately and consistently represent shapes, thus preventing the process from being fast enough. This prevents it from being effective. 10 Text-based search methods analyze the typed text and the information provided by the people performing the search. It can yield varying results depending on the differences. Expressing shapes in texts. The fact that explanations are not always available reduces the effectiveness of this method. In addition, document formats containing similar textual expressions are analyzed by field experts. It needs to be examined, and since this process is not automated, it takes time and effort. 15 It requires a vector dataset containing the embedded vectors of the images. Searching the database may not be effective in all cases. For these reasons, Current methods do not provide the desired accuracy and effectiveness in visual and textual searches. It is insufficient in providing this. A new method to eliminate the disadvantages mentioned above 20 The need for its development has arisen. Purposes of the Invention The purpose of this invention is to provide organizations and individuals with the ability to perform visual and textual searches. It is about implementing an effective search method. 3 Another purpose of this invention is to create a visual, image-embedding feature not found in the literature. a vector data vector containing the vector and the embedded vector of the text expression of the image It is the implementation of a method that includes a base. Another purpose of this invention is to improve the MRR (Mean Time Ratio) of visual and text search results. Combining the reciprocal rank method with the high performance of both searches (5) a method that provides a ranking that reveals It is the realization of. Detailed Description of the Invention The computer-based method used to achieve the purpose of this invention is attached. This is shown in the figures. 10 These shapes; Figure 1: Schematic view of the hybrid vector database. Figure 2: Schematic view of the flowchart of the method. The parts shown in the figures are individually numbered, and these numbers... The corresponding answers are given below. 15 1. Hybrid vector database Image 2 3. Image of the embedded model. 4. Text embedding model 5. Text 20 6. Image embedding vector 7. Text embedding vector 8th Ranking 4 The invention involves inputs defined as both visual and textual content for the related content. By searching, the most similar content in the hybrid vector database (1) is ranked. is related to a method, - visual embedding model (3) with visual embedding (2) of images Generating vectors (6), 5 - the generated visual embedding vector (6), in the hybrid vector database (1) cosine similarity of the embedding vectors of the relevant images Comparison and ranking of results using these methods, - the visuals (2), the visual meaning model and the text (5) correspondence production, 10 - generated texts (5), text embedding model (4) with text embedding the production of vectors (7), - the generated text embedding vector (7), in the hybrid vector database (1) using cosine similarity with text embedding vectors Comparison and ranking of results, 15 - text embedding vector (7) and visual embedding vector (6) The rankings obtained as a result of comparisons, ranking (8) In this section, a single MRR (Mean Reciprocal Rank) is used. ranking It includes the following steps. 20 With advancements in artificial intelligence methods, intelligent information retrieval is becoming widespread. It is becoming a field of study. Vector databases are used in studies in this field. By recording the embedding vectors of the data using the embedding model. Vector databases are created. Although they differ in some aspects... although it usually includes text and embedding vectors of the related text and 25 Quick semantic searching of user-entered input. This allows us to identify the most similar ones based on different distance functions. It is possible for them to be found in a sequential order. Multi-model vector databases, which have become widespread in recent years, accept input as data. Saving text, image, and audio data using embedding vectors. The hybrid vector database proposed in the invention (1) provides the relevant image (2) 2 different embedding vectors (image embedding vector (6) and text embedding vector (7) emphasizes that it is necessary to record them at the same time. This innovative 5 This approach enables hybrid exploration. In the first stage of the computer-based method, the subject of the invention is the images (2) and text embedding vectors (5) (visual embedding vector (6) and text embedding vector (7)) is created. Here the visual embedding model (3) and text embedding model (4) is used. Similarly, to make sense of the shapes of the contents 10 Texts are generated using a large language model, and these texts are embedded. Vectors are created. The generated text embedding vectors (7) are visual. The hybrid vector database (1) is saved together with the embedding vectors (6). In the hybrid search step, first the image entered by the user (2), image embedding vector (6) is generated. The generated visual embedding vector (6) is hybrid vector data 15 cosine similarity with the embedding vectors of the relevant images in the base (1) They are compared and ranked using these methods. In the second step of the hybrid search, the image provided by the user (2), image Texts (5) are produced using the meaning-making model. The texts produced (5), The text embedding model (4) is converted into text embedding vectors (7). The 20 generated text embedding vectors (7), text embedding in hybrid vector database (1) The vectors are compared using cosine similarity, and the generated results are analyzed. They are arranged in order. Cosine similarity is shown in Formula 1. 𝑆𝐶(𝐴, 𝐵): = 𝑐𝑜𝑠(𝜃) = A ⋅ B ‖𝐴‖‖𝐵‖ = ∑ ⬚𝑛𝑖=1 𝐴𝑖𝐵𝑖 √∑ ⬚𝑛𝑖=1 𝐴𝑖 2√∑ ⬚𝑛𝑖=1 𝐵𝑖 2 Formula 1 25 6 Comparison of text embedding vector (7) and image embedding vector (6) The rankings obtained as a result are for obtaining suggestions and information in section (8) of the ranking. MRR (Mean Reciprocal Rank) is a metric that evaluates the quality of systems. This method combines two different inputs into a single final ranking. a structure that enables more effective identification of results most similar to the type 5 This is how the MRR (Mean Reciprocal Rank) is formed. It is shown in Formula 2. 𝑀𝑅𝑅 = 1 N ∑ ⬚𝑁𝑖=1 1 𝑟𝑎𝑛𝑘𝑖 Formula 2

Claims

7 REQUESTS 1. The invention is defined as containing both visual and textual content for the related subject. By searching with inputs, the most similar (1) in the hybrid vector database is found. It relates to a method of sorting content, - visual embedding model (3) with visual embedding (2) visual embedding 5 the production of vectors (6), - the generated visual embedding vector (6), in the hybrid vector database (1) cosine similarity of the embedding vectors of the relevant images Comparison and ranking of results using these methods, - the visuals (2), the visual meaning model and the text (5) correspondence 10 production, - generated texts (5), text embedding model (4) with text embedding the production of vectors (7), - the generated text embedding vector (7), in the hybrid vector database (1) using cosine similarity with text embedding vectors 15 Comparison and ranking of results, - text embedding vector (7) and visual embedding vector (6) The rankings obtained as a result of comparisons, ranking (8) In this section, a single MRR (Mean Reciprocal Rank) is used. Ranking achieved: 20 It is characterized by including its steps.