Efficient and accurate query method for multimedia data and database system

By building a multimedia retrieval database and combining the RAG big model, a multi-level query method is adopted to solve the problem of insufficient efficiency and accuracy of multimedia data query in the existing technology, and efficient and accurate response to complex and fuzzy queries is achieved.

CN119493870APending Publication Date: 2025-02-21SHANGHAI SHIJIU INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411129418.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2025-02-21

AI Technical Summary

Technical Problem

The existing multimedia data query methods cannot efficiently and accurately handle users' complex and fuzzy query needs, especially in terms of multimodal data processing.

Method used

A multimedia retrieval database is constructed, combined with the Retrieval-Augmented Generation (RAG) large model, and a multi-level query method including precise query, correlation query and RAG vector query are adopted.

Benefits of technology

It realizes efficient and accurate response to user complex and fuzzy queries, improves query speed and accuracy, reduces response time, and is suitable for intelligent operations in the field of multimedia entertainment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119493870A_ABST
    Figure CN119493870A_ABST
Patent Text Reader

Abstract

The invention relates to the field of multimedia query. An efficient and accurate query method for multimedia data is characterized by comprising the following steps: firstly, constructing a retrieval database, and training a large model; the retrieval database comprises a precise query library, a relevancy query library and a vector library, and the large model adopts a retrieval enhancement generation model; secondly, during query, an input terminal monitors triggering of a wake-up word, obtains input of a user, carries out precise matching through a precise query library, directly outputs a result if a result is obtained through matching, carries out matching through a relevancy query library according to relevancy if the result is not obtained through matching, directly outputs the result if the result is obtained through matching, and directly outputs the result if the result is not obtained through matching. If yes, vectorization processing is conducted on user input content, the user input content is put into a vector database for preliminary retrieval and RAG vector retrieval, then a retrieval result is transmitted to a large model, retrieval enhanced data is generated through the large model, the extracted data is inquired again, and a result is output after inquiry.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of multimedia, and in particular to an efficient and accurate query method and database system for multimedia data. Background Art

[0002] With the growing demand for entertainment and the development of the entertainment field, the number and types of multimedia entertainment films are increasing, and users' demand for efficient and accurate film queries is also increasing. However, traditional methods mainly include keyword-based retrieval and metadata matching, which have many shortcomings. First, the keyword retrieval method relies on the precise vocabulary input by the user, but the user often cannot accurately describe the required content, resulting in unsatisfactory retrieval results. Secondly, keyword retrieval is insensitive to synonyms, near-synonyms and semantic fuzziness, and cannot effectively handle the complexity and diversity in natural language. Although metadata matching can provide some auxiliary information, the metadata annotation process is cumbersome and error-prone, and the quality of annotation directly affects the query results. In addition, these traditional methods have difficulty in processing the correlation between multimodal data such as video, audio and text, and cannot fully utilize the multidimensional characteristics of the film for comprehensive analysis, resulting in unsatisfactory query accuracy and effect. These shortcomings significantly limit the efficiency and satisfaction of users when querying multimedia entertainment films, and a more intelligent and efficient query method is urgently needed to overcome these shortcomings.

[0003] Nowadays, machine learning methods play an increasingly important role in the efficient and accurate query of multimedia entertainment films, but these methods also face many challenges and limitations. Traditional machine learning methods usually rely on manual feature engineering, which requires manual design and extraction of specific features, which is not only time-consuming and labor-intensive, but also prone to missing key features, resulting in limited model performance. In addition, most of these methods are only suitable for single-modal data processing, and have weak comprehensive processing capabilities for multimodal data, and cannot fully utilize the various information in the film, such as video, audio, and text.

[0004] When applied to media entertainment film queries, existing machine learning methods can often only process static, structured data, and have poor processing effects on dynamic, unstructured multimedia data, such as video frames, audio signals, etc. Although some methods try to combine deep learning technology to improve feature extraction and processing capabilities, the generalization and robustness of the model are still limited due to the lack of sufficient training data and computing resources. In addition, these methods often have difficulty in accurately understanding user intent when faced with fuzzy user queries or natural language queries, resulting in unsatisfactory query results. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an efficient and accurate query method and database system for multimedia data, so as to solve the technical problem that the existing query method cannot meet the needs of efficient and accurate movie query.

[0006] Technical Solution

[0007] An efficient and accurate query method for multimedia data, characterized by:

[0008] First, a retrieval database is constructed to train a large model; the retrieval database includes a precise query library, a relevance query library and a vector library, and the large model uses retrieval enhancement to generate a language model;

[0009] Secondly, when querying, the input terminal monitors the triggering of the wake-up word. After the user says or enters the wake-up word and the target speech, the user's input is obtained, and precise matching is first performed through the precise query library. If the result is matched, the result is directly output. If no result is matched, the relevance query library is used to match according to the relevance. If the result is matched, the result is directly output. If no result is matched, the user input content is vectorized and placed in the vector database for preliminary search, RAG vector search is performed, and the search results are passed and connected to the large model. The excellent understanding ability of the large model is used to extract complex user needs, generate search-enhanced data, and query the extracted data again, and output the results after the query.

[0010] Furthermore, the method for constructing the retrieval database includes: step 1-1, preparing data: collecting and organizing relevant data of multimedia such as various films, music, video resources, etc., and eliminating invalid and outdated parts of the data; step 1-2, data processing: cleaning data, removing special characters, punctuation marks, spaces and other irrelevant content, and unifying English content into lowercase; step 1-3, data pre-storage: constructing a data file in .csv format according to the data form for pre-storage; step 1-4, using the csv data file constructed in step 1-3 to construct a vector library, a relevance query library and a precision query library.

[0011] In steps 1-4, the methods for constructing the vector library, the relevance query library and the precision query library are respectively:

[0012] The steps of building a vector database include: firstly segmenting the data, vectorizing the segmented data and inserting it into the vector database, and then building an index to improve the search speed;

[0013] The steps of constructing a relevance query library include: first inserting pre-stored documents into the relevance query library, then configuring the inserted data, including a relevance calculation method, and then constructing an index to improve the search speed;

[0014] The steps of building a precise query library include: first inserting the pre-stored documents into the precise query library, and then building an index to improve the search speed.

[0015] Furthermore, the content found by the query is post-processed, and the results can be output for selection or the final query results can be passed to the playback module for selection and playback or direct playback.

[0016] Furthermore, when querying, after obtaining the user's input, the input data is preprocessed, the format of the preprocessed text is unified, and it is accurately matched with the precise query library to perform a precise query operation; for the content found by the precise query, subsequent selection and playback can be performed directly; for the content not found by the precise query, after unifying the format, a relevance matching query operation is performed.

[0017] When neither exact matching nor relevance matching can produce any results, the user input is vectorized and an attempt is made to find directly matching content in the vector database. The content retrieved from the vector database together with the user input is passed to the big model. The big model understands and analyzes the user input to extract the user's core intent and task entity. The big model generates relevant candidate queries or operation instructions based on the user's needs. Based on the operation instructions, the corresponding tool is called to search the media library. Based on the results of the media library search, a selection is made or subsequent selection and playback or direct playback operations are performed.

[0018] A retrieval database system for multimedia data is characterized in that it includes a vector library, a relevance query library and a precision query library. The three libraries also include indexes to improve the search speed. The retrieval database is obtained by collecting and collating relevant data of various multimedia such as movies, music, and video resources, and then constructing a data document in .csv format according to the data form for pre-storage. The data is then segmented respectively, and the segmented data is vectorized and inserted into the vector database; or the pre-stored document is inserted into the relevance query library, and then the inserted data is configured, including a relevance calculation method; or the pre-stored document is directly inserted into the precision query library, and then an index is constructed to improve the search speed.

[0019] The vector library is connected to the retrieval enhancement generation model, and the data after sorting or processing the results of the retrieval in the vector library is used as the input of the retrieval enhancement generation model. The retrieval enhancement generation model generates relevant candidate queries or operation instructions. According to the operation instructions, the corresponding tools are called to perform media resource library retrieval, and the playback module is connected to perform subsequent playback.

[0020] A storage medium stores a program, wherein the program implements any of the above methods when executed by a processor.

[0021] Beneficial Effects

[0022] The present invention constructs a multimedia retrieval database, combines the RAG large model, and adopts a multi-level query method including precise query, relevance query and RAG vector query, which can not only improve the response to users, especially complex and unclear inputs, but also achieve accurate response, and also achieve rapid response, increase response speed, and reduce response time. Especially for large-screen TV operating systems, it can achieve more intelligent operation without much computing power, more humane and cost control, and can help the large-screen field to achieve more intelligent operation without much hardware change. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the system structure of the present invention.

[0024] Figure 2 This is a schematic diagram of the retrieval database of the present invention without loading a large model.

[0025] Figure 3 It is a query flow chart of the present invention.

[0026] Figure 4 Schematic diagram of three database retrieval processes of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described below in conjunction with specific embodiments and drawings.

[0028] The multimedia entertainment film query method based on the RAG large model of the present invention is shown in Figure 1 , the main parts include retrieval, large model and input and output of query method. The specific process of the multimedia entertainment film query method based on the large model is as follows: first obtain the user's input, perform preliminary processing and then perform precise retrieval, and at the same time perform relevance retrieval on the input. If these two retrieval methods match the film data, the user can choose or play it directly. If these two retrieval methods cannot match the film data, the user's input content will be put into the vector library and retrieved using the RAG vector retrieval method, and loaded into the model's prompt words as the knowledge base of the large model. Through the analysis of the model and tool calls, the query format is matched and subsequent operation instructions are generated, and the relevant films are directly searched and selected in the media resource library, and finally the response is realized for playback. The media resource library is the media resource library of the system.

[0029] Figure 2 The database construction method and the multiple databases constructed for implementing the multimedia entertainment film query method based on the large model are shown. The database in this method is used for retrieval in the process of this method, and the process of building the database is independent of the process of the query method. Figure 2As shown, in this implementation method, the data preprocessing operation of the database includes preparing data, processing data and pre-storing. The database includes the following three construction methods:

[0030] First, the construction of the vector library:

[0031] Step 1.1: First, perform data segmentation operation on the pre-stored documents;

[0032] Step 1.2: Vectorize the segmented data and insert it into the vector database

[0033] Step 1.3: Build an index to improve search speed.

[0034] Secondly, the construction of the relevance query library:

[0035] In more detail, step 2 includes step 2.1, step 2.2 and step 2.3.

[0036] Step 2.1, first insert the pre-stored documents into the relevance query library;

[0037] Step 2.2: Configure the inserted data, including the correlation calculation method, etc.

[0038] Step 2.3: Build an index to improve search speed.

[0039] Again, the construction of the precise query library:

[0040] In more detail, step 3 includes step 3.1 and step 3.2.

[0041] Step 3.1: First, insert the pre-stored documents into the precise database;

[0042] Step 3.2: Build the index.

[0043] The constructed film retrieval database is based on the relevant features of the film and saves them to the local database to achieve classification management, recommendation and search of the film. Here, the relevant features of the film may include: the name of the film, category, director, cast list, release year, country, alias and content description information.

[0044] Figure 3 Show Figure 1The specific implementation method flow chart of the system structure block diagram. In the implementation method flow chart, the big model will solve the user input that cannot be found by relevance search and precise search. This type of user input includes unclear user input, vague requirements, ambiguous intentions and movie names, search requirements, chat requirements and other functions. This type of user input cannot obtain the corresponding movie through relevance search or precise search, and the big model has a very effective understanding and analysis ability for these user inputs. After extracting the intention and entity content, it will perform subsequent retrieval and playback operations.

[0045] Figure 4 Show Figure 1 The specific process of the three database retrieval methods of the system structure block diagram. This method includes three databases. According to the specific format of each database, there is a corresponding retrieval method. The specific retrieval method is as follows:

[0046] 1: Vector retrieval method, using the following steps:

[0047] Step 11: Vectorize the content to be retrieved to obtain vectorized input.

[0048] Step 12: Calculate and compare the vectorized input with the vectors in the vector library to obtain multiple vectors in the library with the closest vector distance.

[0049] Step 13: Put the queried vectors in the library into the generated model to obtain the retrieval results.

[0050] Step 14: Sort the results and set the threshold.

[0051] Step 15: Organize the results and export them to the large model.

[0052] 2: Relevance retrieval method, using the following steps:

[0053] Step 21: Preprocess the content to be retrieved and unify the format with the relevance query library;

[0054] Step 22: Search the content in the relevance query library to obtain more query results based on the relevance;

[0055] Step 23: Sort the query results, select the top items as the final results, or set a threshold to determine whether there is a matching content;

[0056] Step 24: Organize the results and output them.

[0057] 3: Accurate search method, using the following steps:

[0058] Step 31: Preprocess the content to be retrieved and unify the format with the precise query library:

[0059] Step 32: Search the content in the precise query library to obtain the query results;

[0060] Step 33: Organize the results and output them.

[0061] It can be seen from the above exemplary embodiments that the present application provides a multimedia entertainment film query method based on a large model. In the field of multimedia entertainment, the basic query method is combined with the capabilities of the large model to empower the user's audio and video entertainment with AI algorithms. Since the direct use of large models for querying has the disadvantages of high cost and time-consuming; in the method of the present application, the specially trained large model is combined with a uniquely designed precise retrieval process, the delay of the entire retrieval process is greatly reduced, and the real-time capability of the method of the present application is effectively improved.

[0062] The process of the overall query method includes the input terminal monitoring the triggering of the wake-up word, obtaining the user's input after the user says the wake-up word and the target speech; then processing and analyzing the user's input through the model, making corresponding judgments on different types of input, and performing video operations based on different judgments.

[0063] Specifically:

[0064] Step 1: Collect and organize multimedia data such as movies and music to build a target database.

[0065] Step 2: The input terminal monitors the triggering of the wake-up word, and obtains the user's actual input after the user says the wake-up word.

[0066] Step 3, based on the extraction of user input in steps 1 and 2 and the construction of the database, construct a precise data query method unique to the multimedia entertainment field. If the precise query method does not find the corresponding content, obtain the information closest to the user input from the database through correlation query.

[0067] Step 4: Based on the data provided in step 3, if the user's input can be directly matched to the content in the database, select or directly play according to the user's needs.

[0068] Step 5. Based on the data provided in step 3, if the user input cannot be directly matched to the content in the database, the user input is put into the vector database for preliminary search, and then the search results are passed to the big model. The user's complex needs are extracted through the excellent understanding ability of the big model, and the extracted needs are then selected or played.

[0069] Step 6: Based on the information provided in step 4 or step 5, the query result is post-processed, and the final query result is passed to the playback module for selection or playback.

[0070] The above step 1 also includes:

[0071] Step 1.1, collect the required data and remove invalid and outdated data;

[0072] Step 1-2: Clean the data, remove special characters, punctuation marks, spaces and other irrelevant content, and convert the English content to lowercase;

[0073] Step 1-3, construct a data document in .csv format according to the data form for pre-storage;

[0074] Step 1-4, use the csv data file constructed in step 1-3 to build a vector library;

[0075] Step 1-5, use the csv data document constructed in step 1-3 to build a relevance query library;

[0076] Step 1-6, use the csv data document built in step 1-3 to build a precise query library;

[0077] The above steps 1-4 also include:

[0078] Step 1-4-1: Vectorize the segmented data and insert it into the vector database

[0079] Step 1-4-2: Build an index to improve search speed;

[0080] The above steps 1-5 also include:

[0081] Step 1-5-1, first insert the pre-stored documents into the relevance query library;

[0082] Step 1-5-2: Configure the inserted data, including the correlation calculation method, etc.

[0083] Step 1-5-3: Build an index to improve search speed;

[0084] The above steps 1-6 also include:

[0085] Step 1-6-1: First, insert the pre-stored documents into the precise query library;

[0086] Step 1-6-2: Build an index to improve search speed;

[0087] The above step 3 also includes:

[0088] Step 3-1, prepare the user's input, perform cleaning and preprocessing, such as removing special characters, punctuation marks, spaces and other irrelevant content, and unifying English content into lowercase;

[0089] Step 3-2: unify the format of the preprocessed text and perform precise query operations.

[0090] Step 3-3: For the content found by precise query, subsequent selection and playback can be carried out directly.

[0091] Step 3-4: For the content that is not found by the precise search, unify the format and perform a relevance query.

[0092] The above step 5 also includes:

[0093] Step 5-1: vectorize the user input. The processing flow includes segmenting the text, generating vectors through the embedding model, storing the vectors, building an index, etc., and trying to find direct matching content in the vector database;

[0094] Step 5-2, passing the content queried by the user in the vector database together with the user input to the large model;

[0095] Step 5-3: The large model understands and analyzes the user input and extracts the user's core intent and task entity;

[0096] Step 5-4, the big model generates relevant candidate queries or operation instructions based on the user's needs;

[0097] Step 5-5: According to the operation instructions, call the corresponding tool to search the media resource library.

[0098] Step 5-6: Select or directly play based on the results of the media resource library search.

[0099] Compared with the prior art, the efficient and accurate query method for multimedia entertainment films based on a large model of the present application has the following advantages:

[0100] 1) A special RAG method suitable for the multimedia entertainment field. In the multimedia entertainment field, the RAG method has unique advantages. RAG (Retrieval-Augmented Generation) is called retrieval-augmented generation. RAG combines the two major technologies of retrieval and generation. By retrieving relevant information from a large-scale knowledge base and combining it with a generation model for content generation, it achieves efficient and accurate generation of recommended content. In step 5, RAG technology is used to build a vector library, which is connected to a large model and combined with a large language model to effectively improve the accuracy and diversity of the content. The RAG large model has powerful understanding and generation capabilities, and can provide high-quality background information and creative suggestions. By combining with the RAG method, creators can achieve more attractive recommendations in the multimedia entertainment field, greatly improving the user experience.

[0101] 2) The exclusive trained RAG large model solves the problem of semantic generalization. The large model-based efficient and accurate query method for multimedia entertainment films in this application effectively solves these problems by training a large model dedicated to the field of multimedia entertainment. In this application, the exclusive vector library constructed and the large model that is targeted for various query scenarios in this field based on multimedia corpus have powerful semantic understanding capabilities and can accurately parse the intentions and needs of user input. Compared with traditional generalization models, this large model is specially optimized for the field of multimedia entertainment and can recognize and process unique vocabulary and expressions related to this field. Therefore, when the user inputs vague or unclear information, the model can accurately locate the user's actual needs through its rich semantic understanding capabilities and contextual analysis. In addition, the specially trained large model not only solves the problem of semantic generalization, but also enhances the richness and diversity of recommended content.

[0102] 3) Accurate matching. The high-efficiency and accurate query method for multimedia entertainment films based on a large model of the present application is unique in terms of search query and can achieve very accurate query matching. By adopting a case-by-case decision-making approach, the full field matching method and the RAG method are integrated for decision-making, which greatly improves the understanding and analysis of the generalized context while ensuring the speed.

[0103] 4) Response speed at the millisecond level. The high-efficiency and accurate query method for multimedia entertainment films based on a large model of the present application performs well in terms of response speed, reaching a response time at the millisecond level. The method of the present application has been specially optimized and debugged, which can greatly shorten the generation time of recommended content. The millisecond-level response speed not only improves user satisfaction, but also enables the system to maintain efficient and stable performance when processing a large number of user requests, thereby significantly improving the practicality of the system and user experience.

[0104] 5) Low cost. The efficient and accurate query method for multimedia entertainment films based on large models of the present application also has significant advantages in terms of cost. First, the RAG method reduces the dependence on expensive hardware equipment by combining retrieval and generation technologies, greatly reducing hardware costs. Secondly, the specially trained low-cost large model has powerful computing and processing capabilities, and can complete the processing and analysis of large-scale data in a relatively short time, reducing the consumption of computing resources and operating costs.

[0105] Example 1

[0106] The method and system of the present invention are used to perform a query corresponding to a large-screen TV, for example:

[0107] In some scenarios, users can quickly search for "Shanghai Bund". The system can provide results within 5-10ms based on precise matching and relevance matching, including "Shanghai Bund" movie, "Shanghai Bund" TV (Hong Kong version), "Shanghai Bund" TV (mainland version), and "Shanghai Bund" episode.

[0108] In some scenarios, some users may only ask for the search terms Shanghai, Zhou xx (actor name), Feng xx (role name). Based on precise matching and relevance matching, the system may query results such as Shanghai Bund - Zhou xx (actor name), Zhou xx (actor name) Shanghai news, variety shows, etc. If vector retrieval and search enhancement generation are used, the output may be: Movies about Shanghai starring Zhou xx (actor name), with Feng xx (role name) as the main character... Then the corresponding selection to play is output, allowing the user to select or play directly.

[0109] After testing, if the large model is used directly, the response time is about 500ms, while the method and system of the present invention are used, the average response time is reduced by 90% to 95%, that is, the fusion system and method of the present invention that integrates precise query, relevance query and RAG vector query are used, and the average response time is below 50ms. Without significantly improving the hardware configuration, the intelligence level of the large-screen operating system is greatly improved, and the response speed is not affected. It can be promoted as soon as possible to improve the user experience.

[0110] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An efficient and accurate query method for multimedia data, characterized by: First, a retrieval database is constructed to train a large model; the retrieval database includes a precise query library, a relevance query library and a vector library, and the large model uses retrieval enhancement to generate a language model; Secondly, when querying, the input terminal monitors the triggering of the wake-up word. After the user says or enters the wake-up word and the target speech, the user's input is obtained, and precise matching is first performed through the precise query library. If the result is matched, the result is directly output. If no result is matched, the relevance query library is used to match according to the relevance. If the result is matched, the result is directly output. If no result is matched, the user input content is vectorized and placed in the vector database for preliminary search, RAG vector search is performed, and the search results are passed and connected to the large model. The excellent understanding ability of the large model is used to extract complex user needs, generate search-enhanced data, and query the extracted data again, and output the results after the query.

2. The efficient and accurate query method for multimedia data according to claim 1, characterized in that: The method for constructing the retrieval database includes: step 1-1, preparing data: collecting and organizing relevant data of multimedia such as various types of films, music, video resources, etc., and eliminating invalid and outdated parts of the data, wherein the relevant data includes the name, category, director, cast list, release year, country, alias and content description information of the film; step 1-2, data processing: cleaning data, removing special characters, punctuation marks, spaces and other irrelevant content, and unifying the content in English into lowercase; step 1-3, data pre-storage: constructing a data file in .csv format according to the data form for pre-storage; step 1-4, using the csv data file constructed in step 1-3 to construct a vector library, a relevance query library and a precision query library.

3. The efficient and accurate query method for multimedia data according to claim 2, characterized in that: In steps 1-4, the methods for constructing the vector library, the relevance query library and the precision query library are respectively: The steps of building a vector database include: firstly segmenting the data, vectorizing the segmented data and inserting it into the vector database, and then building an index to improve the search speed; The steps of constructing a relevance query library include: first inserting pre-stored documents into the relevance query library, then configuring the inserted data, including a relevance calculation method, and then constructing an index to improve the search speed; The steps of building a precise query library include: first inserting the pre-stored documents into the precise query library, and then building an index to improve the search speed.

4. The efficient and accurate query method for multimedia data according to claim 1, characterized in that: The content found by the query is post-processed and the results can be output for selection or the final query results can be passed to the playback module for selection and playback or direct playback.

5. The efficient and accurate query method for multimedia data according to claim 1, characterized in that: When querying, after obtaining the user's input, the input data is cleaned and preprocessed, the format of the preprocessed text is unified, and it is accurately matched with the precise query library to perform precise query operations; For the content found by precise query, subsequent selection and playback can be carried out directly; For content that is not found by precise search, the format is unified and then a relevance matching query is performed.

6. The efficient and accurate query method for multimedia data according to claim 1 or 5, characterized in that: When neither exact matching nor relevance matching can produce any results, the user input is vectorized and an attempt is made to find directly matching content in the vector library. The content retrieved from the vector library together with the user input is passed to the big model. The big model understands and analyzes the user input to extract the user's core intent and task entity. The big model generates relevant candidate queries or operation instructions based on the user's needs. Based on the operation instructions, the corresponding tool is called to search the media library. Based on the results of the media library search, a selection is made or subsequent selection and playback or direct playback operations are performed.

7. A multimedia data retrieval database system, characterized in that: It includes a vector library, a relevance query library and a precision query library. All three libraries also contain indexes to improve the search speed. The retrieval database collects and organizes relevant data of various multimedia such as movies, music, video resources, and then constructs a data document in .csv format according to the data form for pre-storage. The data is then segmented separately, and the segmented data is vectorized and inserted into the vector library; at the same time, the pre-stored documents are inserted into the relevance query library, and then the inserted data is configured, including the relevance calculation method; at the same time, the pre-stored documents are directly inserted into the precision query library, and then the index is constructed to improve the search speed.

8. The search database system according to claim 7, characterized in that: The vector library is connected to the retrieval enhancement generation model, and the data after sorting or processing the results of the retrieval in the vector library is used as the input of the retrieval enhancement generation model. The retrieval enhancement generation model generates relevant candidate queries or operation instructions. According to the operation instructions, the corresponding tools are called to perform media resource library retrieval, and the playback module is connected to perform subsequent playback.

9. A storage medium storing a program, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.