Compression method, system and device of multi-modal weight file and storage medium

Abstract

The invention discloses a compression method, system and device for a multi-modal weight file and a storage medium, which are used for improving the accuracy of information retrieval. The method comprises the steps that based on a query keyword input by a user, a first query document set is retrieved and acquired; performing correlation analysis on the first query document set to obtain a similar score corresponding to each query document; determining inferior documents with scores smaller than a preset similar threshold in the similar score set, and rejecting the inferior documents from the first query document set to obtain a second query document set; key information is extracted from each query document in the second query document set, corresponding abstract texts are generated based on the key information, and an abstract text set is obtained; performing redundant information elimination on each abstract text in the abstract text set to obtain a target abstract text set; sorting the abstract texts in the target abstract text set; and inputting the sorted abstract text set into the multi-modal large model.

Description

Technical Field

[0001] The present application belongs to the field of information processing technology, and specifically relates to a compression method, system, device and storage medium for a multimodal weight file. Background Art

[0002] Traditional information retrieval methods mainly rely on keyword matching, but in actual applications, a large number of documents or fragments may partially match the query keywords, but many of the contents may be far from the actual needs of the user. In addition, directly extracting useful information from massive data, especially for those data containing multiple modalities (such as text, images, audio, etc.), places extremely high demands on computing resources and processing speed.

[0003] In the prior art, when a user enters a query, a large number of matching documents or fragments are usually returned. However, these documents or fragments often contain a large amount of information that is irrelevant to the user's actual needs or has a low degree of relevance. The existence of this information not only increases the difficulty for users to filter information, but also greatly wastes computing resources, thereby reducing the accuracy and efficiency of information retrieval.

[0004] Based on this, the present application proposes a compression method for a multimodal weight file to solve the above technical problems. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a compression method, system, device and storage medium for a multimodal weight file, which are used to improve the accuracy of information retrieval.

[0006] The technical solution provided in this application is described below: The first aspect of the present application provides a method for compressing a multimodal weight file, comprising: Based on the query keyword input by the user, retrieve and obtain a first query document set, wherein the first query document set includes a plurality of documents or fragments matching the query keyword; Performing correlation analysis on each query document in the first query document set to obtain a similarity score corresponding to each query document and obtain a similarity score set, wherein the similarity score indicates the degree of association between the query document and the query keyword; Determine low-quality documents in the similarity score set whose scores are less than a preset similarity threshold, and remove the low-quality documents from the first query document set to obtain a second query document set; Extracting key information from each query document in the second query document set, and generating summary text corresponding to each query document based on the key information to obtain a summary text set; Extracting keywords from the summary text to form a keyword list; According to the keyword list, redundant information is removed from each summary text in the summary text set, and a conversion is performed into a preset format to obtain a target summary text set; Sorting each summary text in the target summary text set based on a preset sorting rule; The sorted summary text set is input into the multimodal large model, so that the multimodal large model generates knowledge corresponding to the query keywords according to the sorting information.

[0007] Optionally, after sorting the summary texts in the target summary text set based on a preset sorting rule and before inputting the sorted summary text set into the multimodal macro model, the compression method further includes: Acquire a knowledge link between the sorted summary text set and the second query document set, wherein the knowledge link is used by the multimodal macro model to determine a query document corresponding to the summary text according to the summary text; The step of inputting the sorted summary text set into the multimodal macro model includes: The sorted summary texts and their corresponding knowledge links and keywords are input into the multimodal large model.

[0008] Optionally, performing relevance analysis on each query document in the first query document set includes: Calculate the similarity score between the query keyword and each query document using the BM25 algorithm; The BM25 algorithm uses the following formula: Formula ① Formula ② Among them, IDF (q i ) is the query word q i The rare value in the query document set, N is the total number of query documents in the query document set, df(q i ) is the number of query documents containing the query keyword qi, Q is the query keyword set, D is the query document, n is the number of query terms, f(q i ,D) is the frequency of the query keyword qi appearing in the query document D, k1 and b are empirical values ​​set manually, dl is the length of the query document D, and avgdl is the average length of all query documents in the query document set.

[0009] Optionally, the sorting of the summary texts in the target summary text set based on a preset sorting rule includes: Calculate similarity scores for each summary text in the target summary text set; Sort each summary text in descending order based on the calculated similarity score.

[0010] Optionally, removing redundant information from each summary text in the summary text set includes: Redundant information that is not semantically relevant to the keyword is determined in each summary text based on the semantic similarity judgment rule, and the redundant information is removed.

[0011] Optionally, determining redundant information that is semantically irrelevant to the keyword in each summary text based on a semantic similarity judgment rule includes: Vectorizing the text segments in each summary text to obtain multiple text vectors; Vectorize each keyword to obtain a keyword vector; Calculating the angle between the text vector and the keyword vector by using cosine similarity; Determine the target text segment whose angle value is less than the preset similarity threshold, and remove the target text segment from the summary text.

[0012] Optionally, the query keyword based on user input includes: The query keywords are preprocessed, and the preprocessing includes removing stop words, correcting spellings, and replacing synonyms.

[0013] A second aspect of the present application provides a compression system for a multimodal weight file, comprising: A retrieval unit, configured to retrieve and obtain a first query document set based on a query keyword input by a user, wherein the first query document set includes a plurality of documents or fragments matching the query keyword; an analysis unit, configured to perform correlation analysis on each query document in the first query document set, obtain a similarity score corresponding to each query document, and obtain a similarity score set, wherein the similarity score indicates a correlation degree between the query document and the query keyword; A first elimination unit, configured to determine low-quality documents in the similarity score set whose scores are less than a preset similarity threshold, and eliminate the low-quality documents from the first query document set to obtain a second query document set; A generating unit, configured to extract key information from each query document in the second query document set, and generate a summary text corresponding to each query document based on the key information to obtain a summary text set; An extraction unit, used for extracting keywords from the summary text to form a keyword list; A second elimination unit is used to eliminate redundant information from each summary text in the summary text set according to the keyword list, and convert it into a preset format to obtain a target summary text set; A sorting unit, used to sort each summary text in the target summary text set based on a preset sorting rule; An input unit is used to input the sorted summary text set into the multimodal large model, so that the multimodal large model generates knowledge corresponding to the query keyword according to the sorting information.

[0014] Optionally, the compression system further comprises: an acquisition unit, configured to acquire a knowledge link between the sorted summary text set and the second query document set, wherein the knowledge link is used by the multimodal macro model to determine a corresponding query document according to the summary text; The input unit is specifically used to input the sorted summary texts and the corresponding knowledge links and keywords into the multimodal large model.

[0015] Optionally, the analysis unit includes: An analysis module, used to calculate a similarity score between the query keyword and each query document using a BM25 algorithm; The BM25 algorithm uses the following formula: Formula ① Formula ② Among them, IDF (q i ) is the query word q i The rare value in the query document set, N is the total number of query documents in the query document set, df(q i ) is the number of query documents containing the query keyword qi, Q is the query keyword set, D is the query document, n is the number of query terms, f(q i ,D) is the frequency of the query keyword qi appearing in the query document D, k1 and b are empirical values ​​set manually, dl is the length of the query document D, and avgdl is the average length of all query documents in the query document set.

[0016] Optionally, the sorting unit includes: A calculation module, used for calculating similarity scores of each summary text in the target summary text set; The sorting module is used to sort each summary text from high to low according to the calculated similarity score.

[0017] Optionally, the second elimination unit includes: The determination module is used to determine redundant information that is not semantically relevant to the keyword in each summary text based on a semantic similarity judgment rule, and remove the redundant information.

[0018] Optionally, the determining module includes: A first processing submodule is used to vectorize the text segments in each summary text to obtain multiple text vectors; The second processing submodule is used to vectorize each keyword to obtain a keyword vector; A calculation submodule, used for calculating the angle value between the text vector and the keyword vector by using cosine similarity; The determination submodule is used to determine the target text segment corresponding to the angle value being smaller than a preset similarity threshold, and remove the target text segment from the summary text.

[0019] Optionally, the retrieval unit includes: The processing module is used to pre-process the query keywords, wherein the pre-processing includes removing stop words, correcting spellings, and replacing synonyms.

[0020] A third aspect of the present application provides a device for compressing a multimodal weight file, comprising: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the first aspect and the optional compression method of any one of the first aspect.

[0021] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional compression method in the first aspect.

[0022] It can be seen from the above technical solutions that this application has the following advantages: First, based on the query keywords input by the user, a first query document set containing multiple documents or fragments matching the query keywords is retrieved and obtained; then, a correlation analysis is performed on each query document to obtain a similarity score set representing the degree of association between the query document and the query keywords; by setting a preset similarity threshold, those inferior documents with low association can be effectively eliminated, thereby obtaining a more refined second query document set. On this basis, key information is further extracted from each query document, and a corresponding summary text is generated; by extracting keywords from the summary text, a keyword list can be formed, which provides an important reference for subsequent information processing and redundant information elimination; finally, the summary text is sorted according to the preset sorting rules, and the sorted summary text set is input into the multimodal large model. The multimodal large model can first extract knowledge from the query document corresponding to the top-ranked summary text according to the sorting information. The higher the query document is ranked, the more relevant it is to the query keyword input by the user, so it can provide users with more accurate and useful information, improve the relevance and quality of the context, and thus provide more reliable semantic support for the final generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. It should be understood that the accompanying drawings in the following description are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0024] Figure 1 It is a schematic diagram of an embodiment of a method for compressing a multimodal weight file of the present application; Figure 2 It is a schematic diagram of another embodiment of a method for compressing a multimodal weight file of the present application; Figure 3 It is a schematic diagram of another embodiment of a method for compressing a multimodal weight file of the present application; Figure 4 It is a schematic diagram of an embodiment of a compression system for a multimodal weight file of the present application; Figure 5 is a schematic diagram of another embodiment of a compression system for a multimodal weight file of the present application; Figure 6 It is a schematic diagram of an embodiment of a compression device for a multimodal weight file of the present application. DETAILED DESCRIPTION

[0025] It should be noted that a compression method for a multi-modal weight file provided by this application can be applied to a terminal, a system, or a server. For the convenience of description, in this application, the system is taken as the execution entity for illustration.

[0026] Please refer to Figure 1 , this application provides an embodiment of a compression method for a weight file, and this embodiment includes: 101. Based on the query keywords input by the user, retrieve and obtain a first query document set, where the first query document set includes multiple documents or fragments that match the query keywords.

[0027] The keywords can be a phrase, a sentence, or even a complex query expression. The system parses these keywords, understands the user's query intention, and retrieves accordingly.

[0028] The system searches for documents or fragments that match the query keywords in a preset document database, and can specifically use methods such as full-text retrieval, keyword matching, and semantic analysis for searching. The preset document database can be an academic literature database, a web database, a government public information database, an enterprise database, etc.

[0029] Based on the query keywords, retrieve the content related to the query keywords in each database. This content is called the first query document. For example, in database A, 3 documents related to the query keywords are retrieved, and each of these 3 documents is a query document. The first query document set is composed of these 3 query documents.

[0030] For the retrieved documents, if only part of the content in the document is related to the query keywords, semantic analysis can be used to extract this fragment, and also extract the fragments with extended meanings related to this fragment, and independently form a query document.

[0031] In this step, the query keywords also need to be preprocessed, and the preprocessing includes removing stop words, spelling correction, and synonym replacement.

[0032] Among them, stop words include words that frequently appear in documents but contribute little to the understanding of the document content, such as Chinese words like "的", "了", "在", etc., and English words like "the", "is", "of", etc. Specifically, a stop word list can be constructed in advance. After receiving the query keywords input by the user, the system will match these query keywords with the stop word list, and for the matched stop words, the system will remove them from the query keywords (without affecting the overall meaning).

[0033] 102. Perform correlation analysis on each query document in the first query document set to obtain a similarity score corresponding to each query document and obtain a similarity score set, wherein the similarity score indicates the degree of association between the query document and the query keyword.

[0034] There are multiple query documents in the first query document set. Each query document is individually analyzed for relevance with the query keywords input by the user. Specifically, features can be extracted from each query document. The features can be text units such as words, phrases, N-grams, or higher-level features such as the structure and theme of the document. After extracting the features, a text similarity algorithm is used to calculate the similarity score between each query document and the query keywords. The similarity algorithm can be cosine similarity, Jaccard similarity, etc. based on a vector space model, or a more complex semantic matching model.

[0035] Based on the calculated similarity score, each query document can be sorted for the first time. The higher the similarity score, the higher the degree of match with the query keyword, and the higher the ranking position.

[0036] When the BM25 algorithm is used to calculate the similarity score, the following formula is used: Formula ① Formula ② In the formula, IDF (q i ) is the query word q i The rare value in the query document set, N is the total number of query documents in the query document set, df(q i ) is the number of query documents containing the query keyword qi, Q is the query keyword set, D is the query document, n is the number of query terms, f(q i ,D) is the frequency of the query keyword qi appearing in the query document D, k1 and b are empirical values ​​set manually, dl is the length of the query document D, and avgdl is the average length of all query documents in the query document set.

[0037] Through correlation analysis, we can identify which parts of the query document have the greatest impact on the compression effect. During the compression process, we can understand which parts have the greatest impact on the weight distribution, so as to make a more reasonable weight distribution. Finally, it can also help screen out the documents that best match the query keywords.

[0038] 103. Determine low-quality documents in the similarity score set whose scores are less than a preset similarity threshold, and remove the low-quality documents from the first query document set to obtain a second query document set.

[0039] The selection of similarity threshold is based on empirical data, historical statistics, or the needs of specific application scenarios. For example, in information retrieval, a lower threshold is set to obtain a wider range of background information.

[0040] Specifically, the selection of the threshold should also consider the characteristics of the compression method, and a higher threshold should be set to ensure that the filtered documents contain enough key information, retaining the important information in the query document while sacrificing some details.

[0041] After obtaining the similarity score of each document, the system will compare the score with the preset similarity threshold. Documents with scores below the threshold are considered "low-quality", which means that the documents are less relevant to the query keywords or contain less important information.

[0042] When identifying low-quality documents, the system adds the time factor as a removal condition. Based on the time involved in the query keywords, it determines some long-standing historical document content (such as the timestamp in the document), and although the similarity score exceeds the threshold, it is also low. At this time, the document content can also be removed. For example, the query keyword is: the latest scientific and technological developments, and the goal is to find the latest research results related to "the application of artificial intelligence in the medical field". In this case, the timestamp of the document is particularly important, so the relatively old document content can be removed. Specifically, if the two documents found both discuss the application of artificial intelligence in the medical field, but one is an old article from a few years ago and the other was recently published, since the recently published article contains the latest research results and technological advances, the old article can be removed, or the content of the old article can be removed.

[0043] For certain queries with high timeliness requirements, such as news event tracking and market trend analysis, timestamps become a key factor in screening documents. Outdated information may no longer have reference value and may even mislead users.

[0044] After removing low-quality documents, the remaining documents or fragments with a higher degree of relevance to the query keywords in multiple query documents are called the second query document set. The content in the query documents at this time is also more likely to contain information that the user is interested in.

[0045] 104. Extract key information from each query document in the second query document set, and generate summary text corresponding to each query document based on the key information to obtain a summary text set.

[0046] The most critical information is extracted from each query document and presented in a brief and concise manner, which not only saves storage space but also improves the efficiency of information retrieval.

[0047] Specifically, the extracted key information mainly includes topic sentences, keywords, main ideas, etc. The topic sentence is the sentence that expresses the main idea or central thought in the document; keywords are words that frequently appear in the query document and are crucial to understanding the content of the query document. The extracted keywords not only help the system understand the subject of the document, but also can serve as an important part of the summary text to help users quickly understand the main content of the document; the main ideas are extracted from the query document. During the extraction process, the query document content can be deeply understood and analyzed using techniques such as semantic analysis and sentiment analysis.

[0048] After extracting key information from each query document, a summary text is generated for each query document. The formation of the summary text is based on the extracted key information. Furthermore, first, a suitable summary template is selected according to the type, field and purpose of the query document. The summary template contains the title, topic sentence, keyword list, main point overview and other parts; secondly, the key information extracted from a single query document is integrated and reorganized, the topic sentences, keywords and main points are arranged in a logical order, and necessary conjunctions and transition sentences are added to enhance the readability of the text.

[0049] 105. Extract keywords from the summary text to form a keyword list.

[0050] In the process of extracting keywords, the frequency of occurrence of each word in the summary text can be counted, and the words with higher frequency of occurrence can be selected as keywords. These keywords can be collected to form a keyword list. There is a correlation between the keyword list and the summary text and the query document.

[0051] 106. According to the keyword list, redundant information is removed from each summary text in the summary text set, and the summary text is converted into a preset format to obtain a target summary text set.

[0052] The purpose of eliminating redundant information is to remove information that is not directly related to the keywords and repeated information in the summary text. Based on this, in this step, the keyword list is used to identify and eliminate sentences or paragraphs that are not directly related to the keywords in the summary text. This process can be achieved by calculating the similarity between the sentence or paragraph and the keyword list. The lower the similarity, the lower the degree of association. Combined with the semantic analysis method, sentences or paragraphs that are not directly semantically related to the keywords are identified and eliminated, so as to more accurately capture the semantic relationship between words and more accurately eliminate redundant information.

[0053] After removing redundant information, the summary text also needs to be converted into a preset format to meet the format required by the subsequent multimodal large model. The preset format includes a specific text structure, markup language (such as XML, JSON, etc.) or data format (such as table, database, etc.).

[0054] After the redundant information is removed and the format is converted, it means that the summary text has been optimized. At this time, the remaining summary text set is called the target summary text set.

[0055] 107. Sort each summary text in the target summary text set based on a preset sorting rule.

[0056] The sorting rules can be based on relevance or importance, or a combination of the two; among them, relevance indicates the degree of match between the summary text and the query keywords (user needs), and the summary text with a higher degree of match will be ranked higher; importance indicates the degree of contribution of the information contained in the summary text to the user's decision or understanding, and many summary texts are sorted in order of importance.

[0057] The higher the summary text is ranked, the more closely its content matches / is related to the user's query keywords. During the sorting process, the similarity between the summary text and the query keywords can be calculated, and the summary text with higher similarity can be ranked at the front. The calculation adopts algorithms such as cosine similarity and Jaccard similarity.

[0058] 108. Input the sorted summary text set into the multimodal big model, so that the multimodal big model generates knowledge corresponding to the query keywords according to the sorting information.

[0059] Large multimodal models such as BERT, GPT, ViLBERT, UNITER, etc.

[0060] Each summary text is sorted, and in the subsequent multimodal large model, knowledge answers can be given based on the query documents corresponding to the top-ranked summary texts. For example, for question D, three query documents A, B, and C are retrieved, corresponding to the optimized three summary texts a, b, and c, which are sorted as c, a, and b. When question D is input into the multimodal large model, the multimodal large model will first search for summary text c, and will first select the answer from the query document C corresponding to summary text c to answer question D, and output the knowledge answer to question D in the query document to the user.

[0061] In one feasible method, after sorting the summary texts, the knowledge links between the sorted summary text set and the second query document set are obtained. The knowledge links are used by the multimodal big model to determine the query documents corresponding to the summary texts. At this time, the multimodal big model can quickly find the corresponding original query documents according to the knowledge links corresponding to the summary texts, so that the multimodal big model can answer the relevant content of the query keywords according to the original query documents.

[0062] In this embodiment, firstly, based on the user query keywords, a matching first query document set is retrieved, then the relevance of the first query document is analyzed, a similarity score is obtained and a threshold is set, and low-relevance query documents are eliminated to form a refined second query document set. Next, key information is extracted for each query document to generate a summary text, and a keyword list is constructed. Finally, the summary text is sorted according to preset rules and input into a multimodal large model, so that the multimodal large model can preferentially extract knowledge from documents corresponding to high-ranked summaries, thereby being able to provide users with more accurate and useful information, improve the relevance and quality of the context, and provide more reliable semantic support for the final generation.

[0063] In the process of sorting the summary text, the following method is used: See also Figure 2 , the present application provides an embodiment of a sorting process in a weight file compression method, the embodiment comprising: 201. Calculate similarity scores for each summary text in the target summary text set.

[0064] 202. Sort each summary text in descending order according to the calculated similarity score.

[0065] In this embodiment, the similarity score between each summary text and the query keyword input by the user is calculated. Specifically, the keywords are first extracted again from each summary text, and then the weight of each keyword in the summary text is calculated (calculated by TF-IDF (term frequency-inverse document frequency) or BM25 algorithm). The higher the weight value, the higher the frequency of occurrence of the keyword in the summary text; finally, the keywords in the summary text are traversed to check whether these keywords match the original query keywords. For each matching keyword, the similarity score is calculated according to its weight value, and the similarity scores of all matching keywords are added to obtain the similarity score corresponding to each summary text. Based on the size of this similarity score, the summary texts are sorted in descending order.

[0066] In the process of removing redundant information from the summary text, firstly, redundant information that is not semantically relevant to the keyword is determined in each summary text based on the semantic similarity judgment rule, and then the determined redundant information is removed from the summary text. The specific implementation is described as follows: See also Figure 3 , the present application provides an embodiment of highlighting redundant information in a weight file compression method, the embodiment comprising: 301. Vectorize the text fragments in each summary text to obtain multiple text vectors.

[0067] Text fragments include sentences, phrases or words. Before vectorization, the summary text is first preprocessed, such as word segmentation, stop word removal, stemming or word form restoration, to extract meaningful text fragments.

[0068] Then use a pre-trained language model (such as Word2Vec, BERT, GPT, etc.) or a specific vector space model (such as TF-IDF, LSA, etc.) to convert the text fragment into a text vector.

[0069] 302. Perform vectorization processing on each keyword to obtain a keyword vector.

[0070] In this step, the keywords are processed such as removing special characters and converting to lowercase. The process of converting the vectors is similar to the vector conversion in step 301 and will not be described in detail here.

[0071] 303. Calculate the angle between the text vector and the keyword vector using the cosine similarity method.

[0072] In this step, the cosine similarity formula (also known as the cosine angle formula) is used to calculate the similarity between the text vector and the keyword vector. Specifically, the similarity value is obtained by calculating the dot product of the two vectors and dividing it by the product of their moduli. The value range of cosine similarity is [-1, 1]. The closer the value is to 1, the more similar the two vectors are (the more similar the keywords and the text fragments are); the closer the value is to -1, the less similar the two vectors are (the less similar the keywords and the text fragments are); when the value is 0, it means that the two vectors are orthogonal (that is, there is no correlation).

[0073] The calculation formula is:

[0074] in, Represents the text vector corresponding to the text fragment, represents the keyword vector corresponding to the keyword, represents the modulus of the text vector, Indicates the modulus of the keyword vector.

[0075] 304. Determine a target text segment corresponding to an angle value smaller than a preset similarity threshold, and remove the target text segment from the summary text.

[0076] In this step, based on the preset similarity threshold, determine which text segments are not semantically relevant to the keyword. Irrelevant segments are called redundant information, and these redundant information are removed from the summary text. The similarity threshold is set manually. When removing, traverse the similarity values ​​between all text vectors and keyword vectors, and regard those text segments with similarity values ​​less than the preset threshold as redundant information and remove them from the summary text. After removing the redundant information, update the content of the summary text to ensure that the remaining information is highly relevant to the keyword.

[0077] See also Figure 4 , the present application provides an embodiment of a compression system for a weight file, the embodiment comprising: A retrieval unit 401 is used to retrieve and obtain a first query document set based on a query keyword input by a user, where the first query document set includes a plurality of documents or fragments matching the query keyword; An analysis unit 402 is used to perform relevance analysis on each query document in the first query document set, obtain a similarity score corresponding to each query document, and obtain a similarity score set, wherein the similarity score indicates the degree of association between the query document and the query keyword; A first elimination unit 403 is used to determine low-quality documents with scores less than a preset similarity threshold in the similarity score set, and eliminate the low-quality documents from the first query document set to obtain a second query document set; A generating unit 404 is used to extract key information from each query document in the second query document set, and generate a summary text corresponding to each query document based on the key information to obtain a summary text set; An extraction unit 405 is used to extract keywords from the summary text to form a keyword list; The second elimination unit 406 is used to eliminate redundant information from each summary text in the summary text set according to the keyword list, and convert it into a preset format to obtain a target summary text set; A sorting unit 407, used to sort each summary text in the target summary text set based on a preset sorting rule; The input unit 408 is used to input the sorted summary text set into the multimodal large model, so that the multimodal large model generates knowledge corresponding to the query keywords according to the sorting information.

[0078] In this embodiment, firstly, based on the query keyword input by the user, a retrieval unit 401 retrieves and obtains a document or fragment containing multiple documents or fragments matching the query keyword, and these documents or fragments are called a first query document set; then, the analysis unit 402 performs a correlation analysis on each query document in the first query document set, obtains a similarity score corresponding to each query document, and obtains a similarity score set, wherein the similarity score indicates the degree of association between the query document and the query keyword; then, the first elimination unit 403 determines the inferior documents in the similarity score set whose scores are less than a preset similarity threshold, and eliminates the inferior documents from the first query document set, and obtains a second query document set; subsequently, the generation unit 404 extracts key information from each query document in the second query document set, and generates a summary text corresponding to each query document based on the key information, and obtains a summary text set; then, the extraction unit 405 extracts the key information from the query document in the second query document set, and generates a summary text corresponding to each query document based on the key information, and obtains a summary text set; Keywords in the summary text form a keyword list, and then the second elimination unit 406 eliminates redundant information from each summary text in the summary text set according to the keyword list, and converts it into a preset format to obtain a target summary text set; finally, the sorting unit 407 and the input unit 408 sort each summary text in the target summary text set based on the preset sorting rules, and input the sorted summary text set into the multimodal large model, so that the multimodal large model generates knowledge corresponding to the query keywords according to the sorting information. Therefore, the multimodal large model can first extract knowledge from the query documents corresponding to the top-ranked summary texts according to the sorting information. The higher the query document is ranked, the more relevant it is to the query keyword input by the user. Therefore, it can provide users with more accurate and useful information, improve the relevance and quality of the context, and provide more reliable semantic support for the final generation.

[0079] See also Figure 5 , the present application provides an embodiment of a compression system for a weight file, the embodiment comprising: A retrieval unit 501 is used to retrieve and obtain a first query document set based on a query keyword input by a user, where the first query document set includes a plurality of documents or fragments matching the query keyword; The retrieval unit 501 includes: Processing module 5011, used for preprocessing the query keywords, the preprocessing includes removing stop words, spelling correction and synonym replacement; An analysis unit 502 is used to perform relevance analysis on each query document in the first query document set, obtain a similarity score corresponding to each query document, and obtain a similarity score set, wherein the similarity score indicates the degree of association between the query document and the query keyword; The analysis unit 502 includes: An analysis module 5021 is used to calculate a similarity score between the query keyword and each query document using a BM25 algorithm; The BM25 algorithm uses the following formula: Formula ① Formula ② Among them, IDF (q i ) is the query word q i The rare value in the query document set, N is the total number of query documents in the query document set, df(q i ) is the number of query documents containing the query keyword qi, Q is the query keyword set, D is the query document, n is the number of query terms, f(q i ,D) is the frequency of the query keyword qi appearing in the query document D, k1 and b are empirical values, set manually, dl is the length of the query document D, and avgdl is the average length of all query documents in the query document set; A first elimination unit 503 is used to determine low-quality documents with scores less than a preset similarity threshold in the similarity score set, and eliminate the low-quality documents from the first query document set to obtain a second query document set; A generating unit 504 is used to extract key information from each query document in the second query document set, and generate a summary text corresponding to each query document based on the key information to obtain a summary text set; An extraction unit 505 is used to extract keywords from the summary text to form a keyword list; The second elimination unit 506 is used to eliminate redundant information from each summary text in the summary text set according to the keyword list, and convert it into a preset format to obtain a target summary text set; The second proposing unit 506 includes: A determination module 5061 is used to determine redundant information that is not semantically relevant to the keyword in each summary text based on a semantic similarity judgment rule, and remove the redundant information; The determination module 5061 includes: The first processing submodule 50611 is used to vectorize the text segments in each summary text to obtain multiple text vectors; The second processing submodule 50612 is used to vectorize each keyword to obtain a keyword vector; A calculation submodule 50613 is used to calculate the angle between the text vector and the keyword vector by using the cosine similarity method; A determination submodule 50614 is used to determine a target text segment corresponding to an angle value smaller than a preset similarity threshold, and remove the target text segment from the summary text; A sorting unit 507, used to sort each summary text in the target summary text set based on a preset sorting rule; The sorting unit 507 includes: A calculation module 5071 is used to calculate similarity scores for each summary text in the target summary text set; A sorting module 5072, used to sort each summary text from high to low according to the calculated similarity score; An acquisition unit 508 is used to acquire a knowledge link between the sorted summary text set and the second query document set, where the knowledge link is used by the multimodal large model to determine the query document corresponding to the summary text according to the summary text; The input unit 508 is used to input the sorted summary text set into the multimodal large model, so that the multimodal large model generates knowledge corresponding to the query keywords according to the sorting information.

[0080] See also Figure 6 , the present application also provides a compression device for a multimodal weight file, comprising: Processor 601, memory 602, input and output unit 603, bus 604; The processor 601 is connected to the memory 602, the input and output unit 603 and the bus 604; The memory 602 stores a program, and the processor 601 calls the program to execute the above Figure 1-Figure 3 Any method in the embodiments.

[0081] The present application also relates to a computer-readable storage medium, on which a program is stored, and when the program is run on a computer, the computer executes the above Figure 1-Figure 3 Any method in the embodiments.

[0082] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0083] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0084] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0086] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for compressing a multimodal weight file, characterized in that: include: Based on the query keyword input by the user, retrieve and obtain a first query document set, wherein the first query document set includes a plurality of documents or fragments matching the query keyword; Performing correlation analysis on each query document in the first query document set to obtain a similarity score corresponding to each query document and obtain a similarity score set, wherein the similarity score indicates the degree of association between the query document and the query keyword; Determine low-quality documents in the similarity score set whose scores are less than a preset similarity threshold, and remove the low-quality documents from the first query document set to obtain a second query document set; Extracting key information from each query document in the second query document set, and generating summary text corresponding to each query document based on the key information to obtain a summary text set; Extracting keywords from the summary text to form a keyword list; According to the keyword list, redundant information is removed from each summary text in the summary text set, and a conversion is performed into a preset format to obtain a target summary text set; Sorting each summary text in the target summary text set based on a preset sorting rule; The sorted summary text set is input into the multimodal large model, so that the multimodal large model generates knowledge corresponding to the query keywords according to the sorting information.

2. The compression method according to claim 1, characterized in that: After sorting the summary texts in the target summary text set based on the preset sorting rule, and before inputting the sorted summary text set into the multimodal macro model, the compression method further includes: Acquire a knowledge link between the sorted summary text set and the second query document set, wherein the knowledge link is used by the multimodal macro model to determine a query document corresponding to the summary text according to the summary text; The step of inputting the sorted summary text set into the multimodal macro model includes: The sorted summary texts and their corresponding knowledge links and keywords are input into the multimodal large model.

3. The compression method according to claim 1, characterized in that: The performing relevance analysis on each query document in the first query document set includes: Calculate the similarity score between the query keyword and each query document using the BM25 algorithm; The BM25 algorithm uses the following formula: Formula ① Formula ② Among them, IDF (q i ) is the query word q i The rare value in the query document set, N is the total number of query documents in the query document set, df(q i ) is the number of query documents containing the query keyword qi, Q is the query keyword set, D is the query document, n is the number of query terms, f(q i ,D) is the frequency of the query keyword qi appearing in the query document D, k1 and b are empirical values ​​set manually, dl is the length of the query document D, and avgdl is the average length of all query documents in the query document set.

4. The compression method according to claim 1, characterized in that: The step of sorting the summary texts in the target summary text set based on a preset sorting rule includes: Calculate similarity scores for each summary text in the target summary text set; Sort each summary text in descending order based on the calculated similarity score.

5. The compression method according to claim 1, characterized in that: The removing of redundant information from each summary text in the summary text set includes: Redundant information that is not semantically relevant to the keyword is determined in each summary text based on the semantic similarity judgment rule, and the redundant information is removed.

6. The compression method according to claim 5, characterized in that: The method of determining redundant information irrelevant to the semantics of the keyword in each summary text based on the semantic similarity judgment rule includes: Vectorizing the text segments in each summary text to obtain multiple text vectors; Vectorize each keyword to obtain a keyword vector; Calculating the angle between the text vector and the keyword vector by using cosine similarity; Determine the target text segment whose angle value is less than the preset similarity threshold, and remove the target text segment from the summary text.

7. The compression method according to claim 1, characterized in that: The query keywords based on user input include: The query keywords are preprocessed, and the preprocessing includes removing stop words, correcting spellings, and replacing synonyms.

8. A compression system for a multimodal weight file, characterized in that: include: A retrieval unit, configured to retrieve and obtain a first query document set based on a query keyword input by a user, wherein the first query document set includes a plurality of documents or fragments matching the query keyword; an analysis unit, configured to perform correlation analysis on each query document in the first query document set, obtain a similarity score corresponding to each query document, and obtain a similarity score set, wherein the similarity score indicates a correlation degree between the query document and the query keyword; A first elimination unit, configured to determine low-quality documents in the similarity score set whose scores are less than a preset similarity threshold, and eliminate the low-quality documents from the first query document set to obtain a second query document set; A generating unit, configured to extract key information from each query document in the second query document set, and generate a summary text corresponding to each query document based on the key information to obtain a summary text set; An extraction unit, used for extracting keywords from the summary text to form a keyword list; A second elimination unit is used to eliminate redundant information from each summary text in the summary text set according to the keyword list, and convert it into a preset format to obtain a target summary text set; A sorting unit, used to sort each summary text in the target summary text set based on a preset sorting rule; An input unit is used to input the sorted summary text set into the multimodal large model, so that the multimodal large model generates knowledge corresponding to the query keyword according to the sorting information.

9. A compression device for a multimodal weight file, characterized in that: include: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the compression method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the compression method according to any one of claims 1 to 7.

Images