Integrated retrieval dialogue method and system based on large model knowledge base and medium

Abstract

The invention relates to the field of integrated retrieval dialogues, in particular to an integrated retrieval dialogue method and system based on a large model knowledge base and a medium, and the method comprises the following steps: S1, knowledge base preprocessing: standardizing a knowledge base, including cleaning, word segmentation and stem extraction of texts in the knowledge base; s2, double filtering is carried out, a sparse retriever and a dense retriever are used for retrieving the preprocessed documents, and N documents are returned; s3, combining the preliminary results, and respectively selecting p documents with high similarity from the sparse retriever and the dense retriever; s4, performing calculation on all the selected documents by applying an RRF algorithm, and adding the obtained scores for calculation; and S5, returning a final document list according to the new ranking calculated by the RRF. The method has the advantages that a novel DFRAG technology is adopted, a dense retriever is used for double filtering, and a retrieval result is finely adjusted.

Description

[Technical field]

[0001] The present invention relates to the field of integrated retrieval dialogue, and in particular to an integrated retrieval dialogue method, system and medium based on a large model knowledge base. [Background technology]

[0002] The traditional information retrieval method is to extract question entities and attribute relationships from the question text, and then search in the knowledge base. The questioner enters the specific question text, analyzes the question text online, and then searches and outputs the most matching answer text to get the answer to the question. The retrieval speed is slow and the accuracy is poor.

[0003] BM25 is a commonly used technology in retrieval dialogue systems. It is a ranking function based on the bag-of-words model, used for text relevance sorting, and can effectively process documents of different lengths and effectively evaluate the relevance of documents and queries. However, when BM25 alone processes large-scale data sets and generates text or answers complex queries, efficiency and accuracy remain the main challenges. Retrieval efficiency is low, for example, there is a lot of noise information. If the knowledge base contains a large amount of irrelevant or erroneous information, the system may retrieve and use this erroneous information and easily mislead. There is also a lot of information redundancy, and the retrieval process will return redundant information, resulting in the lack of novelty or repetition in the generated answers, which cannot meet user needs.

[0004] Therefore, an integrated retrieval dialogue method, system and medium based on a large model knowledge base are proposed, which adopts the new DFRAG technology, namely Dual-Filter Retrieval-Augmented Generation technology, uses dense retrievers for double filtering, and fine-tunes the retrieval results. [Summary of the invention]

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an integrated retrieval dialogue method, system and medium based on a large model knowledge base, adopting the new DFRAG technology to use dense searchers for double filtering to fine-tune the retrieval results.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] An integrated retrieval dialogue method based on a large model knowledge base includes the following steps:

[0008] S1: Knowledge base preprocessing, standardization of the knowledge base, including cleaning, word segmentation, and stem extraction of the text in the knowledge base;

[0009] S2: Perform double filtering, use sparse retriever and dense retriever to retrieve the preprocessed documents, and return N documents;

[0010] S3: Merge the preliminary results and select p documents with high similarity from the sparse searcher and the dense searcher respectively;

[0011] S4: Use the RRF algorithm to perform calculations on all selected documents and add up the scores obtained for calculation;

[0012] S5: Return the final document list based on the new ranking calculated by RRF.

[0013] Preferably, S41: for the documents returned by the sparse retriever and the dense retriever, the scores are calculated using RRF according to their positions in the original rankings. The RRF formula is:

[0014]

[0015] Among them, rank1 is the ranking of documents retrieved by the sparse retriever, rank2 is the ranking of documents retrieved by the dense retriever, and k is a constant.

[0016] Preferably, it also includes S6: using an evaluator to evaluate, and outputting according to the confidence of the evaluation. When the evaluator confidence is high, the retrieval result outputs an answer; when the evaluator confidence is not high, the user is requested to provide more information and a final answer is output.

[0017] As a preferred method, S21: first use the sparse searcher BM25 algorithm to search the preprocessed documents, use the sparse searcher to search, search for documents that mention the keyword according to the word frequency, and return N documents;

[0018] Preferably, S22: then use the dense retriever BERT model to retrieve the preprocessed documents, use the dense retriever to generate embedding vectors of the query and the document, and retrieve the document by calculating the similarity between the vectors. Find the documents that are closely related to the semantic content of the entire query and return N documents.

[0019] As a preferred embodiment, the dense retriever BERT model performs adaptive preprocessing on the original sample data obtained for training to obtain input samples for training;

[0020] Construct the BERT similarity model through dynamic learning rate mechanism and layered adaptive adjustment mechanism;

[0021] Convert the training input samples into input sequences through the BERT similarity model;

[0022] Generate adversarial input samples based on the training input samples and input sequences; and

[0023] The BERT similarity model is adversarially trained using adversarial input samples and training input samples to obtain a trained BERT similarity model.

[0024] Preferably, S221 includes:

[0025] C1. Build the Embedding layer. The Embedding layer is the sum of three types of Embeddings: Token Embeddings, Segment Embeddings, and Position Embeddings.

[0026] C2, Masked LM, is used to train deep bidirectional language representation. Specifically: mask part of the original corpus, and then predict the masked words or characters. Randomly mask 30% of the characters in each sentence and use their context to make predictions. Of all the original corpus, 70% uses [mask], 20% randomly takes a word to replace the masked word, and the remaining 10% remains unchanged;

[0027] C3. Construct a bidirectional Transformer layer structure based on the self-attention mechanism.

[0028] An integrated retrieval dialogue system based on a large model knowledge base. The system adopts an integrated retrieval dialogue method based on a large model knowledge base. The system includes a knowledge base preprocessing module for standardizing the knowledge base, including cleaning, word segmentation, and stem extraction of text in the knowledge base.

[0029] The double filtering module is used to retrieve the preprocessed documents using a sparse retriever and a dense retriever, and return N documents;

[0030] The merging module is used to select p documents with high similarity in the sparse searcher and the dense searcher respectively;

[0031] RRF algorithm module, used to perform calculations on all selected documents and add up the scores obtained for calculation;

[0032] New ranking module, used to return the final list of documents based on calculations;

[0033] An evaluation module is also included for outputting according to the evaluation confidence.

[0034] A computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned integrated retrieval dialogue method based on a large model knowledge base.

[0035] A processor is used to run a program, wherein the program executes the above-mentioned integrated retrieval dialogue method based on a large model knowledge base when running.

[0036] The beneficial effects of the present invention are as follows:

[0037] 1. The dual filtering mechanism of the integrated dialogue in this embodiment, also known as DFRAG technology, combines the advantages of the BM25 sparse retriever and the BERT dense retriever to achieve efficient retrieval and accurate screening of documents related to the query document field or the bidding field. First, the BM25 retriever uses its powerful keyword matching ability to quickly retrieve preliminary candidate documents related to bidding from massive documents. Subsequently, the BERT retriever further evaluates these candidate documents at the semantic level to ensure that the retrieval results are not only literally matched, but also semantically consistent with the professional knowledge in the query document field or the bidding field.

[0038] In the double filtering process, DFRAG technology places special emphasis on fine-tuning the retrieval results. By selecting the top 20% of the top-ranked documents of the two retrievers, DFRAG ensures that the information finally input into the large language model is strictly screened, thereby reducing the generation of misleading illusions and improving the timeliness and accuracy of information.

[0039] In addition, DFRAG technology also introduces a lightweight evaluator, such as the T5 model, which preprocesses and evaluates the retrieved documents before inputting them into the large language model. This step not only improves processing efficiency, but also ensures the suitability and relevance of the input information, further enhancing the model's professional insight in the query document field or bidding field.

[0040] In general, DFRAG technology aims to improve the performance and efficiency of generative models by combining sparse retrieval and dense retrieval methods and using lightweight estimators to preprocess data. This approach can more effectively utilize relevant information while reducing the impact of irrelevant or low-quality data on model performance.

[0041] 2. Reduce misleading information: Ensure that the input information is strictly screened through a double filtering mechanism, thereby reducing the risk of misleading output.

[0042] 3. Improve information timeliness: By retrieving the latest documents in real time, ensure that the model can access and utilize the latest query document field or bidding field information.

[0043] 4. Improve the efficiency of processing specific knowledge: When processing knowledge in specific fields, such as bidding systems, large language models may not be efficient. Through specially designed retrieval and filtering mechanisms, the efficiency of the model in processing knowledge in specific fields is improved.

[0044] 5. Enhanced deep insights into professional fields: By combining domain-specific retrieval and evaluators, the model’s deep insights into professional fields such as bidding systems are enhanced.

[0045] 6. Optimize information input quality: In order to improve the accuracy and relevance of generated content, the retrieved documents are preprocessed and evaluated through a lightweight evaluator to ensure the quality of information input to the large language model.

[0046] These features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings.

Brief Description of the Drawings

[0047] The present invention will be further described below in conjunction with the accompanying drawings:

[0048] Figure 1 The figure is a flow chart of the formula search dialogue method according to the first embodiment of the present invention. [Specific implementation method]

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The following first describes the concepts involved in this application in conjunction with the accompanying drawings. It should be noted that the following description of each concept is only to make the content of this application easier to understand and does not limit the scope of protection of this application.

[0051] Embodiment 1:

[0052] Integrated retrieval dialogue method based on large model knowledge base, such as Figure 1 As shown, the steps include

[0053] S1: Knowledge base preprocessing, standardization of the knowledge base, including cleaning, word segmentation, and stem extraction of the text in the knowledge base;

[0054] S2: Perform double filtering, use sparse retriever and dense retriever to retrieve the preprocessed documents, and return N documents;

[0055] S21: First use the sparse searcher BM25 algorithm to search the preprocessed documents, use the sparse searcher to search, search for documents that mention the keyword based on the word frequency, and return N documents;

[0056] The preprocessed documents are retrieved using the sparse retriever BM25, which is a widely used information retrieval algorithm that evaluates the relevance of documents to queries based on keyword matching. Suppose there is a search query: "How will artificial intelligence affect the development of future cars". Using the BM25 model for retrieval, it will return documents that mention keywords such as "artificial intelligence", "future" and "cars" based on word frequency. Finally, N documents are returned.

[0057] S22: Then use the dense retriever BERT model to retrieve the preprocessed documents, use the dense retriever to generate embedding vectors of the query and document, and retrieve the document by calculating the similarity between the vectors. Find the documents that are closely related to the semantic content of the entire query and return N documents.

[0058] The preprocessed documents are retrieved using the dense retriever BERT. BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained deep learning model that performs well in natural language processing tasks and can understand the semantic content of documents and queries. First, the BERT model is used to generate embedding vectors for queries and documents, and documents are retrieved by calculating the similarity between the vectors. Documents that are more closely related to the semantic content of the entire query are found, even if these documents do not necessarily contain all the keywords. Assume that N documents are returned.

[0059] S3: Combine the preliminary results, select p documents with high similarity from the sparse searcher and the dense searcher respectively; select p documents with the highest similarity ranking from each searcher, and take the top 20% from each.

[0060] S4: Use the RRF algorithm to perform calculations on all selected documents and add up the scores obtained for calculation;

[0061] S41: For the documents returned by the sparse retriever and the dense retriever, the scores are calculated using RRF according to their positions in the original ranking. The RRF formula is:

[0062]

[0063] Here, rank1 is the ranking of the document retrieved by the sparse retriever, rank2 is the ranking of the document retrieved by the dense retriever, and k is a constant. Assuming that document A is ranked 1st in the BM25 retriever and 3rd in the BERT retriever, its RRF score will be 1 / (60+1)+1 / (60+3). Finally, the same calculation is performed on all selected documents and the scores are added together.

[0064] If you decide to include all documents returned by BM25, even if they do not appear in the BERT results, you can still assign these documents a default lower RRF score, for example based on their rank in BM25, but with a larger constant b added to the score to reflect their absence in BERT.

[0065] S5: Return the final document list based on the new ranking calculated by RRF.

[0066] By combining the power of both retrievers in this way, even very complex queries can provide a comprehensive list of documents based on keyword matching and semantic similarity.

[0067] It also includes S6: using an evaluator to evaluate and outputting according to the confidence of the evaluation. When the evaluator confidence is high, the retrieval result outputs an answer; when the evaluator confidence is not high, the user is requested to provide more information and a final answer is output.

[0068] Before the retrieved documents are fed into a very large-scale language model, such as GPT3.5, they are first evaluated using a lightweight evaluator, such as the T5 model. The purpose of this evaluator may be to filter or adjust the retrieved documents to ensure that they are more suitable for input into the large-scale model.

[0069] When the evaluator has high confidence, it will use the search results to output the answer; when the confidence is not high, it will ask the user to provide more information to improve the accuracy of the search, and finally output the answer. Using a lightweight evaluator can help improve efficiency and is faster and more resource-efficient than extremely large-scale models when processing large amounts of data.

[0070] It is worth mentioning that the dual filtering mechanism of the integrated dialogue in this embodiment, also known as DFRAG technology, combines the advantages of the BM25 sparse retriever and the BERT dense retriever to achieve efficient retrieval and accurate screening of documents related to the query document field or the bidding field. First, the BM25 retriever uses its powerful keyword matching ability to quickly retrieve preliminary candidate documents related to bidding from massive documents. Subsequently, the BERT retriever further evaluates these candidate documents at the semantic level to ensure that the retrieval results are not only literally matched, but also semantically consistent with the professional knowledge in the query document field or the bidding field.

[0071] In the double filtering process, DFRAG technology places special emphasis on fine-tuning the retrieval results. By selecting the top 20% of the top-ranked documents of the two retrievers, DFRAG ensures that the information finally input into the large language model is strictly screened, thereby reducing the generation of misleading illusions and improving the timeliness and accuracy of information.

[0072] In addition, DFRAG technology also introduces a lightweight evaluator, such as the T5 model, which preprocesses and evaluates the retrieved documents before inputting them into the large language model. This step not only improves processing efficiency, but also ensures the suitability and relevance of the input information, further enhancing the model's professional insight in the query document field or bidding field.

[0073] In general, DFRAG technology aims to improve the performance and efficiency of generative models by combining sparse retrieval and dense retrieval methods and using lightweight estimators to preprocess data. This approach can more effectively utilize relevant information while reducing the impact of irrelevant or low-quality data on model performance.

[0074] The dense retriever BERT model performs adaptive preprocessing on the original sample data obtained for training to obtain input samples for training;

[0075] Construct the BERT similarity model through dynamic learning rate mechanism and layered adaptive adjustment mechanism;

[0076] Convert the training input samples into input sequences through the BERT similarity model;

[0077] Generate adversarial input samples based on the training input samples and input sequences; and

[0078] The BERT similarity model is adversarially trained using adversarial input samples and training input samples to obtain a trained BERT similarity model.

[0079] In addition, the S221 includes:

[0080] C1. Build the Embedding layer. The Embedding layer is the sum of three types of Embeddings: Token Embeddings, Segment Embeddings, and Position Embeddings.

[0081] Token Embeddings is a word vector, the first word is the CLS flag, which can be used for subsequent classification tasks;

[0082] Segment Embeddings is used to distinguish between two sentences, because pre-training not only does LM but also does classification tasks with two sentences as input;

[0083] Position Embeddings are trigonometric functions but are learned;

[0084] C2, Masked LM, is used to train deep bidirectional language representation. Specifically: mask part of the original corpus, and then predict the masked words or characters. Randomly mask 30% of the characters in each sentence and use their context to make predictions. Of all the original corpus, 70% uses [mask], 20% randomly takes a word to replace the masked word, and the remaining 10% remains unchanged;

[0085] For example, “How will artificial intelligence affect the development of future cars” → “How will artificial intelligence affect the [mask][mask] of future cars?”, 10% is to randomly select a word to replace the masked word, “How will artificial intelligence affect the development of future cars” → “How will artificial intelligence affect the prospects of future cars”, 10% remains unchanged, China’s “How will artificial intelligence affect the development of future cars” → “How will artificial intelligence affect the development of future cars”.

[0086] C3. Construct a bidirectional Transformer layer structure based on the self-attention mechanism.

[0087] Embodiment 2:

[0088] An integrated retrieval dialogue system based on a large model knowledge base. The system adopts an integrated retrieval dialogue method based on a large model knowledge base. The system includes a knowledge base preprocessing module for standardizing the knowledge base, including cleaning, word segmentation, and stem extraction of text in the knowledge base.

[0089] The double filtering module is used to retrieve the preprocessed documents using a sparse retriever and a dense retriever, and return N documents;

[0090] The merging module is used to select p documents with high similarity in the sparse searcher and the dense searcher respectively;

[0091] RRF algorithm module, used to perform calculations on all selected documents and add up the scores obtained for calculation;

[0092] New ranking module, used to return the final list of documents based on calculations;

[0093] An evaluation module is also included for outputting according to the evaluation confidence.

[0094] The advantages are: reducing misleading information and ensuring that the input information is strictly screened through a double filtering mechanism, thereby reducing the risk of misleading output.

[0095] Improve information timeliness by retrieving the latest documents in real time to ensure that the model can access and utilize the latest query document or bidding information.

[0096] Improve the efficiency of processing specific knowledge. When processing knowledge in specific fields, such as bidding systems, large language models may not be efficient. Through specially designed retrieval and filtering mechanisms, the efficiency of the model in processing knowledge in specific fields is improved.

[0097] Enhance deep insights into professional fields. By combining domain-specific retrieval and evaluators, the model's deep insights into professional fields such as bidding systems are enhanced.

[0098] Optimize the quality of information input,In order to improve the accuracy and relevance of the generated content, the retrieved documents are preprocessed and evaluated through a lightweight evaluator to ensure the quality of information input to the large language model.

[0099] A computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned integrated retrieval dialogue method based on a large model knowledge base.

[0100] A processor is used to run a program, wherein the program executes the above-mentioned integrated retrieval dialogue method based on a large model knowledge base when running.

[0101] Those of ordinary skill in the art will appreciate that the units (or modules, steps, etc., the same below) of the examples described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0102] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored, etc.

[0103] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nly Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. An integrated retrieval dialogue method based on a large model knowledge base, characterized in that: Steps include S1: Knowledge base preprocessing, standardization of the knowledge base, including cleaning, word segmentation, and stem extraction of the text in the knowledge base; S2: Perform double filtering, use sparse retriever and dense retriever to retrieve the preprocessed documents, and return N documents; S3: Merge the preliminary results and select p documents with high similarity from the sparse searcher and the dense searcher respectively; S4: Use the RRF algorithm to perform calculations on all selected documents and add up the scores obtained for calculation; S5: Return the final document list based on the new ranking calculated by RRF.

2. The integrated retrieval dialogue method based on a large model knowledge base as claimed in claim 1, characterized in that: S41: For the documents returned by the sparse retriever and the dense retriever, the scores are calculated using RRF according to their positions in the original ranking. The RRF formula is: Among them, rank1 is the ranking of documents retrieved by the sparse retriever, rank2 is the ranking of documents retrieved by the dense retriever, and k is a constant.

3. The integrated retrieval dialogue method based on a large model knowledge base as described in claim 1 or 2, characterized in that: It also includes S6: using an evaluator to evaluate and outputting according to the confidence of the evaluation. When the evaluator confidence is high, the retrieval result outputs an answer; when the evaluator confidence is not high, the user is requested to provide more information and a final answer is output.

4. The integrated retrieval dialogue method based on a large model knowledge base as claimed in claim 1, characterized in that: S21: First use the sparse searcher BM25 algorithm to search the preprocessed documents, use the sparse searcher to search, search for documents that mention the keyword based on the word frequency, and return N documents; S22: Then use the dense retriever BERT model to retrieve the preprocessed documents, use the dense retriever to generate embedding vectors of the query and document, and retrieve the document by calculating the similarity between the vectors. Find the documents that are closely related to the semantic content of the entire query and return N documents.

5. The integrated retrieval dialogue method based on a large model knowledge base as claimed in claim 4, characterized in that: The dense retriever BERT model performs adaptive preprocessing on the acquired original sample data for training to obtain input samples for training; Construct the BERT similarity model through dynamic learning rate mechanism and layered adaptive adjustment mechanism; Converting the training input sample into an input sequence through the BERT similarity model; Generate an adversarial input sample according to the training input sample and the input sequence; as well as The BERT similarity model is adversarially trained using the adversarial input sample and the training input sample to obtain a trained BERT similarity model.

6. The integrated retrieval dialogue method based on a large model knowledge base as claimed in claim 1, characterized in that: S221 includes: C1. Construct an Embedding layer, which is formed by summing three types of Embeddings: Token Embeddings, Segment Embeddings, and Position Embeddings. C2, Masked LM, is used to train deep bidirectional language representation. Specifically: mask part of the original corpus, and then predict the masked words or characters. Randomly mask 30% of the characters in each sentence and use their context to make predictions. Of all the original corpus, 70% uses [mask], 20% randomly takes a word to replace the masked word, and the remaining 10% remains unchanged; C3. Construct a bidirectional Transformer layer structure based on the self-attention mechanism.

7. An integrated retrieval dialogue system based on a large model knowledge base, the system adopts the integrated retrieval dialogue method based on a large model knowledge base according to any one of claims 1 to 6, characterized in that: The system includes a knowledge base preprocessing module for standardizing the knowledge base, including cleaning, word segmentation, and stem extraction of text in the knowledge base; The double filtering module is used to retrieve the preprocessed documents using a sparse retriever and a dense retriever, and return N documents; The merging module is used to select p documents with high similarity in the sparse searcher and the dense searcher respectively; RRF algorithm module, used to perform calculations on all selected documents and add up the scores obtained for calculation; New ranking module, used to return the final list of documents based on calculations.

8. The integrated retrieval dialogue system based on a large model knowledge base as claimed in claim 7, characterized in that: An evaluation module is also included for outputting according to the evaluation confidence.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute the integrated retrieval dialogue method based on a large model knowledge base according to any one of claims 1 to 6.

10. A processor, characterized in that: The processor is used to run the program, wherein the program executes the integrated retrieval dialogue method based on a large model knowledge base according to any one of claims 1 to 6 when running.

Images