A text data enhancement method to improve vector retrieval performance

Abstract

The present invention provides a text data enhancement method for improving vector retrieval performance, which belongs to the field of computer data analysis. The method first uses a prompt template of a large language model to compress long text data and decompose it into several short texts; during the training process, these short texts will replace the original long texts and be used as training data, thereby saving the size of the video memory occupied by a single piece of information; in order to address the problem of decreased representation ability that may result from shortened text length, the present invention constructs a guidance vector by combining multiple short texts from the same long text, and uses the guidance vector as auxiliary information to guide the encoding process of a single short text. In this way, the present invention can effectively reduce the adverse effects of text shortening on the model's representation ability, thereby improving the training effect and generalization ability of the model while using shorter single pieces of information.

Description

Technical Field

[0001] The present invention relates to the field of computer data analysis, and in particular to a method for constructing a semantic space based on deep learning and enhancing text data based on a large language model. Background Art

[0002] Retrieval tasks typically involve finding the set of documents most similar to a query within a large dataset. Currently, article retrieval techniques based on dense semantic vectors have demonstrated widespread practical application in a number of key areas, including but not limited to open-domain question answering, conversational systems, and web search.

[0003] Data augmentation is one of the classic techniques for improving the quality of information retrieval. It creates additional training data by slightly perturbing the training samples, such as rotating, translating, and scaling. Data augmentation includes a series of techniques for generating new training samples from the original samples through jitter and interference, but without changing the class labels. The purpose of data augmentation is to increase the generalization ability of the model. If the network we want to train constantly sees new, slightly changing input sample points, then it can learn more robust features. However, current data augmentation methods in the field of vector retrieval are limited to simple word and sentence transformations in the source text, such as repetition, deletion, and insertion of meaningless characters. The enhanced data generated by these methods is of low quality and can be easily recognized by the model, resulting in poor data augmentation effects.

[0004] Contrastive learning is an unsupervised or self-supervised learning method whose primary goal is to learn a representation model by automatically constructing similar and dissimilar instances, such that similar instances are close together in the projected space, while dissimilar instances are farther apart. This method has been widely used in natural language processing. However, how to effectively construct similar and dissimilar instances and how to design the model structure to adhere to this guiding principle are key issues in contrastive learning. Inappropriate instance construction can cause the model to learn incorrect feature representations, thus affecting its performance in downstream tasks.

[0005] Based on this, the present invention proposes a method for data augmentation using a large language model to improve retrieval performance. This method uses zero-shot hints to achieve more flexible data augmentation of text data and screens the data based on semantic features at the time of data generation to ensure data augmentation quality. After obtaining high-quality data, the present invention leverages the orthogonality of high-dimensional vectors to integrate the semantic information of the augmented data. This integrated information serves as a guide vector to guide the model fine-tuning process, ultimately improving the performance of model vector retrieval. Summary of the Invention

[0006] This paper addresses the issue of data augmentation in information retrieval and proposes a method for enhancing text data using a large language model to improve retrieval performance. This method uses the large language model's prompt templates to semantically decouple the raw data, breaking it down into several short texts. The data generated by the large language model is then filtered using semantic features to generate enhanced data. During training, contrastive learning is used as the first step. The model generated in the first step is then used to aggregate the enhanced data to generate guidance vectors. The guidance vectors are then used as auxiliary information to guide the text representation process in contrastive learning, thereby improving the performance of model vector retrieval.

[0007] The technical solution of the present invention is:

[0008] A text data enhancement method for improving vector retrieval performance includes the following steps:

[0009] S1. Acquire a data set and preprocess the data set;

[0010] S2. Use the preprocessed data set to perform comparative learning training on the pre-trained model to obtain a trained primary retrieval model;

[0011] S3. Based on the large language model, semantically decouple the positive example documents in the preprocessed data set, decompose the positive example documents into several short texts; and filter the short text data to obtain an enhanced data set;

[0012] S4. Aggregate the data in the enhanced data set based on the primary retrieval model to construct a guidance vector, and use the guidance vector to guide the training of the primary retrieval model to obtain a final retrieval model.

[0013] Furthermore, in S1, the data set includes:

[0014] A query file, which defines all query contents;

[0015] A file that lists the correspondence between queries and positive examples, which can determine the relationship between queries and corresponding correct documents;

[0016] A document collection file that identifies all document contents.

[0017] Furthermore, in S1, the data set is preprocessed, specifically:

[0018] If the dataset contains a correspondence file between the query and the negative example, the positive example document and the negative example document are determined based on the correspondence file between the query and the positive example and the negative example respectively, and the dataset format is processed into (query, positive example, negative example); if the dataset lacks a correspondence file between the query and the negative example, the retrieval model is used to retrieve similar documents in the entire document collection as query negative examples, and the positive example document is determined based on the correspondence file between the query and the positive example, and the dataset format is processed into (query, positive example, negative example).

[0019] Furthermore, in S2, the training process of the primary retrieval model is as follows:

[0020] S2.1. Initialize its pre-trained checkpoint using the pre-trained model. The pre-trained model can act as an encoder to map all queries and positive examples in the dataset obtained in S1 into the same vector space.

[0021] S2.2. Divide the dataset obtained in S1 into several training batches according to the storage space limit of the computing device. The training task is also divided into several batches. In any training task of a batch, the query, positive example documents, and negative example documents in the batch are first encoded into vectors simultaneously.

[0022] S2.3. In the vector space of the same training batch, for any query vector, its corresponding negative examples include all documents except its own positive example document. The distances between the query vector and each positive example document vector and each negative example document vector are calculated, and the calculated distances are smoothed using the softmax function to make them distributed between (0, 1). Finally, cross entropy is used as the loss function for training to obtain a trained primary retrieval model, thereby shortening the distance between the query and the positive examples and increasing the distance between the query and the negative examples. The loss function formula is as follows:

[0023]

[0024] in, is the result of the softmax function processing of the distance between the query vector and the positive document vector or the negative document vector; y is the actual label. When the distance between the query vector and the positive document vector is calculated, y = 1; when the distance between the query vector and the negative document vector is calculated, y = 0.

[0025] Furthermore, in S2.1, the pre-trained models include RetroMAE, GTE, etc.

[0026] Furthermore, the specific process of S3 is as follows:

[0027] S3.1. For a training set of the dataset in S1, construct a prompt template for each positive example document into an instruction input into the large language model, obtain a semantically decoupled short text output for each positive example document, and then change the training set format to (query, positive example, negative example, short text collection);

[0028] S3.2. For the short texts in the generated (query, positive examples, negative examples, short text collection), the quality of the short texts is quantitatively evaluated using the quality control indicator M. The quality control indicator M comprehensively considers two dimensions of similarity: the first is the similarity between the short text and its corresponding long text, which reflects the short text's ability to preserve the original information; the second is the difference in similarity between different short texts corresponding to the same long text, which measures the degree of difference in the short texts when expressing the same long text content in a diversified way. Its calculation formula is as follows:

[0029] M i =∑ Aj∈Z sim(A,Ai)-∑ Ai,Aj∈Z sim(Ai,Aj)

[0030] Where Z is the short text set corresponding to the positive example A, Ai and Aj are two different short texts in the short text set Z; sim(A, Ai) represents the similarity between the positive example A and the generated short text Ai, which is obtained by encoding the two texts A and Ai into the vector space using the primary retrieval model obtained by S2 and then calculating the cosine similarity of the two vectors; M i Represents the quality control index of the short text Ai;

[0031] S3.3. Set the hyperparameters K and R. For all short text sets, the quality control index of each short text data must be greater than R. At most K short texts with the largest quality control index are retained to form an enhanced data set in the format of (query, positive example, negative example, enhanced short text set).

[0032] Furthermore, in S3.1, the large language model includes GPT-4, Tongyi, etc.

[0033] Furthermore, the specific process of S4 is as follows:

[0034] S4.1. First, load the primary retrieval model obtained in S2, and divide the enhanced data set obtained in S3 into several training batches according to the storage space limit of the computing device. The training task is also divided into several batches. In a batch of training tasks, the queries, positive examples, negative examples, and enhanced short text sets in the batch are all encoded into high-dimensional vectors, where the enhanced short text set does not pass through the pooling layer. Add all the high-dimensional vectors of short texts from the same query to obtain a splicing vector. Then pass the splicing vector through a fully connected layer and a residual connection layer, and project it into a high-level semantic space. Split the splicing vector of each query according to the vector position, and use the knn algorithm to cluster the vectors based on the Euclidean distance between the vectors at each position to generate several phrase vectors. For each query vector, calculate the KL divergence between the query vector and each phrase vector, and use the phrase vector corresponding to the minimum KL divergence as the guide vector. The calculation formula of the guide vector is:

[0035] P=min u∈U (D KL (Q||u))

[0036] Among them, P represents the guidance vector, Q represents the high-dimensional vector encoded by the query, U represents the set of phrase vectors, u is a single phrase vector, and D KL represents the divergence function;

[0037] S4.2. Use the loss function L to train and learn to obtain the final retrieval model; the loss function L = L1 + L2, which are as follows:

[0038] For a set of queries and short texts, mark any short text corresponding to a query as a positive example and the rest as negative examples, and calculate the cross entropy as the loss function L1;

[0039] L1=∑-ylogsim(Q,S)-(1-y)log(1-sim(Q,S))

[0040] Where sim represents the cosine distance between vectors; Q represents the high-dimensional vector encoded by the query; S represents the high-dimensional vector encoded by the short text; y is the actual label, y = 1 when the short text is a positive example, and y = 0 when the short text is a negative example;

[0041] For the guidance vector and the positive example, the cross entropy between the guidance vector corresponding to any query and the positive example corresponding to the query and the negative example corresponding to the query is calculated as the loss function L2;

[0042] L2=∑-ylogsim(P,D)-(1-y)log(1-sim(P,D))

[0043] Wherein, sim represents the cosine distance of the vector; P represents the guide vector; D represents the vector encoded by the positive or negative example; y is the actual label, when D is the vector encoded by the positive example, y=1, when D is the vector encoded by the negative example, y=0.

[0044] The present invention has the following beneficial effects compared to the prior art:

[0045] (1) To address the problem of non-compliance with instructions when using large language models for data compression, a quality assessment method that focuses on the consistency between generated text and the original text and the differences between generated texts was adopted to improve the quality of training data;

[0046] (2) To address the information loss problem caused by text compression, a method is adopted to construct a guidance vector through the interaction between multiple generated short texts to restore the text information within the model, thereby reducing the loss of compressed information and improving the model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the overall framework diagram of the present invention;

[0048] Figure 2 is an example of an instruction in the present invention;

[0049] Figure 3 This is a flowchart of obtaining phrase vectors in the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] like Figure 1 As shown, the embodiment of the present invention provides a text data enhancement method for improving vector retrieval performance, the steps are as follows:

[0052] S1. Obtaining a data set and preprocessing the data set

[0053] The Multi-CPR dataset was selected and preprocessed. Specifically, each document in the dataset was vector-embedded using the BGE model to obtain a vector representation for each document. The generated vector embeddings were then stored in the FAISS library developed by Facebook AI to construct an efficient vector index database. Within the vector index database constructed using FAISS, the approximate nearest neighbor search (ANNS) algorithm was used to retrieve the entire document collection for document embedding and obtain negative documents. Similar documents were retrieved from the entire document collection as query negatives, and positive documents were determined based on the correspondence between the query and positive documents. The dataset was then formatted as (query, positive, negative).

[0054] S2. Use the data set obtained in S1 to perform comparative learning training on the pre-trained model to obtain a trained primary retrieval model. The specific process is as follows:

[0055] S2.1. Use the pre-trained checkpoint provided by the pre-trained model RetroMAE as initialization. The pre-trained model RetroMAE can act as an encoder to map all queries and positive examples in the dataset obtained in S1 into the same vector space.

[0056] S2.2. The training batch size is set to 32. In any batch training task, the query, positive document, and negative document in the batch are first encoded into vectors. The negative example corresponding to any query includes all documents except the positive example document of the query itself.

[0057] S2.3. In the vector space of the same training batch, for any query vector, its corresponding negative examples include all documents except its own positive example document. The distances between the query vector and each positive example document vector and each negative example document vector are calculated, and the calculated distances are smoothed using the softmax function so that they are distributed between (0, 1). Finally, cross entropy is used as the loss function for training to obtain a trained primary retrieval model, thereby shortening the distance between the query and the positive examples and increasing the distance between the query and the negative examples. The loss function formula is as follows:

[0058]

[0059] in, is the result of the softmax function processing of the distance between the query vector and each positive document vector or negative document vector; y is the actual label. When the distance between the query vector and the positive document vector is calculated, y = 1; when the distance between the query vector and the negative document vector is calculated, y = 0.

[0060] S3: Based on the large language model, semantically decouple the positive example documents in the dataset of S1 and decompose them into several short texts. The short text data is then filtered to obtain an enhanced dataset. The specific process is as follows:

[0061] S3.1. For a training set of the dataset in S1, construct the instructions shown in the figure for each positive example document using a prompt template and input them into the large language model GPT-4 to obtain the semantically decoupled short text output of each positive example document. Then, the training set format is changed to (query, positive example, negative example, short text collection);

[0062] S3.2. For the short texts in the generated (query, positive examples, negative examples, short text collection), use the quality control indicator M to quantitatively evaluate the quality of the short texts. The calculation formula is as follows:

[0063]

[0064] Where Z is the short text set corresponding to the positive example A, Ai and Aj are two different short texts in the short text set Z; sim(A, Ai) represents the similarity between the positive example A and the generated short text Ai, which is obtained by encoding the two texts A and Ai into the vector space using the primary retrieval model obtained by S2 and then calculating the cosine similarity of the two vectors; M i Represents the quality control index of the short text Ai;

[0065] S3.3. Set hyperparameters K=3 and R=0. For all short text sets, the quality control index of each short text data must be greater than 0. At most the top 3 short texts with the largest quality control index are retained to form an enhanced data set in the format of (query, positive example, negative example, enhanced short text set).

[0066] S4. Aggregate the data in the enhanced dataset based on the primary retrieval model to obtain a guidance vector, and use the guidance vector as auxiliary information to train the primary retrieval model to obtain a final retrieval model. The specific process is as follows:

[0067] S4.1. First, load the primary retrieval model obtained in S2, and divide the enhanced data set (query, positive example, negative example, enhanced short text set) obtained in S3 into several training batches with a batch size of 32. The training task is therefore divided into several batches. In a batch of training tasks, the query, positive example, negative example, and enhanced short text set in the batch are all encoded into high-dimensional vectors, and the enhanced short text set does not pass through the pooling layer. Add all the short text high-dimensional vectors from the same query to obtain a splicing vector; then pass the splicing vector through a fully connected layer and a residual connection layer, and project it into a high-level semantic space; the splicing vector of each query is split according to the vector position, and the knn algorithm is used to perform clustering based on the Euclidean distance between the vectors at each position to generate several phrase vectors; for each query vector, calculate the KL divergence between the query vector and each phrase vector, and use the phrase vector corresponding to the minimum KL divergence as the guide vector; the calculation formula of the guide vector is:

[0068] P=min u∈U (D KL (Q||u))

[0069] Among them, P represents the guidance vector, Q represents the high-dimensional vector encoded by the query, U represents the set of phrase vectors, u is a single phrase vector, and D KL represents the divergence function;

[0070] S4.2. Use the loss function L to train and learn to obtain the final retrieval model; the loss function L = L1 + L2, which are as follows:

[0071] For a set of queries and short texts, mark any short text corresponding to a query as a positive example and the rest as negative examples, and calculate the cross entropy as the loss function L1;

[0072] L1=Σ-ylpgsim(Q,S)-(1-y)log(1-sim(Q,S))

[0073] Where sim represents the cosine distance between vectors; Q represents the high-dimensional vector encoded by the query; S represents the high-dimensional vector encoded by the short text; y is the actual label, y = 1 when the short text is a positive example, and y = 0 when the short text is a negative example;

[0074] For the guidance vector and the positive example, the cross entropy between the guidance vector corresponding to any query and the positive example corresponding to the query and the negative example corresponding to the query is calculated as the loss function L2;

[0075] L2=Σ-ylogsim(P,D)-(1-y)log(1-sim(P,D))

[0076] Wherein, sim represents the cosine distance of the vector; P represents the guide vector; D represents the vector encoded by the positive or negative example; y is the actual label, when D is the vector encoded by the positive example, y=1, when D is the vector encoded by the negative example, y=0.

[0077] This embodiment conducts experiments on the paragraph retrieval dataset Multi-CPR, using MRR@10 as the evaluation metric. Multi-CPR is a comprehensive dataset designed for multi-domain Chinese article retrieval. This dataset covers three very different fields: e-commerce, entertainment video, and medical care, and aims to provide research resources for cross-domain information retrieval. The data subset for each field contains millions of text paragraphs. In addition, this embodiment also conducts experiments on the dataset T2Ranking, which is a dataset constructed for large-scale Chinese article retrieval tasks. The dataset brings together more than 300,000 user queries and more than two million unique paragraph texts from real-world search engines. The method of the present invention is compared with the QL, BM25, DEw / BM25Neg, DEw / MinedNeg, and DPTDR algorithms, and the experimental results are shown in Tables 1 and 2.

[0078] Among them, QL (query likelihood) is a representative statistical language model that measures the relevance of paragraphs by modeling the generation of queries. BM25 is a widely used sparse retrieval baseline. DEw / BM25Neg is equivalent to DPR, which is the first work to use a pre-trained language model as the backbone for article retrieval tasks. DEw / MinedNeg, as described in ANCE, enhances the performance of DPR by globally sampling hard negative examples from the entire corpus. DPTDR is a work that uses on-the-fly tuning for dense vector retrieval.

[0079] Table 1 Experimental results of T2Ranking dataset

[0080]

[0081] Table 2 Comparison of MRR@10 indicators in Multi-CPR dataset experiments

[0082]

[0083] Experimental results show that the data augmentation strategy implemented in this paper significantly improves the model's performance on the average reciprocal ranking metric. This performance improvement can be attributed to three factors: first, data augmentation allows the model to perform more difficult tasks, thereby stimulating its learning ability; second, the reduction in text length after data augmentation allows more training samples to fit within the same graphics memory space, alleviating the problem of inconsistent prediction targets during model training; and third, by introducing guidance vectors, this method effectively integrates diverse information from a single source, significantly reducing the difficulty of the model directly learning difficult data.

[0084] This paper proposes a text data augmentation method for improving vector retrieval performance for retrieval tasks. This method leverages a large language model to decouple data to generate higher-quality training data, and then filters this data using quality assessment metrics. A two-stage training approach, particularly the construction of guidance vectors, fully utilizes the augmented data, ultimately improving the performance of the vector retrieval model.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A text data enhancement method for improving vector retrieval performance, characterized in that: The following steps are involved: S1. Acquire a data set and preprocess the data set; S2. Use the preprocessed data set to perform comparative learning training on the pre-trained model to obtain a trained primary retrieval model; S3. Based on the large language model, semantically decouple the positive example documents in the preprocessed dataset and decompose the positive example documents into several short texts; and filter the short text data to obtain an enhanced dataset in the format of (query, positive example, negative example, enhanced short text set); S4. Aggregate the data in the enhanced dataset based on the primary retrieval model to construct a guidance vector, and use the guidance vector to guide the training of the primary retrieval model to obtain a final retrieval model; the guidance vector is constructed as follows: Loading the primary retrieval model, dividing the enhanced dataset into several training batches; in a training task of a batch, encoding the query, positive examples, negative examples, and enhanced short text set in the batch into high-dimensional vectors, wherein the enhanced short text set does not pass through the pooling layer; The high-dimensional vectors of all short texts from the same query are added together to obtain a concatenated vector. The concatenated vector is then passed through a fully connected layer and a residual connection layer and projected into a high-level semantic space. The concatenated vector of each query is split according to the vector position, and clustered based on the Euclidean distance between the vectors at each position to generate several phrase vectors. For each query vector, the KL divergence between the query vector and each phrase vector is calculated, and the phrase vector corresponding to the minimum KL divergence is used as the guide vector.

2. A text data enhancement method for improving vector retrieval performance according to claim 1, characterized in that: In S1, the data set includes a query file, a query-positive example correspondence file, and a document collection file.

3. A text data enhancement method for improving vector retrieval performance according to claim 2, characterized in that: In S1, the data set is preprocessed, specifically: If the dataset contains a correspondence file between the query and the negative example, the positive example document and the negative example document are determined based on the correspondence file between the query and the positive example and the negative example respectively, and the dataset format is processed into (query, positive example, negative example); if the dataset lacks a correspondence file between the query and the negative example, the retrieval model is used to retrieve similar documents in the entire document collection as query negative examples, and the positive example document is determined based on the correspondence file between the query and the positive example, and the dataset format is processed into (query, positive example, negative example).

4. A text data enhancement method for improving vector retrieval performance according to claim 3, characterized in that: In S2, the training process of the primary retrieval model is as follows: S2.

1. Initialize its pre-trained checkpoint using the pre-trained model. The pre-trained model can act as an encoder to map all queries and positive examples in the dataset obtained in S1 into the same vector space. S2.

2. Divide the dataset obtained in S1 into several training batches. In the training task of any batch, first encode the query, positive document, and negative document in the batch into vectors simultaneously. S2.

3. In the vector space of the same training batch, for any query vector, its corresponding negative examples include all documents except its own positive example document. Calculate the distance between the query vector and each positive example document vector and each negative example document vector, and smooth the calculated distances using the softmax function to distribute them between (0, 1). Finally, use cross entropy as the loss function for training to obtain a trained primary retrieval model. The loss function formula is as follows: in, is the result of the softmax function processing of the distance between the query vector and the positive document vector or the negative document vector; y is the actual label. When the distance between the query vector and the positive document vector is calculated, y=1; when the distance between the query vector and the negative document vector is calculated, y=0.

5. A text data enhancement method for improving vector retrieval performance according to claim 3 or 4, characterized in that: The specific process of S3 is as follows: S3.

1. For a training set of the dataset in S1, construct a prompt template for each positive example document into an instruction input into the large language model, obtain a semantically decoupled short text output for each positive example document, and then change the training set format to (query, positive example, negative example, short text collection); S3.

2. For the short texts in the generated (query, positive examples, negative examples, short text collection), use the quality control indicator M to quantitatively evaluate the quality of the short texts. The calculation formula is as follows: Among them, Z is a positive example The corresponding short text set, 、 are two different short texts in the short text set Z; Express a positive example With the generated short text The similarity between 、 The two texts are encoded into a vector space using the primary retrieval model and then the cosine similarity of the two vectors is calculated; Represents short text Quality control indicators; S3.3, set the hyperparameters K and R. For all short text sets, the quality control index of each short text data must be greater than , retain at most K short texts with the largest quality control indicators to form an enhanced data set, the format of which is (query, positive example, negative example, enhanced short text set).

6. A text data enhancement method for improving vector retrieval performance according to claim 5, characterized in that: In S4, the loss function L is used for training and learning to obtain the final retrieval model; the loss function L = L1 + L2, which are as follows: For a set of queries and short texts, mark any short text corresponding to a query as a positive example and the rest as negative examples, and calculate the cross entropy as the loss function L1; in, Represents the cosine distance between vectors; Q represents the high-dimensional vector encoded by the query; S represents the high-dimensional vector encoded by the short text; y is the actual label, y=1 when the short text is a positive example, and y=0 when the short text is a negative example; For the guidance vector and the positive example, the cross entropy between the guidance vector corresponding to any query and the positive example corresponding to the query and the negative example corresponding to the query is calculated as the loss function L2; ) in, Represents the cosine distance of a vector; Represents the guidance vector; D represents the vector encoded by the positive or negative example; y is the actual label, when D is the vector encoded by the positive example, y=1, when D is the vector encoded by the negative example, y=0.

7. A text data enhancement method for improving vector retrieval performance according to claim 1, characterized in that: The pre-trained model includes RetroMAE or GTE.

8. A text data enhancement method for improving vector retrieval performance according to claim 1, characterized in that: The large language model includes GPT-4 or a general meaning model.

Images