Language model construction method and device, equipment and medium

Abstract

The invention relates to the technical field of computers, and discloses a language model construction method and device, equipment and a medium, and the method comprises the steps: obtaining a first image text pair set; the first image text pair set is input into a pre-constructed language model for iterative training, and the language model comprises a text extractor and a self-attention mechanism neural network sub-model; in the language model, image features and a plurality of word vectors included in a first image text pair in the first image text pair set are extracted through a text extractor, and the first image text pair is any one image text pair in the first image text pair set; and performing iterative training on the self-attention mechanism neural network sub-model by using the image features and / or the plurality of word vectors, and stopping training of the language model until the self-attention mechanism neural network sub-model reaches an iteration stopping condition, thereby obtaining a constructed language model.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a language model construction method, device, equipment and medium. Background Art

[0002] Dialogue systems have always been an important research direction in the field of artificial intelligence, aiming to achieve natural and coherent dialogues between machines and humans. In today's information age, with the rapid development of multimodal data processing and artificial intelligence technology, multimodal data has become a research field that has attracted much attention. Multimodal data usually contains multiple types of information, such as images, text, audio, etc. These information are interrelated and can provide a more comprehensive and rich perspective, which is of great significance for in-depth understanding and solving complex problems in the real world.

[0003] Traditional research mainly focuses on the processing and analysis of single-modal data, such as single image recognition, text classification and other tasks. However, real-world data often contains multiple types of information, and single-modal processing methods are difficult to meet the needs. Therefore, processing and analysis based on multimodal data has become a new research hotspot. The processing effect of existing multimodal models on single-modal tasks needs to be improved. Summary of the invention

[0004] In view of this, the present invention provides a language model construction method, device, equipment and medium to solve the problem in the related art that the dialogue system can only process and analyze data preferred by specific application scenarios and is not applicable to other application scenarios.

[0005] In a first aspect, the present invention provides a method for constructing a language model, the method comprising:

[0006] Obtain a first image-text pair set;

[0007] Inputting the first image-text pair set into a pre-built language model for iterative training, wherein the language model includes a text extractor and a self-attention mechanism neural network sub-model;

[0008] In the language model, the text extractor extracts image features and multiple word vectors included in the first image-text pair in the first image-text pair set, wherein the first image-text pair is any image-text pair in the first image-text pair set;

[0009] The self-attention mechanism neural network sub-model is iteratively trained using the image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stop condition, and the training of the language model is stopped to obtain the constructed language model.

[0010] The language model construction method provided by the present invention has the following advantages:

[0011] In this method, image features and text information are combined so that the language model can better understand the relationship between image content and text description, thereby improving the model's generalization ability and understanding of real-world scenarios. The dialogue system can more comprehensively understand the user's intentions and expressions, and generate more accurate and effective responses. For the pure text model, new features are added to improve the model's ability to handle tasks. Because the training process includes both image features and text information, the problem of difficulty in reducing the loss of simultaneous training of unimodal and multimodal tasks is solved, and ultimately the model can achieve good performance in multiple scenarios. Compared with the traditional dialogue system method, each input word is given a clearer feature and contains more information. By adding image features, image feature information is added to each word to make the description more precise. These features are also helpful for unimodal tasks.

[0012] Furthermore, by using a text extractor, image features and word vectors can be efficiently extracted from image-text pairs, reducing the amount of computation during training and improving training efficiency. The self-attention mechanism can automatically learn the correlation between different elements in the input data, allowing the model to pay more attention to the correspondence between important visual elements in the image and key words in the text description. Through iterative training, the language model can continuously optimize its parameters to more accurately match the complex relationship between images and text. Because the model combines image and text information, it can generate more natural and accurate text descriptions, especially when complex visual content is involved. By combining image and text information, the model can more accurately capture the image content, thereby reducing the error rate in text generation.

[0013] In an optional implementation, the text extractor is a trained text extractor, and before inputting the first image-text pair set into a pre-built language model for iterative training, the method further includes:

[0014] In the text extractor, a vectorization operation is performed on the text in each image-text pair in the first image-text pair set to obtain word segments corresponding to the text and convert them into a word embedding matrix, wherein the word embedding matrix includes a plurality of word vectors;

[0015] Performing a feature extraction operation on the image corresponding to the text to obtain image features;

[0016] In the current iteration round, a matrix constructed by a two-dimensional array is generated according to the image features and word vectors respectively generated for the first image-text pair, the word vectors respectively generated for other image-text pairs in the first image-text pair set except the first image-text pair, and the relationship between the image features and each of the word vectors, wherein the elements in the two-dimensional array include the image features and the word vectors, and the arrangement position of the two-dimensional array in the matrix is ​​determined according to the relationship between the image features and the word vectors;

[0017] Calculate a first cross entropy loss according to the elements in the two-dimensional array corresponding to the rows in the matrix respectively;

[0018] and calculating a second cross entropy loss according to the elements in the two-dimensional array corresponding to the vertical rows in the matrix;

[0019] Determining whether the loss function value corresponding to the text extractor meets the preset condition according to the first cross entropy loss and the second cross entropy loss;

[0020] When it is determined that the loss function value corresponding to the text extractor reaches the preset condition, the iterative training is stopped to obtain the trained text extractor.

[0021] Specifically, through vectorization operations and the construction of word embedding matrices, the text extractor can more accurately convert text information into vector representations, thereby improving the accuracy of extraction. By associating image features and word vectors, the model can better understand the relationship between image content and text descriptions and enhance the relevance of feature representations. By calculating the cross-entropy loss, the model can continuously optimize its parameters to reduce prediction errors and improve model performance. The preset loss function conditions can be used as a basis for stopping training, avoiding overfitting and improving training efficiency. When the loss function reaches the preset conditions, model training stops, ensuring the stability and predictability of the model. The trained text extractor can be used in a variety of practical applications, such as image content description, visual search, and image-text matching, enhancing the practicality of the model. Furthermore, the feature extraction part of the model is trained using the image-text alignment method, so that the image and text features obtained by the model during feature extraction can be mapped to the same space to a great extent, and the corresponding image-text features have a high degree of similarity. This can greatly improve the model's semantic understanding of images, greatly improve task scenarios using images, and provide better text features for the model's subsequent expansion of image generation tasks; at the same time, highly similar image-text features can make the text richer in information that can be used for clustering, greatly improving text tasks.

[0022] In an optional implementation, the self-attention mechanism neural network sub-model is iteratively trained using the image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stop condition, the training of the language model is stopped, and the constructed language model is obtained, including:

[0023] Using the image features and the plurality of word vectors, iteratively and cyclically train the self-attention mechanism neural network sub-model of the pre-constructed language model to obtain a first language model;

[0024] Using the plurality of word vectors, iteratively and cyclically train the self-attention mechanism neural network sub-model of the first language model to obtain a second language model;

[0025] The image features are used to iteratively and cyclically train the self-attention mechanism neural network sub-model of the second language model to obtain a third language model, wherein the third language model is the constructed language model.

[0026] Specifically, by combining image features and word vectors, the model can process visual and text information at the same time, thereby integrating multimodal information in the language model and improving the comprehensive understanding ability of the model. The self-attention mechanism can capture the complex relationship between words, and the combination of image features can further enrich the semantic understanding, so that the model has a deeper grasp of the context. Through iterative training, the model can learn richer language patterns, thereby generating more natural, fluent and accurate text. Iterative training helps the model learn a wider range of features and enhances the adaptability and generalization ability of the model to different types of text. In applications that require image and text interaction (such as question-and-answer systems, chatbots, etc.), this multimodal language model can provide a richer and more natural interactive experience. In an embodiment of the present application, the model is iteratively trained from three levels, including iterative training using image features and multiple word vectors, and then iterative training using multiple word vectors. Finally, the second language model is iteratively trained using image features to obtain a third language model. Iterative training can use the results of previous training to accelerate the subsequent training process and improve overall training efficiency. Combining training with image features can make the model more robust to noise and incomplete information, and improve the stability and accuracy of the model. Through iterative training, the model can better adapt to texts in different fields and topics, and based on the above training, it can be applied to single-modal fields or multi-modal fields alone. Improve the cross-domain adaptability of the model. Combining user image features and text preferences, the model can provide more personalized language generation services.

[0027] In an optional implementation, the using the image features and the plurality of word vectors to iteratively cycle train a self-attention mechanism neural network sub-model of a pre-constructed language model to obtain a first language model includes:

[0028] Sorting the plurality of word vectors according to semantics;

[0029] In each iteration cycle, k word vectors are selected from the sorted word vectors;

[0030] Input the image features and the k word vectors into the self-attention mechanism neural network sub-model of the pre-built language model, predict the k+1th word vector, and obtain the k+1th predicted word vector, wherein k is initially taken as a positive integer of T / 2, and as the number of iterative loop rounds increases once, the value of k is automatically increased by 1, and T is the total number of the multiple word vectors;

[0031] Stop the prediction operation until the target word that stops the iteration loop is recognized;

[0032] Calculate a first loss function value according to the predicted word vectors, the multiple word vectors, and the image features respectively obtained in all iterative loop rounds;

[0033] When the first loss function value reaches a first preset condition, the iterative training of the language model is stopped to obtain the first language model.

[0034] Specifically, by gradually increasing the number of word vectors, the model can gradually learn more complex semantic relationships, thereby improving the semantic understanding ability of the model. As the number of iterations increases, the model automatically increases the amount of learning data. This adaptive learning strategy helps the model better adapt to tasks of different difficulties. By gradually predicting word vectors based on the first-ranked word vectors combined with image features, the model can more accurately capture the correlation between word vectors and the correlation between word vectors and image features, thereby improving the accuracy of prediction. Gradually increasing the amount of learning data helps the model generalize to unseen data and improve the generalization ability of the model. By continuously adjusting and optimizing the model during the iteration process, the risk of overfitting can be reduced and the robustness of the model can be improved. This method is relatively easy to implement and does not require a complex model structure or additional computing resources. The process of gradually increasing the number of word vectors provides transparency into the model learning process and helps understand how the model gradually builds semantic representations. This method can effectively train a language model with better performance and better adaptability to complex semantic relationships.

[0035] In an optional implementation, the using the plurality of word vectors to iteratively cycle train the self-attention mechanism neural network sub-model of the first language model to obtain the second language model includes:

[0036] In each iteration cycle, k word vectors are selected from the sorted word vectors;

[0037] Input the k word vectors into the self-attention mechanism neural network sub-model of the first language model, predict the k+1th word vector, and obtain the k+1th predicted word vector;

[0038] The value of k is initially a positive integer of T / 2, and as the number of iterations increases, the value of k increases by 1, and T is the total number of the word vectors;

[0039] Stop the prediction operation until the target word that stops the iteration loop is recognized;

[0040] Calculate a second loss function value according to the predicted word vectors and the multiple word vectors respectively obtained in all iterative loop rounds;

[0041] When the second loss function value reaches a second preset condition, the iterative training of the first language model is stopped, and the second language model is obtained.

[0042] Specifically, by gradually increasing the number of selected word vectors (gradual increase in k), the model can gradually learn more complex language patterns and improve the accuracy of predicted word vectors. As the number of iterations increases, the model can handle more complex language structures and semantic relationships, thereby improving the model's complexity processing capabilities. By controlling the number of word vectors processed in each iteration, the use of computing resources can be optimized to avoid excessive computing burdens. Through iterative training, the model can learn a wider range of vocabulary and contextual information and improve the generalization ability of the model in different contexts. Iterative training of the self-attention mechanism helps to strengthen the model's internal understanding of the relationship between word vectors, enabling the model to better capture contextual information. Through iterative training and gradually increasing the complexity of training data, the model may reduce overfitting of specific training samples and improve its adaptability to new data. When the preset second loss function value is reached, training stops, which helps to quickly converge to satisfactory model performance and reduce unnecessary calculations. Through iterative training, we can better understand how the model gradually learns new vocabulary and complex language structures, thereby improving the interpretability of the model.

[0043] In an optional example, the iterative loop training of the self-attention mechanism neural network sub-model of the second language model using the image features to obtain the third language model includes:

[0044] Inputting the image features into the self-attention mechanism neural network sub-model of the second language model, predicting each word vector in turn, and obtaining a plurality of predicted word vectors;

[0045] Calculating a third loss function value according to the predicted word vectors and the image features respectively obtained in all iterative loop rounds;

[0046] When the third loss function value reaches a third preset condition, the iterative training of the second language model is stopped, and the third language model is obtained.

[0047] Specifically, by combining image features, the model can better understand the semantic relationship between text content and images, thereby improving the semantic accuracy of the language model. Using image features can increase the input information of the model, help the model learn richer features, and thus improve the generalization ability of the model when processing unseen text and image data. Image information can provide additional context to help reduce ambiguity in text expression, so that the model can predict word vectors more accurately. Through iterative cycle training, the model can continuously optimize its prediction ability, and the final third language model has better performance and generates text more naturally and fluently. This method can help build a cross-language model, namely a third language model, which can process text and image information between different languages ​​and promote cross-language understanding and communication. Although the introduction of image features increases the complexity of the model, by improving the generalization ability of the model, it may reduce the dependence on a large amount of specific language data. By optimizing the value of the third loss function, the model can converge to the optimal solution faster, thereby shortening the training time. The model combined with image features may be more robust when facing complex text content and can better handle noise and incomplete information. This method can effectively improve the performance of the language model, making it more accurate and efficient when processing complex text and image information.

[0048] In an optional implementation, the self-attention mechanism neural network sub-model is iteratively trained using the image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stop condition, the training of the language model is stopped, and after the constructed language model is obtained, the method further includes:

[0049] Acquire a first sample data set, wherein the first sample data set includes sample images, answers, and prompt information corresponding to the sample images; the answer corresponding to each of the sample images is a plurality of answers generated by the third language model, and the plurality of answers are sorted according to answer quality, and answers with answer quality higher than or equal to a preset threshold and answers with answer quality lower than the preset threshold are divided;

[0050] In each iterative training in the first iterative cycle round, the prompt information and an answer respectively corresponding to the sample image are input into the first feedback model to obtain a first reward value corresponding to each answer, wherein the first feedback model is a model generated by replacing the inverse operation layer of the embedding layer in the third language model with a linear layer;

[0051] Determine a first difference value according to the first reward values ​​corresponding to the answers whose answer quality is higher than or equal to the preset threshold and the first reward values ​​corresponding to the answers whose answer quality is lower than the preset threshold;

[0052] When the first difference value is greater than a first preset difference threshold, the iterative training is stopped to obtain a second feedback model;

[0053] In each iterative training of the second iterative cycle round, the sample image, the prompt information corresponding to the sample image, and an answer corresponding to the sample image are input into the second feedback model to obtain a second reward value corresponding to each answer;

[0054] Determine a second difference value according to the second reward values ​​corresponding to the answers whose answer quality is higher than or equal to the preset threshold and the second reward values ​​corresponding to the answers whose answer quality is lower than the preset threshold;

[0055] When the second difference value is greater than a second preset difference threshold, the iterative training is stopped to obtain a third feedback model;

[0056] Acquire a second sample data set, wherein the second sample data set includes sample images in the first sample data set, prompt information corresponding to the sample images, and a sample data set acquired by a third party, wherein the sample data set acquired by the third party includes third-party images and prompt information corresponding to the third-party images;

[0057] In each iterative training of the third iterative cycle round, each sample image in the second sample data set and the corresponding prompt information are input into the pre-built initial reinforcement learning model to obtain a predicted answer for the sample image, wherein the initial reinforcement learning model is the third language model;

[0058] Inputting the sample image, the predicted answer corresponding to the sample image, and the prompt information corresponding to the sample image into the third feedback model to obtain an evaluation factor;

[0059] The reinforcement learning model is iteratively trained using the evaluation factor and the predicted answer corresponding to the sample image until the relative entropy corresponding to the reinforcement learning model is less than a preset entropy threshold, the iterative training is stopped, and the final reinforcement learning model is obtained.

[0060] Specifically, the self-attention mechanism neural network sub-model is iteratively trained by combining image features, word vectors and feedback mechanisms to build a high-performance language model. Among them, by introducing image features and word vectors, the model can understand the input data more comprehensively, thereby improving the ability to understand the text content. Through iterative training and feedback mechanisms, the model can automatically adjust its prediction and generation strategies according to answers of different quality levels. Through preset thresholds, the model can distinguish between high-quality and low-quality answers, which helps to improve the quality of output content. Through iterative training of the first feedback model, the second feedback model and the third feedback model, the model can continuously optimize its prediction ability and ultimately improve the overall performance. Combined with reinforcement learning technology, the model can self-adjust according to actual output and expected goals to achieve more intelligent iterative training. By integrating third-party data sets, the model can be exposed to more diverse data, thereby improving the generalization ability and adaptability of the model. By setting a relative entropy threshold, the model can stop training after reaching a certain performance stability, avoiding overfitting and overtraining. Through evaluation factors, the model can make effective decisions on the reinforcement learning model, so that the model can learn as quickly as possible before reaching the preset performance. Through multi-stage iterative training, the model can gradually optimize its performance, ensuring that the final model has a high level of accuracy and efficiency. This method not only improves the performance of the language model in processing and generating text, but also enhances the model's ability to handle tasks combining images and text, and has broad application prospects.

[0061] In an optional implementation, the expression for calculating the first cross entropy loss is as follows:

[0062]

[0063] The expression for calculating the second cross entropy loss is as follows:

[0064]

[0065] Where i is the image feature after L2 normalization, t is the text feature after L2 normalization, C is the cross entropy, label is the attribute value corresponding to the relationship between the image feature and the word vector, dot is the dot product, norm is the modulus, axis=0 is the row axis, and axis=1 is the column axis. represents the first cross entropy loss, represents the second cross entropy loss;

[0066] Determining that the loss function value corresponding to the text extractor meets the preset condition comprises: when When it is less than or equal to the cross entropy loss threshold, it is determined that the loss function value corresponding to the text extractor meets the preset condition.

[0067] In an optional implementation, the expression of the first loss function is as follows:

[0068]

[0069] The expression of the second loss function is as follows:

[0070]

[0071] The expression of the third loss function is as follows:

[0072]

[0073] in, is the first loss function value, is the second loss function value, is the value of the third loss function, T is the total number of word vectors, and w T-k is the Tkth word vector, w i is the i-th word vector, v is the image feature, (w,v)~D is the data set consisting of the image and the word vector associated with it, P θ is the conditional probability, θ is the parameter to be adjusted in the language model, and E is the expected value.

[0074] In an optional implementation, the expression for calculating the first difference value is as follows:

[0075]

[0076] in, is the first difference value, (x, y w ,y l )~D is the answer set y consisting of prompt information x, sample image feature v, and quality higher than or equal to the preset threshold w , the answer set y whose quality is lower than the preset threshold l The data set is composed of: σ is: sigmoid function, r θ (x,y w ) is the answer set y with quality higher than or equal to the preset threshold w The set of first reward values ​​corresponding to each answer in r θ (x,v,y l ) is the answer set y whose quality is lower than the preset threshold lThe first reward value corresponding to each answer in the answer is a set; E represents the expected value; Q represents the number of answers corresponding to each prompt, because there are Q answers, so there will be Comparison.

[0077] The expression for calculating the second difference value is as follows:

[0078]

[0079] in, is the second difference value, r θ (x,v,y w ) is the answer set y with quality higher than or equal to the preset threshold w The set of second reward values ​​corresponding to each answer in , r θ (x,v,y l ) is the answer set y whose quality is lower than the preset threshold l The set consisting of the second reward value corresponding to each answer in .

[0080] In a second aspect, the present invention provides a language model construction device, the device comprising:

[0081] An acquisition module, used to acquire a first image-text pair set;

[0082] A training module is used to input the first image-text pair set into a pre-constructed language model for iterative training, wherein the language model includes a text extractor and a self-attention mechanism neural network sub-model; in the language model, the image features and multiple word vectors included in the first image-text pair in the first image-text pair set are extracted by the text extractor, wherein the first image-text pair is any image-text pair in the first image-text pair set; the self-attention mechanism neural network sub-model is iteratively trained using the image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stopping condition, then the training of the language model is stopped to obtain the constructed language model.

[0083] The language model construction device provided by the present invention has the following advantages:

[0084] Combining image features and text information enables the language model to better understand the relationship between image content and text description, thereby improving the model's generalization ability and understanding of real-world scenarios. The dialogue system can more comprehensively understand the user's intentions and expressions, and generate more accurate and effective responses. For the pure text model, new features are added to improve the model's ability to handle tasks. Because the training process includes both image features and text information, the problem of difficulty in reducing the loss of simultaneous training of single-modal tasks and multi-modal tasks is solved, and ultimately the model can achieve good performance in multiple scenarios. Compared with the traditional dialogue system method, each input word is given a clearer feature and contains more information. By adding image features, image feature information is added to each word to make the description more precise. These features are also helpful for single-modal tasks.

[0085] Furthermore, by using a text extractor, image features and word vectors can be efficiently extracted from image-text pairs, reducing the amount of computation during training and improving training efficiency. The self-attention mechanism can automatically learn the correlation between different elements in the input data, allowing the model to pay more attention to the correspondence between important visual elements in the image and key words in the text description. Through iterative training, the language model can continuously optimize its parameters to more accurately match the complex relationship between images and text. Because the model combines image and text information, it can generate more natural and accurate text descriptions, especially when complex visual content is involved. By combining image and text information, the model can more accurately capture the image content, thereby reducing the error rate in text generation.

[0086] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the language model construction method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0087] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the language model construction method of the first aspect or any corresponding embodiment thereof.

[0088] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the language model construction method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0090] Figure 1 It is a flow chart of a method for constructing a language model provided by an embodiment of the present invention;

[0091] Figure 2 is a flow chart of another language model construction method provided by an embodiment of the present invention;

[0092] Figure 3 It is an overall process architecture diagram of obtaining a text extractor provided by the present invention;

[0093] Figure 4 It is an overall process framework diagram of a language model training provided by the present invention;

[0094] Figure 5 It is a flowchart of another language model construction method provided by an embodiment of the present invention;

[0095] Figure 6 It is a schematic diagram of the overall process of a language model construction method provided by an embodiment of the present invention;

[0096] Figure 7 It is a structural block diagram of a language model building device provided by an embodiment of the present invention;

[0097] Figure 8 It is a schematic diagram of the hardware structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0098] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0099] An embodiment of the present invention provides an embodiment for building a language model. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system (computer device) including, for example, a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0100] In this embodiment, a language model construction method is provided, which can be used in the above-mentioned terminal devices, such as mobile phones, tablet computers, etc. Figure 1 is a flow chart of a method for constructing a language model provided by an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0101] Step S101, obtaining a first image-text pair set.

[0102] Step S102: input the first image-text pair set into a pre-built language model for iterative training.

[0103] Specifically, the language model includes a text extractor and a self-attention mechanism neural network sub-model.

[0104] Step S103: In the language model, extracting image features and multiple word vectors included in the first image-text pair in the first image-text pair set by a text extractor.

[0105] The first image-text pair is any image-text pair in the first image-text pair set.

[0106] Step S104, iteratively train the self-attention mechanism neural network sub-model using image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches the iteration stop condition, stop training the language model, and obtain the constructed language model.

[0107] Specifically, the text extractor is a trained text extractor. Before inputting the first image-text pair set into the pre-built language model for iterative training, the method further includes the following method steps, see Figure 2 As shown:

[0108] Step S201, in the text extractor, a vectorization operation is performed on the text in each image-text pair in the first image-text pair set, word segments corresponding to the text are obtained, and converted into a word embedding matrix.

[0109] Specifically, the text image data in the image-text set of pairs may be image-text pairs extracted from multimodal datasets such as NUS-WIDE, ImageNet, Conceptual Captions, and COCO.

[0110] For each image-text pair in the dataset, the image part uses the object detection method to segment the objects contained in the image; the text part uses the GPT dictionary to perform vectorization operations, which is specifically implemented by first adding [CLS] and [SEP] tags to the beginning and end of each sentence respectively. The sentence is segmented, and then each segmented word (token) is converted into a dictionary index. After entering the model, the index sequence is converted into a word embedding matrix. Therefore, the embedding matrix can include multiple word vectors.

[0111] Step S202: performing a feature extraction operation on the image corresponding to the text to obtain image features.

[0112] Specifically, for the extraction of image features, ViT can be used. ViT is used to process the image into a size of 224*224*3 and the output feature vector is an image feature of 768 dimensions, which is consistent with the dimension of the word vector in the word embedding matrix.

[0113] Step S203, in the current iteration round, generates a matrix constructed by a two-dimensional array based on the image features and word vectors generated for the first image-text pair, the word vectors generated for the other image-text pairs in the first image-text pair set except the first image-text pair, and the relationship between the image features and each word vector.

[0114] Specifically, the first image-text pair is any image-text pair in the first image-text pair set. The elements in the two-dimensional array include image features and word vectors, and the arrangement position of the two-dimensional array in the matrix is ​​determined according to the relationship between the image features and the word vectors.

[0115] In a specific example, since each image feature has an association relationship with multiple word vectors in the same image, and there may be no association relationship between the image feature and the word vector in the image that does not correspond to the image feature. Assume that the identification information corresponding to the two-dimensional array corresponding to the image feature and the word vector that has an association relationship is 1, and the identification information corresponding to the two-dimensional array between the image feature and the word vector that does not have an association relationship is 0, and in order to more clearly represent the positions of different two-dimensional arrays in the matrix, the positions of the two-dimensional arrays are identified with 0 and 1, and the final matrix can be identified in the form of a unit matrix. That is, in the same matrix, the arrangement position of the two-dimensional array composed of the word vector that has an association relationship with a certain image feature and the image feature is the position of 1 in the unit matrix, and the arrangement position of the two-dimensional array composed of the word vector that has no association relationship with the image feature and the image feature is the position of 0 in the unit matrix.

[0116] Step S204, calculating a first cross entropy loss according to the elements in the two-dimensional array corresponding to the rows in the matrix.

[0117] Step S205 , calculating a second cross entropy loss according to the elements in the two-dimensional array corresponding to the vertical rows in the matrix.

[0118] In an optional example, the expression for calculating the first cross entropy loss is as follows:

[0119]

[0120] The expression for calculating the second cross entropy loss is as follows:

[0121]

[0122] Among them, i is the image feature after L2 normalization, t is the text feature after L2 normalization, C is the cross entropy, label is the attribute value corresponding to the relationship between the image feature and the word vector, dot is the dot product, norm is the modulus, axis=0 is the row axis, and axis=1 is the column axis. represents the first cross entropy loss, represents the second cross entropy loss.

[0123] Step S206: Determine whether the loss function value corresponding to the text extractor meets a preset condition based on the first cross entropy loss and the second cross entropy loss.

[0124] Specifically, determining whether the loss function value corresponding to the text extractor meets the preset conditions includes:

[0125] when When it is less than or equal to the cross entropy loss threshold, it is determined that the loss function value corresponding to the text extractor meets the preset condition.

[0126] Step S207, when it is determined that the loss function value corresponding to the text extractor reaches a preset condition, stop the iterative training and obtain the trained text extractor.

[0127] Specifically, if the loss function value corresponding to the text extractor reaches the preset condition, the next iteration round can be entered, and the parameters in the text extractor can be adjusted and the above operation can be performed again. Extract any image-text pair other than the first image-text pair in the first image-text pair set, and all word vectors and perform the above operation again. Of course, it can also include re-extracting image features and word vectors from the beginning to perform the corresponding operation.

[0128] After the text extractor is determined, in the language model, the text extractor may be used to extract image features and multiple word vectors included in the first image-text pair in the first image-text pair set.

[0129] Then, the self-attention mechanism neural network sub-model is iteratively trained using image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches the iteration stop condition, the training of the language model is stopped, and the constructed language model is obtained.

[0130] Specifically, in order to ensure that the final trained language model can be more adaptable to a variety of different application scenarios. Therefore, during training, the self-attention mechanism neural network can be trained in different forms. For example, the self-attention mechanism neural network sub-model is trained based on image features. Then, the self-attention mechanism neural network sub-model is trained through multiple word vectors, or the self-attention mechanism neural network sub-model is trained in combination with image features and multiple word vectors, and so on, until each training is completed, and the final language model is obtained.

[0131] Figure 3 The overall process architecture diagram of obtaining the text extractor is shown in FIG. The specific implementation process has been introduced in detail in the previous article, so it will not be repeated here.

[0132] Figure 4 The overall process framework of language model training is shown in Figure 1. It includes inputting images and texts, preprocessing them respectively, and then inputting them into the text extractor to perform corresponding operations, obtaining output results, and then inputting them into the self-attention mechanism neural network sub-model for iterative training. When the self-attention mechanism neural network sub-model reaches the iteration stop condition, the training of the language model is stopped and the constructed language model is obtained.

[0133] The language model construction method provided in this embodiment combines image features and text information, so that the language model can better understand the relationship between image content and text description, thereby improving the generalization ability of the model and the understanding of real-world scenes. The dialogue system can understand the user's intentions and expressions more comprehensively and generate more accurate and effective responses. For the pure text model, new features are added to improve the model's ability to handle tasks. Because the training process includes both image features and text information, the problem that the loss of simultaneous training of single-modal tasks and multi-modal tasks is difficult to reduce is solved, and ultimately the model can achieve better performance in multiple scenarios. Compared with the traditional dialogue system method, each input word is given a clearer feature and contains more information. By adding image features, image feature information is added to each word to make the description more precise. These features are also helpful for single-mode tasks.

[0134] Furthermore, by using a text extractor, image features and word vectors can be efficiently extracted from image-text pairs, reducing the amount of computation during training and improving training efficiency. The self-attention mechanism can automatically learn the correlation between different elements in the input data, allowing the model to pay more attention to the correspondence between important visual elements in the image and key words in the text description. Through iterative training, the language model can continuously optimize its parameters to more accurately match the complex relationship between images and text. Because the model combines image and text information, it can generate more natural and accurate text descriptions, especially when complex visual content is involved. By combining image and text information, the model can more accurately capture the image content, thereby reducing the error rate in text generation.

[0135] In this embodiment, a language model construction method is provided, which can be used in the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 5 is a flow chart of a method for constructing a language model provided by an embodiment of the present invention. Figure 5 As shown, based on any of the foregoing embodiments, the self-attention mechanism neural network sub-model is iteratively trained using image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stop condition, the training of the language model is stopped, and the constructed language model is obtained, which may include the following steps:

[0136] Step S501, using image features and multiple word vectors, iteratively and cyclically train the self-attention mechanism neural network sub-model of the pre-constructed language model to obtain a first language model.

[0137] Specifically, multiple word vectors are divided into two parts, one part and the image features are input into the self-attention mechanism neural network sub-model, and the other part of the word vector is used as the output prediction target. The process stops when the next token is predicted to be the END token.

[0138] For example, the extracted image area is represented as v, and the extracted text is represented as w = {w1, ..., w T}. In this task, select part of the text w I ={w1,...,w k}(k<=T) is used as the text input of the model. The goal is to predict the subsequent text w based on the image features and the previous text. T-I ={w k+1 ,...,w T}Make predictions.

[0139] Repeat the above operation until the loss function value finally obtained meets the condition for stopping iteration, then it is determined that the training of the first language model is completed.

[0140] In a specific example, the method includes the following steps:

[0141] Step a1: sort multiple word vectors according to semantics.

[0142] Step a2: In each iteration loop round, k word vectors are selected from the sorted word vectors.

[0143] Step a3: input the image features and k word vectors into the self-attention mechanism neural network sub-model of the pre-built language model, predict the k+1th word vector, and obtain the k+1th predicted word vector.

[0144] The value of k is initially a positive integer of T / 2, and as the number of iterations increases, the value of k increases by 1, and T is the total number of multiple word vectors.

[0145] When the target word is identified, the prediction operation stops.

[0146] Step a4, calculating the first loss function value according to the predicted word vectors, multiple word vectors, and image features obtained in all iterative loop rounds.

[0147] Step a5: When the first loss function value reaches a first preset condition, stop iterative training of the language model and obtain a first language model.

[0148] In a specific example, the expression of the first loss function is as follows:

[0149]

[0150] in, is the first loss function value, T is the total number of word vectors, w T-k is the Tkth word vector, w i is the i-th word vector, v is the image feature, (w,v)~D is the data set consisting of the image and the word vector associated with it, P θ is the conditional probability, θ is the parameter to be adjusted in the language model, and E is the expected value.

[0151] Specifically, taking the image feature as v and the word vector including T as an example, the first half of the token of the word vector is used as input, and each token in the latter part will be used as the actual output result corresponding to a prediction result.

[0152] Then, based on each prediction result, the word vector input each time, the remaining word vectors not input, and the image features, the overall loss function is calculated using Formula 3 to obtain the first loss function value.

[0153] Based on the value of the first loss function, the trainable parameter θ in Formula 3 is continuously adjusted to optimize the model and finally obtain the first language model.

[0154] Step S502, using multiple word vectors, iteratively and cyclically train the self-attention mechanism neural network sub-model of the first language model to obtain a second language model.

[0155] Specifically, in this process, only some word vectors are selected and input into the self-attention mechanism neural network sub-model of the first language model, and the other word vectors are predicted to obtain the prediction results until the next token is predicted to be the END tag. The final loss function value is calculated until the loss function value meets the stop iteration condition, and then the second language model is obtained.

[0156] For example, the input word is represented as w = {w1, ..., w T}. In this task, select some representation words w I ={w1,...,w k}(k<=T) is used as the text input of the model. The goal is to predict the subsequent text w based on the previous text. T-I ={w k+1 ,...,w T}Make predictions.

[0157] In a specific example, the following steps are included:

[0158] Step b1: In each iterative loop round, k word vectors are selected from the sorted word vectors.

[0159] Step b2: input the k word vectors into the self-attention mechanism neural network sub-model of the first language model, predict the k+1th word vector, and obtain the k+1th predicted word vector.

[0160] The value of k is initially a positive integer of T / 2, and as the number of iterations increases, the value of k increases by 1, and T is the total number of multiple word vectors.

[0161] Stop the prediction operation until the target word that stops the iteration loop is recognized;

[0162] Step b3, calculating the second loss function value according to the predicted word vectors and multiple word vectors obtained in all iterative loop rounds.

[0163] Step b4: when the second loss function value reaches a second preset condition, stop iterative training of the first language model and obtain a second language model.

[0164] Among them, the expression of the second loss function is as follows:

[0165]

[0166] is the value of the second loss function.

[0167] Specifically, this process is similar to the principle of obtaining the first language model mentioned above, except that the input data only includes word vectors and does not involve image features. Therefore, the entire process does not involve image features.

[0168] However, the principles for determining whether the loss function value meets the preset conditions are similar. For example, a threshold can be set. If the direct difference between the corresponding loss function value calculated by Formula 3 or Formula 4 and the threshold corresponding to the threshold is less than the preset threshold, it means that the model training can be terminated and the corresponding language model can be obtained.

[0169] Step S503, using the image features to iteratively cycle train the self-attention mechanism neural network sub-model of the second language model to obtain a third language model.

[0170] In this method, the output prediction target is the corresponding text feature. Each text token is used as the output prediction target until the next token is predicted to be the END tag. The goal of the task is to predict text based on the semantic information of the image.

[0171] Specifically, the image area is represented as v, and the text is represented as w = {w1,...,w T}. The goal is to predict the text corresponding to the image features based on the image features.

[0172] Among them, the third language model is the constructed language model.

[0173] Specifically, the image features are used to iteratively train the self-attention mechanism neural network sub-model of the second language model to obtain the third language model, including:

[0174] The image features are input into the self-attention mechanism neural network sub-model of the second language model, and each word vector is predicted in turn to obtain multiple predicted word vectors;

[0175] Calculate the third loss function value according to the predicted word vectors and image features obtained in all iterative loop rounds;

[0176] When the third loss function value reaches a third preset condition, the iterative training of the second language model is stopped, and a third language model is obtained.

[0177] Among them, the expression of the third loss function is as follows:

[0178]

[0179] is the value of the third loss function.

[0180] Specifically, this process is similar to the principle of the aforementioned training model, except that this training is based on predicting word vectors based on image features.

[0181] Through the execution of the above three training tasks, the final model can have the following functions:

[0182] Based on images and partial language text, predict and improve language text. Based on images, predict complete language text. Through the above process, the language model itself already has multimodal application capabilities. Then, the model is strengthened and trained using image features or word vectors, and the language model finally obtained can also be applied to single-modal application scenarios. That is, the model can be applied to different application scenarios, process data of various modalities, and achieve smarter, more flexible, and more accurate dialogue interactions to better meet user needs.

[0183] In an optional example, the self-attention mechanism neural network sub-model is iteratively trained using image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stop condition, and the training of the language model is stopped. After the constructed language model is obtained, the method further includes the following method steps:

[0184] Step c1, obtaining a first sample data set.

[0185] Among them, the first sample data set (D2) includes sample images, answers, and prompt information corresponding to the sample images; the answer corresponding to each sample image is a plurality of answers generated by a third language model, and the plurality of answers are sorted according to the answer quality, and the answers whose answer quality is higher than or equal to a preset threshold and the answers whose answer quality is lower than the preset threshold are divided.

[0186] Step c2: After replacing the inverse operation layer of the embedding layer in the third language model with a linear layer, a first feedback model is obtained.

[0187] Step c3, in each iterative training in the first iterative cycle round, the prompt information and an answer corresponding to the sample image are input into the first feedback model to obtain a first reward value corresponding to each answer, wherein the first feedback model is a model generated by replacing the inverse operation layer of the embedding layer in the third language model with a linear layer.

[0188] Step c4, determining a first difference value according to the first reward values ​​corresponding to the answers whose answer quality is higher than or equal to the preset threshold and the first reward values ​​corresponding to the answers whose answer quality is lower than the preset threshold.

[0189] When the first difference value is greater than a first preset difference threshold, the iterative training is stopped to obtain a second feedback model.

[0190] Step c5, in each iterative training of the second iterative cycle round, the sample image, the prompt information corresponding to the sample image and an answer corresponding to the sample image are input into the second feedback model to obtain a second reward value corresponding to each answer.

[0191] Step c6, determining a second difference value according to the second reward values ​​corresponding to the answers whose answer quality is higher than or equal to the preset threshold and the second reward values ​​corresponding to the answers whose answer quality is lower than the preset threshold.

[0192] When the second difference value is greater than the second preset difference threshold, the iterative training is stopped to obtain the third feedback model.

[0193] Step c7: in each iterative training of the third iterative cycle, a second sample data set is obtained.

[0194] The second sample data set includes the sample images in the first sample data set, prompt information corresponding to the sample images, and the sample data set obtained by a third party, and the sample data set obtained by the third party includes the third-party images and prompt information corresponding to the third-party images;

[0195] Step c8, input each sample image in the second sample data set and the corresponding prompt information into the pre-built initial reinforcement learning model to obtain the predicted answer of the sample image, wherein the initial reinforcement learning model is the third language model.

[0196] Step c9: input the sample image, the predicted answer corresponding to the sample image, and the prompt information corresponding to the sample image into the third feedback model to obtain the evaluation factor.

[0197] Step c10, using the evaluation factor and the predicted answer corresponding to the sample image, iteratively train the reinforcement learning model until the relative entropy corresponding to the reinforcement learning model is less than the preset entropy threshold, then stop the iterative training and obtain the final reinforcement learning model.

[0198] In an optional example, the expression for calculating the first difference value is as follows:

[0199]

[0200] in, is the first difference value, (x,y w ,y l )~D is the answer set y consisting of prompt information x, sample image feature v, and quality higher than or equal to the preset thresholdw , the answer set y whose quality is lower than the preset threshold l The data set is composed of: σ is: sigmoid function, r θ (x,y w ) is the answer set y with quality higher than or equal to the preset threshold w The set of first reward values ​​corresponding to each answer in r θ (x,v,y l ) is the answer set y whose quality is lower than the preset threshold l The set of the first reward values ​​corresponding to each answer in , E is the expected value;

[0201] The expression for calculating the second difference value is as follows:

[0202]

[0203] in, is the second difference value, r θ (x,v,y w ) is the answer set y with quality higher than or equal to the preset threshold w The set of second reward values ​​corresponding to each answer in , r θ (x,v,y l ) is the answer set y whose quality is lower than the preset threshold l The set consisting of the second reward value corresponding to each answer in .

[0204] In a specific example, the data used in the first sample data set is mainly composed of a generated data set and a manually annotated data set. Among them, the data of the manually annotated data set comes from the generated data set. The generated data set contains images, prompts and answers. The prompt is a question or the first half of a complete sentence. The image has a certain correlation with the prompt, and the answer is the answer corresponding to the question or the second half of the complete sentence. In order to get the answer, it is necessary to first process it through the final language model obtained after the training, and then input the image and prompt into LM to get it.

[0205] In an embodiment of the present application, a beam search method can be used to generate Q different answers corresponding to the same prompt, that is, the data format of the generated data set is one prompt corresponding to Q answers. The manually annotated data set is derived from the generated data set. The specific method is: manually sort the answer part of the generated data set according to its quality, that is, sort one prompt corresponding to one answer. This can reduce the complexity of the manually annotated data set, because the complexity of sorting and annotating the prompts is much lower than the complexity involved in generating answers to the prompts.

[0206] The output layer of the language model finally obtained is modified, and the inverse operation layer of the embedding layer in the third language model is replaced with a linear layer to obtain the model architecture of the first feedback model (Reward Model, referred to as RM). The output is a scalar (i.e., output score), which represents the quality of the answer. The higher the score, the higher the quality.

[0207] The model RLM is obtained by pre-training the model using multiple pre-training tasks.

[0208] Task 1 (first iteration):

[0209] The prompt and an answer in the generated data set are input into the first feedback model. Since the manually annotated data set has one prompt corresponding to Q answers, this step needs to be repeated K times for one data, that is, the first reward value corresponding to the Q answers is obtained, and the first reward value satisfies the ranking of the manually annotated data set. The specific implementation process is as follows:

[0210] Assume that the prompt is denoted as x, and the Q answers corresponding to the prompt are denoted as y = {y1,...,y Q}, where the quality ranking order is y1>y2>…>y Q , denoting the higher quality region as y w ={y1,...,y w}, the lower quality area is denoted by y l ={y w+1 ,...,y Q}.

[0211] Q represents the number of answers corresponding to each clue. Since there are Q answers, there will be Comparison. θ (x,y w ) is the first reward value obtained by inputting the prompt and high-quality answer into the first feedback model, r θ (x,y l ) is the first reward value obtained after inputting the prompt and the lower quality answer into the first feedback model. The purpose of Task 1 is to make the difference between the first reward value of the high-quality answer and the first reward value of the low-quality answer as large as possible. In other words, the larger the first difference value, the better. The expression formula of the first difference value is shown in Formula 5.

[0212] Task 2 (second iteration):

[0213] The image, prompt and one answer in the generated dataset are input into the second feedback model. Since the manually annotated dataset has one prompt corresponding to Q answers, this step needs to be repeated Q times for one piece of data, that is, the second reward value corresponding to the Q answers is obtained, and this second reward value satisfies the ranking of the manually annotated dataset. The specific implementation process is as follows:

[0214] The text and the corresponding image are represented as (x, v), and the corresponding Q outputs are represented as y = {y1,...,y Q}, where the quality ranking order is y1>y2>…>y Q , denoting the higher quality region as y w ={y1,...,y w}, the lower quality area is denoted by y l ={y w+1 ,...,y Q}. Q represents the number of outputs corresponding to each prompt. Since there are Q outputs, there will be Comparison. θ (x,y w ) is the second reward obtained by feeding the prompt and high-quality answer into the model, r θ (x,y l ) is the second reward value obtained by inputting the prompt and the lower quality answer into the model. The task objective is to make the second difference between the second reward value of the high-quality answer and the second reward value of the low-quality answer as large as possible. For the specific formula for obtaining the second difference value, see Formula 6.

[0215] When the above steps are performed and the above conditions are met, it is determined that the RM model has been trained.

[0216] According to the trained RM model, the language model trained in the previous article is further strengthened using the PPO algorithm.

[0217] First, obtain the second sample data set.

[0218] The second sample data set includes the sample images in the first sample data set, prompt information corresponding to the sample images, and the sample data set obtained by a third party. The sample data set obtained by the third party includes third-party images and prompt information corresponding to the third-party images.

[0219] Specifically, the sample data set obtained by the third party includes, for example, a large amount of corpus selected from other corpora to be added to the data set, including but not limited to questions, first half sentences, dialogues, etc. The text and image parts of the data set need to be processed according to the operation steps of step S201 described above before being input into the model.

[0220] Then, the model is trained using the PPO algorithm.

[0221] Input the image (x, v) in the dataset into the pre-built reinforcement learning model RLM to generate the corresponding output y, which is the predicted answer corresponding to the sample image. Then input the three together into the third feedback model RM to obtain an evaluation factor r θ , and use y θ Fine-tune the model.

[0222] Specifically, KL divergence can be used to restrict the model so that it cannot deviate significantly from the original LM model in order to obtain a higher evaluation factor.

[0223] The specific expression is as follows:

[0224]

[0225] in, Represents the RLM model, which is the model that needs to be optimized in this training; π LM represents the LM model, β and γ are adjustment coefficients. pretrain is the dataset before the answer is generated, that is, the dataset including images and prompts, is the initial model of the RLM model, that is, the loss function value output by the LM model after processing the data set before generating the answer. represents the KL divergence. The meaning of formula 7 is to make the output standardized, and the difference between the RLM model and the LM model should not be too big. The whole RLM model is optimized using the fractional value. As a constraint model, it is used to indicate that the RLM model cannot deviate from the original output. In other words, it cannot excessively pursue high scores, causing the predicted output results to deviate too much from the actual output results.

[0226] Figure 6 The overall process diagram of the above embodiment is shown in FIG. 1 , including preprocessing the original data set to obtain D1, that is, Figure 1 The first image-text pair set introduced in the corresponding embodiment. Iteratively train the language model using the first image-text pair set to obtain a language model LM that meets the conditions for stopping iteration. Input the sample image data and the prompt information corresponding to the sample image data into the LM model to generate Q answers. Based on the sample image, the Q answers and the prompt information, generate a first sample data set D2. Then, iteratively train the RM model based on D2 to obtain a trained RM model. Then, train the RM model using the PPO algorithm. During the training process, a predicted answer is generated based on the prompt information and the sample image after being input into the RLM model. Input the answer, prompt information, and sample image into the RM model together, and then get a reward y.θ (r shown in the figure). So use r θ Fine-tune the model. The above process has been explained in detail in the previous article, so I will not go into details here.

[0227] A language model construction method provided by an embodiment of the present invention uses a model training process of reinforcement learning and PPO, and on this basis, innovatively uses newly designed multimodal tasks to train the model. These tasks are more suitable for the actual application environment of the dialogue system, so that the model can also obtain better performance in small sample tasks, and can achieve zero-shot in some tasks. Due to the use of reinforcement learning and PPO, compared with the existing multimodal and unimodal dialogue systems, the model output of the embodiment of the present application will be closer to human expectations. Moreover, corresponding image features are added in the training process of the language model. For the pure text model, new features are added. The design of the pre-training method and pre-training tasks involving the model improves the model's ability to handle tasks. A new pre-training task is designed so that the model can effectively use multimodal data and unimodal data for training together, and solves the problem that it is difficult to reduce the loss of simultaneous training of unimodal tasks and multimodal tasks, and ultimately enables the model to achieve better performance in multiple scenarios. Compared with traditional dialogue system training methods, each input word is given clearer features and contains more information. By adding image features, image feature information is added to each word to make the description more precise. These features are also helpful for single-mode tasks.

[0228] In this embodiment, a language model construction device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0229] This embodiment provides a language model construction device, such as Figure 7 As shown, it includes: an acquisition module 701 and a training module 702.

[0230] An acquisition module 701 is used to acquire a first image-text pair set;

[0231] The training module 702 is used to input the first image-text pair set into a pre-constructed language model for iterative training, wherein the language model includes a text extractor and a self-attention mechanism neural network sub-model; in the language model, the image features and multiple word vectors included in the first image-text pair in the first image-text pair set are extracted by the text extractor, wherein the first image-text pair is any image-text pair in the first image-text pair set; the self-attention mechanism neural network sub-model is iteratively trained using the image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stopping condition, and the training of the language model is stopped to obtain the constructed language model.

[0232] In an optional example, the device further includes: a processing module 703 , an extraction module 704 , and a generation module 705 .

[0233] The processing module 703 is used to perform a vectorization operation on the text in each image-text pair in the first image-text pair set in the text extractor, obtain the word segmentation corresponding to the text, and convert it into a word embedding matrix, wherein the word embedding matrix includes multiple word vectors.

[0234] An extraction module 704 is used to perform a feature extraction operation on the image corresponding to the text to obtain image features;

[0235] A generating module 705 is used to generate a matrix constructed by a two-dimensional array in a current iteration round according to the image features and word vectors respectively generated for the first image-text pair, the word vectors respectively generated for the other image-text pairs in the first image-text pair set except the first image-text pair, and the relationship between the image features and each word vector, wherein the elements in the two-dimensional array include the image features and the word vectors, and the arrangement position of the two-dimensional array in the matrix is ​​determined according to the relationship between the image features and the word vectors;

[0236] The processing module 703 is further used to calculate the first cross entropy loss according to the elements in the two-dimensional array corresponding to the rows in the matrix;

[0237] And calculate the second cross entropy loss according to the elements in the two-dimensional array corresponding to the vertical rows in the matrix;

[0238] Determining whether a loss function value corresponding to the text extractor meets a preset condition according to the first cross entropy loss and the second cross entropy loss;

[0239] When it is determined that the loss function value corresponding to the text extractor reaches a preset condition, the iterative training is stopped to obtain the trained text extractor.

[0240] In an optional example, the training module 702 is specifically configured to:

[0241] Using image features and multiple word vectors, the self-attention mechanism neural network sub-model of the pre-built language model is iteratively trained to obtain the first language model;

[0242] Using multiple word vectors, the self-attention mechanism neural network sub-model of the first language model is iteratively trained to obtain the second language model.

[0243] The image features are used to iteratively and cyclically train the self-attention mechanism neural network sub-model of the second language model to obtain a third language model, wherein the third language model is the constructed language model.

[0244] In an optional example, the training module 702 is specifically configured to:

[0245] Sort multiple word vectors according to semantics;

[0246] In each iteration cycle, k word vectors are selected from the sorted word vectors;

[0247] Input the image features and k word vectors into the self-attention mechanism neural network sub-model of the pre-built language model, predict the k+1th word vector, and obtain the k+1th predicted word vector, where k is a positive integer of T / 2 for the first time, and as the number of iterative loop rounds increases once, the value of k increases by 1, and T is the total number of multiple word vectors;

[0248] Stop the prediction operation until the target word that stops the iteration loop is recognized;

[0249] Calculate a first loss function value according to the predicted word vectors, multiple word vectors, and image features obtained in all iterative loop rounds;

[0250] When the first loss function value reaches a first preset condition, the iterative training of the language model is stopped to obtain a first language model.

[0251] In an optional example, the training module 702 is specifically configured to:

[0252] In each iteration cycle, k word vectors are selected from the sorted word vectors;

[0253] Input the k word vectors into the self-attention mechanism neural network sub-model of the first language model, predict the k+1th word vector, and obtain the k+1th predicted word vector;

[0254] The value of k is initially a positive integer of T / 2, and as the number of iterations increases, the value of k increases by 1, and T is the total number of multiple word vectors.

[0255] Stop the prediction operation until the target word that stops the iteration loop is recognized;

[0256] Calculate the second loss function value according to the predicted word vectors and multiple word vectors obtained in all iterative loop rounds;

[0257] When the second loss function value reaches a second preset condition, the iterative training of the first language model is stopped, and the second language model is obtained.

[0258] In an optional example, the training module 702 is specifically configured to:

[0259] The image features are input into the self-attention mechanism neural network sub-model of the second language model, and each word vector is predicted in turn to obtain multiple predicted word vectors;

[0260] Calculate the third loss function value according to the predicted word vectors and image features obtained in all iterative loop rounds;

[0261] When the third loss function value reaches a third preset condition, the iterative training of the second language model is stopped, and a third language model is obtained.

[0262] In an optional example, the acquisition module 701 is further used to acquire a first sample data set, wherein the first sample data set includes a sample image, an answer, and prompt information corresponding to the sample image; the answer corresponding to each sample image is a plurality of answers generated by a third language model, and the plurality of answers are sorted by answer quality, and answers with answer quality higher than or equal to a preset threshold and answers with answer quality lower than the preset threshold are divided;

[0263] The training module 702 is further used to input the prompt information and an answer corresponding to the sample image into the first feedback model in each iteration training in the first iteration cycle round, and obtain a first reward value corresponding to each answer, wherein the first feedback model is a model generated by replacing the inverse operation layer of the embedding layer in the third language model with a linear layer; determine a first difference value according to the first reward value corresponding to the answer whose answer quality is higher than or equal to the preset threshold and the first reward value corresponding to the answer whose answer quality is lower than the preset threshold; until the first difference value is greater than the first preset difference threshold, stop the iteration training and obtain the second feedback model; in each iteration training of the second iteration cycle round, input the sample image, the prompt information corresponding to the sample image and the answer corresponding to the sample image into the second feedback model, and obtain a second reward value corresponding to each answer; determine a second difference value according to the second reward value corresponding to the answer whose answer quality is higher than or equal to the preset threshold and the second reward value corresponding to the answer whose answer quality is lower than the preset threshold; until the second difference value is greater than the second preset difference threshold, stop the iteration training and obtain the third feedback model;

[0264] The acquisition module 701 is further used to acquire a second sample data set, wherein the second sample data set includes the sample images in the first sample data set, prompt information corresponding to the sample images, and a sample data set acquired by a third party, wherein the sample data set acquired by the third party includes a third party image and prompt information corresponding to the third party image;

[0265] The training module 702 is also used to input each sample image in the second sample data set and the corresponding prompt information into a pre-built initial reinforcement learning model in each iterative training of the third iterative cycle round to obtain a predicted answer for the sample image, wherein the initial reinforcement learning model is a third language model; input the sample image and the predicted answer corresponding to the sample image, as well as the prompt information corresponding to the sample image into a third feedback model to obtain an evaluation factor; and iteratively train the reinforcement learning model using the evaluation factor and the predicted answer corresponding to the sample image until the relative entropy corresponding to the reinforcement learning model is less than a preset entropy threshold, then stop the iterative training and obtain the final reinforcement learning model.

[0266] The language model building device in this embodiment is presented in the form of a functional module, where the module refers to an application specific integrated circuit (ASIC), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0267] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0268] A language model construction device provided by an embodiment of the present invention combines image features and text information, so that the language model can better understand the relationship between image content and text description, thereby improving the generalization ability of the model and the understanding of real-world scenes. The dialogue system can understand the user's intentions and expressions more comprehensively and generate more accurate and effective responses. For the pure text model, new features are added to improve the model's ability to handle tasks. Because the training process includes both image features and text information, the problem that the loss of simultaneous training of single-modal tasks and multi-modal tasks is difficult to reduce is solved, and ultimately the model can achieve better performance in multiple scenarios. Compared with the traditional dialogue system method, each input word is given a clearer feature and contains more information. By adding image features, image feature information is added to each word to make the description more precise. These features are also helpful for single-mode tasks.

[0269] Furthermore, by using a text extractor, image features and word vectors can be efficiently extracted from image-text pairs, reducing the amount of computation during training and improving training efficiency. The self-attention mechanism can automatically learn the correlation between different elements in the input data, allowing the model to pay more attention to the correspondence between important visual elements in the image and key words in the text description. Through iterative training, the language model can continuously optimize its parameters to more accurately match the complex relationship between images and text. Because the model combines image and text information, it can generate more natural and accurate text descriptions, especially when complex visual content is involved. By combining image and text information, the model can more accurately capture the image content, thereby reducing the error rate in text generation.

[0270] The embodiment of the present invention also provides a computer device having the above Figure 7 The language model building device shown.

[0271] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0272] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include an integrated circuit. The integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0273] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0274] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0275] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0276] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0277] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0278] The embodiment of the present invention also provides a computer-readable storage medium. The method provided in the above embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or is implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0279] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0280] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for constructing a language model, characterized in that: The method comprises: Obtain a first image-text pair set; Inputting the first image-text pair set into a pre-built language model for iterative training, wherein the language model includes a text extractor and a self-attention mechanism neural network sub-model; In the language model, the text extractor extracts image features and multiple word vectors included in the first image-text pair in the first image-text pair set, wherein the first image-text pair is any image-text pair in the first image-text pair set; The self-attention mechanism neural network sub-model is iteratively trained using the image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stop condition, and the training of the language model is stopped to obtain the constructed language model.

2. The method according to claim 1, characterized in that: The text extractor is a trained text extractor. Before inputting the first image-text pair set into a pre-built language model for iterative training, the method further includes: In the text extractor, a vectorization operation is performed on the text in each image-text pair in the first image-text pair set to obtain word segments corresponding to the text and convert them into a word embedding matrix, wherein the word embedding matrix includes a plurality of word vectors; Performing a feature extraction operation on an image corresponding to the text to obtain image features; In the current iteration round, a matrix constructed by a two-dimensional array is generated according to the image features and word vectors respectively generated for the first image-text pair, the word vectors respectively generated for other image-text pairs in the first image-text pair set except the first image-text pair, and the relationship between the image features and each of the word vectors, wherein the elements in the two-dimensional array include the image features and the word vectors, and the arrangement position of the two-dimensional array in the matrix is ​​determined according to the relationship between the image features and the word vectors; Calculate a first cross entropy loss according to the elements in the two-dimensional array corresponding to the rows in the matrix respectively; and calculating a second cross entropy loss according to the elements in the two-dimensional array corresponding to the vertical rows in the matrix; Determining whether a loss function value corresponding to the text extractor meets a preset condition according to the first cross entropy loss and the second cross entropy loss; When it is determined that the loss function value corresponding to the text extractor reaches the preset condition, the iterative training is stopped to obtain the trained text extractor.

3. The method according to claim 1 or 2, characterized in that: Iteratively training the self-attention mechanism neural network sub-model by using the image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stop condition, stopping the training of the language model, and obtaining the constructed language model, specifically including: Using the image features and the plurality of word vectors, iteratively and cyclically train the self-attention mechanism neural network sub-model of the pre-constructed language model to obtain a first language model; Using the plurality of word vectors, iteratively and cyclically train the self-attention mechanism neural network sub-model of the first language model to obtain a second language model; The image features are used to iteratively and cyclically train the self-attention mechanism neural network sub-model of the second language model to obtain a third language model, wherein the third language model is the constructed language model.

4. The method according to claim 3, characterized in that The method of using the image features and the plurality of word vectors to iteratively train the self-attention mechanism neural network sub-model of the pre-constructed language model to obtain a first language model includes: Sorting the plurality of word vectors according to semantics; In each iteration cycle, k word vectors are selected from the sorted word vectors; Input the image features and the k word vectors into the self-attention mechanism neural network sub-model of the pre-built language model, predict the k+1th word vector, and obtain the k+1th predicted word vector, wherein k is initially taken as a positive integer of T / 2, and as the number of iterative loop rounds increases once, the value of k is automatically increased by 1, and T is the total number of the multiple word vectors; Stop the prediction operation until the target word that stops the iteration loop is recognized; Calculate a first loss function value according to the predicted word vectors, the multiple word vectors, and the image features respectively obtained in all iterative loop rounds; When the first loss function value reaches a first preset condition, the iterative training of the language model is stopped to obtain the first language model.

5. The method according to claim 4, characterized in that The step of using the plurality of word vectors to iteratively and cyclically train the self-attention mechanism neural network sub-model of the first language model to obtain a second language model includes: In each iteration cycle, k word vectors are selected from the sorted word vectors; Input the k word vectors into the self-attention mechanism neural network sub-model of the first language model, predict the k+1th word vector, and obtain the k+1th predicted word vector; The value of k is initially a positive integer of T / 2, and as the number of iterations increases, the value of k increases by 1, and T is the total number of the word vectors; Stop the prediction operation until the target word that stops the iteration loop is recognized; Calculate a second loss function value according to the predicted word vectors and the multiple word vectors respectively obtained in all iterative loop rounds; When the second loss function value reaches a second preset condition, the iterative training of the first language model is stopped, and the second language model is obtained.

6. The method according to claim 4, characterized in that The iterative cycle training of the self-attention mechanism neural network sub-model of the second language model by using the image features to obtain a third language model includes: Inputting the image features into the self-attention mechanism neural network sub-model of the second language model, predicting each word vector in turn, and obtaining a plurality of predicted word vectors; Calculating a third loss function value according to the predicted word vectors and the image features respectively obtained in all iterative loop rounds; When the third loss function value reaches a third preset condition, the iterative training of the second language model is stopped, and the third language model is obtained.

7. The method according to any one of claim 3, characterized in that: The method further comprises: iteratively training the self-attention mechanism neural network sub-model by using the image features and / or multiple word vectors until the self-attention mechanism neural network sub-model reaches an iteration stop condition, stopping the training of the language model, and obtaining the constructed language model. Acquire a first sample data set, wherein the first sample data set includes sample images, answers, and prompt information corresponding to the sample images; the answer corresponding to each of the sample images is a plurality of answers generated by the third language model, and the plurality of answers are sorted according to answer quality, and answers with answer quality higher than or equal to a preset threshold and answers with answer quality lower than the preset threshold are divided; In each iterative training in the first iterative cycle round, the prompt information and an answer respectively corresponding to the sample image are input into the first feedback model to obtain a first reward value corresponding to each answer, wherein the first feedback model is a model generated by replacing the inverse operation layer of the embedding layer in the third language model with a linear layer; Determine a first difference value according to the first reward values ​​corresponding to the answers whose answer quality is higher than or equal to the preset threshold and the first reward values ​​corresponding to the answers whose answer quality is lower than the preset threshold; When the first difference value is greater than a first preset difference threshold, the iterative training is stopped to obtain a second feedback model; In each iterative training of the second iterative cycle round, the sample image, the prompt information corresponding to the sample image, and an answer corresponding to the sample image are input into the second feedback model to obtain a second reward value corresponding to each answer; Determine a second difference value according to the second reward values ​​corresponding to the answers whose answer quality is higher than or equal to the preset threshold and the second reward values ​​corresponding to the answers whose answer quality is lower than the preset threshold; When the second difference value is greater than a second preset difference threshold, the iterative training is stopped to obtain a third feedback model; Acquire a second sample data set, wherein the second sample data set includes sample images in the first sample data set, prompt information corresponding to the sample images, and a sample data set acquired by a third party, wherein the sample data set acquired by the third party includes third-party images and prompt information corresponding to the third-party images; In each iterative training of the third iterative cycle round, each sample image in the second sample data set and the corresponding prompt information are input into the pre-built initial reinforcement learning model to obtain a predicted answer for the sample image, wherein the initial reinforcement learning model is the third language model; Inputting the sample image, the predicted answer corresponding to the sample image, and the prompt information corresponding to the sample image into the third feedback model to obtain an evaluation factor; The reinforcement learning model is iteratively trained using the evaluation factor and the predicted answer corresponding to the sample image until the relative entropy corresponding to the reinforcement learning model is less than a preset entropy threshold, the iterative training is stopped, and the final reinforcement learning model is obtained.

8. The method according to claim 2, characterized in that: The expression for calculating the first cross entropy loss is as follows: The expression for calculating the second cross entropy loss is as follows: Where i is the image feature after L2 normalization, t is the text feature after L2 normalization, C is the cross entropy, label is the attribute value corresponding to the relationship between the image feature and the word vector, dot is the dot product, norm is the modulus, axis=0 is the row axis, and axis=1 is the column axis. represents the first cross entropy loss, represents the second cross entropy loss; Determining that the loss function value corresponding to the text extractor meets the preset condition comprises: when When it is less than or equal to the cross entropy loss threshold, it is determined that the loss function value corresponding to the text extractor meets the preset condition.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the language model construction method according to any one of claims 1 to 8 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the language model construction method according to any one of claims 1 to 8.

Images