Display device and iterative optimization method of language model

By combining comparative learning methods and quality evaluation models, iteratively optimized the language model, the problem of poor training effect of language model is solved, the accuracy and generalization ability of output reply words are improved, and the user experience is improved.

CN118916452APending Publication Date: 2024-11-08HISENSE VISUAL TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410914148.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-09
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the prior art, the training effect of the language model is poor, resulting in low accuracy of the output reply language and affecting the user experience. It is mainly due to the low data labeling efficiency and uneven data quality, resulting in a decrease in the generalization ability of the supervised fine-tuning method.

Method used

By obtaining the Q&A dataset, generating the training dataset and test dataset, using the quality evaluation model to evaluate the scores of the reply corpus, updating the learning status marks, and iteratively optimizing the language model through comparative learning methods, deleting the sample data of the learning successful state, and retaining the learning Sample data of failed states to improve the output accuracy of the language model.

Benefits of technology

It improves the accuracy of the language model output reply language, enhances the generalization ability and robustness of the model, and improves the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916452A_ABST
    Figure CN118916452A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a display device and an iterative optimization method for a language model, and the method can obtain a question and answer data set, generates a training data set and a test data set, and carries out the training of a target language model through the training data set. Evaluating the third reply corpus by using a quality evaluation model so as to update a learning state mark of the sample data in the training data set; inputting a question corpus of the test data set into the target language model, and marking a training state of the target language model based on an output fourth reply corpus; and if the training state is an iterative optimization state, deleting the sample data in the learning success state, and retaining the sample data in the learning failure state to update the training data set. According to the method, a training data set is updated by deleting learned sample data, and iterative training is performed on a target language model in a comparative learning mode, so that the accuracy of outputting reply languages by the target language model is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to an iterative optimization method for a display device and a language model. Background Art

[0002] The language model is a deep learning model trained based on text data, which can be applied to technical fields such as voice interaction control of display devices and image recognition. The language model can send corresponding replies to users based on their questions. In order to improve the generalization ability of the language model and the quality of the replies generated by the language model, it is necessary to select diverse and highly accurate training data to train the language model.

[0003] Using supervised fine-tuning to train a language model allows the language model to learn language knowledge from limited training data. The supervised fine-tuning method uses data with target task labels to perform further training and adjustments on the pre-trained model, thereby optimizing the performance of the language model on the target task. However, the above method requires labeling the training data to obtain data with target task labels. The data labeling process is inefficient, and the quality of the labeled data varies. Therefore, using supervised fine-tuning to train a language model will result in poor training results for the language model, and the accuracy of the response output by the language model is low, affecting the user experience. Summary of the invention

[0004] Some embodiments of the present application provide an iterative optimization method for a display device and a language model to solve the problem of low accuracy of reply output by the language model.

[0005] In a first aspect, some embodiments of the present application provide a display device, comprising: a display and a controller. The display is configured to display a user interface, and the controller is configured to perform the following steps:

[0006] Acquire a question-and-answer dataset, the question-and-answer dataset comprising a first preset number of sample data, the sample data comprising a question corpus, a first reply corpus, a second reply corpus and a learning status mark, the first reply corpus is provided with a first score generated based on a quality assessment model, the second reply corpus is provided with a second score generated based on the quality assessment model; the first score is higher than the second score; the learning status mark comprises a learning success state, a learning failure state and a non-learning state; the quality assessment model is a deep learning model that performs standardized assessment on a text corpus;

[0007] Generate a training data set and a test data set based on the question-answer data set, wherein the training data set includes a second preset amount of the sample data, and the test data set includes a third preset amount of the sample data;

[0008] Using the training data set to train a target language model, and obtaining a third response corpus output by the target language model, wherein the third response corpus is a response language generated based on the question corpus in the training data set;

[0009] Inputting the third reply corpus into the quality assessment model to obtain a third score, and updating the learning state mark of the sample data in the training data set based on the assessment difference, wherein the assessment difference is the difference between the first score and the third score;

[0010] Inputting the question corpus of the test data set into the target language model, and marking the training state of the target language model based on the fourth response corpus output by the target language model, wherein the training state includes a training output state and an iterative optimization state;

[0011] If the training state of the target language model is an iterative optimization state, the training data set is updated based on a corpus optimization rule, and the corpus optimization rule includes deleting sample data in the training data set whose learning state is marked as a learning success state.

[0012] In a second aspect, some embodiments of the present application further provide an iterative optimization method of a language model, which is applied to the display device described in the first aspect, wherein the display device includes a display and a controller, and the method includes:

[0013] Acquire a question-and-answer dataset, the question-and-answer dataset comprising a first preset number of sample data, the sample data comprising a question corpus, a first reply corpus, a second reply corpus and a learning status mark, the first reply corpus is provided with a first score generated based on a quality assessment model, the second reply corpus is provided with a second score generated based on the quality assessment model; the first score is higher than the second score; the learning status mark comprises a learning success state, a learning failure state and a non-learning state; the quality assessment model is a deep learning model that performs standardized assessment on a text corpus;

[0014] Generate a training data set and a test data set based on the question-answer data set, wherein the training data set includes a second preset amount of the sample data, and the test data set includes a third preset amount of the sample data;

[0015] Using the training data set to train a target language model, and obtaining a third response corpus output by the target language model, wherein the third response corpus is a response language generated based on the question corpus in the training data set;

[0016] Inputting the third reply corpus into the quality assessment model to obtain a third score, and updating the learning state mark of the sample data in the training data set based on the assessment difference, wherein the assessment difference is the difference between the first score and the third score;

[0017] Inputting the question corpus of the test data set into the target language model, and marking the training state of the target language model based on the fourth response corpus output by the target language model, wherein the training state includes a training output state and an iterative optimization state;

[0018] If the training state of the target language model is an iterative optimization state, the training data set is updated based on a corpus optimization rule, and the corpus optimization rule includes deleting sample data in the training data set whose learning state is marked as a learning success state.

[0019] As can be seen from the above technical solutions, some embodiments of the present application provide an iterative optimization method for a display device and a language model, the method can obtain a question-answering data set, generate a training data set and a test data set, and use the training data set to perform training on a target language model; and use a quality assessment model to evaluate the third reply corpus to update the learning status mark of the sample data in the training data set; and input the question corpus of the test data set into the target language model, and mark the training status of the target language model based on the output fourth reply corpus; if the training status is an iterative optimization state, then delete the sample data in the learning success state, and retain the sample data in the learning failure state to update the training data set. The method updates the training data set by deleting the learned sample data, and performs iterative training on the target language model by contrast learning to improve the accuracy of the reply output by the target language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0022] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0023] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0024] Figure 4 A flowchart of a language model iterative optimization method provided in some embodiments of the present application;

[0025] Figure 5 A schematic diagram of constructing sample data provided for some embodiments of the present application;

[0026] Figure 6 A schematic diagram of screening sample data provided for some embodiments of the present application;

[0027] Figure 7 A schematic diagram of sample data in a training data set provided in some embodiments of the present application;

[0028] Figure 8 A schematic diagram of the training process of the target language model provided in some embodiments of the present application;

[0029] Fig. 9 A schematic diagram of modifying the second reply corpus in sample data provided in some embodiments of the present application;

[0030] Fig.10 A schematic diagram of marking a fourth reply corpus test state provided in some embodiments of the present application;

[0031] Fig.11 A schematic diagram of marking the training status of a target language model provided in some embodiments of the present application;

[0032] Fig.12 A schematic diagram of an expanded training data set provided for some embodiments of the present application. DETAILED DESCRIPTION

[0033] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0034] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0035] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0036] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0037] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0038] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0039] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0040] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0041] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0042] like Figure 1As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0043] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0044] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0045] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0046] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures, and the image collector can also collect the user interface displayed by the display 260, or the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds, and the sound collector can also collect recorded audio.

[0047] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0048] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0049] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0050] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0051] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0052] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0053] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0054] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0055] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0056] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0057] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0058] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0059] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0060] like Figure 3As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0061] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0062] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0063] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0064] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0065] In some embodiments, the display device 200 uses a language model to recognize the user's voice commands and generates a reply to interact with the user. Among them, the language model (LM) is a deep learning model trained based on text data, which can generate highly readable and credible natural language text, and can deeply understand the meaning of the text, and realize intelligent processing and analysis of the text, including the theme, emotion and intention of the text. By combining the display device 200 with the language model, it is possible to interact with the user in a natural and fluent dialogue, understand the user's intentions and needs, and give corresponding replies and suggestions.

[0066] In order to improve the generalization ability of the language model and the accuracy of the language model in generating responses, it is necessary to select diverse and high-quality training data to train the language model so that the language model can learn sufficiently rich language knowledge in limited training data. The ability of the language model to handle target tasks can be increased by using data augmentation technology to obtain training data sets and train the language model using supervised fine-tuning or semi-supervised training methods. Among them, data augmentation technology transforms and expands the original data to generate new data samples, thereby increasing the diversity of the training data set, allowing the language model to learn more diverse data during the training process, thereby reducing the risk of overfitting the language model and improving the robustness of the language model. However, data augmentation technology cannot fully simulate the changes in actual data, resulting in deviations in the enhanced data, and excessive use of data augmentation technology will lead to a large difference between the enhanced data and the original data, affecting the generalization ability and performance of the language model.

[0067] Supervised fine-tuning is a method of inputting labeled task-specific data into a pre-trained language model to adjust the model parameters of the language model, thereby improving the processing performance of the language model on specific tasks. However, supervised fine-tuning is highly dependent on the quality and quantity of labeled data. When the labeled data is noisy or the amount of data is insufficient, the generalization ability of the language model will decrease.

[0068] Semi-supervised training methods use both labeled and unlabeled data to train language models. They use a large amount of unlabeled data to improve the performance of language models on limited labeled data, thereby improving the generalization and accuracy of language models. However, unlabeled data may contain noise and errors, which may mislead the learning process of the language model and reduce the performance of the language model. In addition, since semi-supervised training methods require higher computational complexity, the training time is longer and more computing resources are used, which increases the complexity of the language model training process.

[0069] Therefore, the above method has the problems of high cost of constructing high-quality training data and poor training effect of the language model, which leads to low accuracy of the reply output by the language model, affecting the user experience. In order to improve the training effect of the language model and enable the language model to output reply words with higher accuracy, some embodiments of the present application provide a display device 200, which trains the language model by using a comparative learning method by executing an iterative optimization method based on the language model, thereby improving the accuracy of the reply output by the language model. Among them, the display device 200 includes a display 260 and a controller 250. Among them, the display 260 is configured to display a user interface.

[0070] like Figure 4 As shown, the controller 250 is configured to perform an iterative optimization method of the language model, including the following:

[0071] S100: Obtain a question-answering dataset.

[0072] In some embodiments, the question-answering data set includes a first preset number of sample data, and the sample data includes a question corpus, a first reply corpus, a second reply corpus, and a learning status mark. The first reply corpus is provided with a first score generated based on a quality assessment model, the second reply corpus is provided with a second score generated based on the quality assessment model, the first score is higher than the second score, the learning status mark includes a learning success state, a learning failure state, and a non-learning state, and the quality assessment model is a deep learning model that performs standardized evaluation on the text corpus.

[0073] In some embodiments, the display device 200 trains the target language model using a contrastive learning method. By inputting a sample pair including a positive sample and a negative sample into the language model, the language model compares the similarities and differences between different samples, thereby improving the accuracy of the response language generated by the language model. Among them, the positive sample is a high-accuracy response, and the negative sample is a low-accuracy response. The language model learns a discriminative feature representation during the training process, that is, the output of the language model should be close to the representation of the positive sample, and should be far away from the representation of the negative sample.

[0074] In some embodiments, the first reply corpus is used as a positive sample, and the second reply corpus is used as a negative sample. The display device 200 generates sample data using a pre-trained large language model, such as Figure 5 As shown, the display device 200 obtains a question corpus (S501), then inputs the question corpus into the first language model to obtain a first reply corpus, and inputs the question corpus into the second language model to obtain a second reply corpus (S502); and adds a learning state mark of an unlearned state to the question corpus (S503); and establishes an association relationship between the question corpus, the first reply corpus, the second reply corpus and the learning state mark to construct sample data (S504). Among them, the first language model is a large language model trained based on the first training set, and the second language model is a large language model trained based on the second training set, and the data volume of the first training set is greater than the data volume of the second training set.

[0075] The first language model and the second language model are both deep learning models that generate response language based on the question corpus, but the accuracy of the response corpus output by the first language model is higher than the accuracy of the response corpus output by the second language model. The accuracy of the response corpus is judged by evaluation indicators, including accuracy, relevance, completeness, language fluency, and natural expression. The question corpus is input into the first language model, and the first response corpus generated by the first language model is obtained. The question corpus is input into the second language model, and the second response corpus generated by the second language model is obtained, so that the accuracy of the first response corpus is higher than the accuracy of the second response corpus.

[0076] For example, "Can you help me write a negative review of "xxx"?" is input into the first language model and the second language model respectively. The first response corpus has a clear structure, in-depth analysis and fluent language. The first response corpus objectively evaluates multiple aspects of the movie from a unique perspective, while the second response corpus has a chaotic structure and simple content. Therefore, the first response corpus is a high-accuracy response corpus, and the second response corpus is a low-accuracy response corpus.

[0077] The display device 200 uses the first reply corpus as a positive sample and the second reply corpus as a negative sample to construct a sample pair. By inputting the sample pair into the target language model, the language model can learn the difference between the two to maximize the similarity between the output corpus and the first reply corpus, while minimizing the similarity between the output corpus and the second reply corpus.

[0078] The display device 200 trains a first language model using a first training set, and trains a second language model using a second training set, wherein the data volume of the first training set is greater than the data volume of the second training set, for example, the data volume of the first training set is 2 billion data items, and the data volume of the second training set is 2 million data items.

[0079] In some embodiments, the display device 200 adds a learning state mark of the unlearned state to the question corpus to mark the question corpus as a corpus that the target language model has not learned, and establishes an association relationship between the question corpus, the first reply corpus, the second reply corpus, and the learning state mark to construct sample data. Among them, the learning state mark includes a learning success state, a learning failure state, and an unlearned state, and the display device 200 updates the learning state mark according to the training result of the target language model.

[0080] Exemplarily, after the construction of sample data A is completed, the learning status is marked as an unlearned state. When sample data A is input into the target language model as training data in the training data set, if the response corpus output by the target language model is a high-accuracy response, the learning status mark of sample data A is updated to a successful learning state to indicate that sample data A has been successfully learned by the target language model; if the response corpus output by the target language model is a low-accuracy response, the learning status mark of sample data A is updated to a failed learning state to indicate that sample data A has not been successfully learned by the target language model, and sample data A needs to be input into the target language model again to perform iterative training on the target language model.

[0081] In some embodiments, the display device 200 is required to perform screening on the sample data to ensure that the difference between the first reply corpus and the second reply corpus is large enough, thereby improving the data validity of the training data set and enabling the target language model to learn a discriminative feature representation. Figure 6 As shown, the display device 200 inputs the first reply corpus into the quality assessment model to obtain a first score, and inputs the second reply corpus into the quality assessment model to obtain a second score (S601); then calculates the difference between the first score and the second score to calculate the corpus difference (S602); and determines the value of the corpus difference and the third preset difference threshold (S603). If the corpus difference is less than the third preset difference threshold, the sample data corresponding to the first reply corpus and the second reply corpus are deleted. If the corpus difference is greater than or equal to the third preset difference threshold, the first score is set for the first reply corpus, and the second score is set for the second reply corpus.

[0082] The quality assessment model is a deep learning model that performs standardized assessment on text corpora. The quality assessment model evaluates the response corpus based on evaluation indicators. For example, the evaluation indicators can be the fluency, relevance, consistency, and information value of the response corpus. Fluency is used to evaluate whether the response corpus is fluent and conforms to grammatical rules. Relevance is used to evaluate whether the response corpus can accurately understand the user's intention and whether the response content is related to the input content. Consistency is used to evaluate whether the response corpus is applicable to different scenarios. Information value is used to evaluate whether the response corpus has practical value. The quality assessment model calculates the similarity between the response corpus and the benchmark response based on the text matching algorithm, and scores the response corpus according to the evaluation indicators.

[0083] The display device 200 performs screening on the sample data by calculating the corpus difference between the first reply corpus and the second reply corpus, thereby improving the data validity of the training data set. When the corpus difference is greater than or equal to the third preset difference threshold, the accuracy difference between the first reply corpus and the second reply corpus is large, and the sample data composed of the first reply corpus and the second reply corpus contains relatively rich information. The target language model can learn a discriminative feature representation by comparing the differences between the first reply corpus and the second reply corpus in the sample data. The display device 200 sets a first score for the first reply corpus and a second score for the second reply corpus, and constructs sample data based on this. When the corpus difference is less than the third preset difference threshold, the accuracy difference between the first reply corpus and the second reply corpus is small, and the group of sample pairs cannot provide effective information for the target language model, then the display device 200 deletes the sample data corresponding to the first reply corpus and the second reply corpus.

[0084] Exemplarily, the third preset difference threshold is 40, the quality assessment model generates a first score based on the first reply corpus, the first score is 89 points, and generates a second score based on the second reply corpus, the second score is 33 points, and the corpus difference is 56 points. In this way, 89 points are set as the first score in the first reply corpus, and 33 points are set as the second score in the second reply corpus.

[0085] Exemplarily, the third preset difference threshold is 40, the quality assessment model generates a first score based on the first reply corpus, the first score is 73 points, and generates a second score based on the second reply corpus, the second score is 60 points, and the corpus difference is 13 points, then the sample data corresponding to the first reply corpus and the second reply corpus are deleted.

[0086] S200: Generate a training dataset and a test dataset based on the question-answering dataset.

[0087] The display device 200 obtains a second preset amount of sample data from the question-answer data set to generate a training data set, and obtains a third preset amount of sample data from the question-answer data set to generate a test data set. Figure 7 As shown, the sample data in the training data set includes the question corpus, the first reply corpus, the second reply corpus, and the learning status mark. The training data set is used to iteratively train the target language model to improve the accuracy of the reply output by the target language model. The test data set has the same data structure as the training data set. The sample data in the test data set includes the question corpus, the first reply corpus, the second reply corpus, and the learning status mark. The test data set is used to test whether the target language model can end the iterative training.

[0088] S300: Utilize the training data set to train the target language model, and obtain a third response corpus output by the target language model.

[0089] In some embodiments, the display device 200 inputs the training data set into the target language model to train the target language model and increase the similarity between the reply corpus output by the target language model and the first reply utterance. Figure 8 As shown, the display device 200 uses a cosine similarity function to calculate a first similarity between the third reply corpus and the first reply corpus, and calculates a second similarity between the third reply corpus and the second reply corpus; and then adjusts the model parameters of the target language model based on the contrastive learning loss function, wherein the adjusted model parameters are used to increase the first similarity and reduce the second similarity.

[0090] In the process of calculating the first similarity, the display device 200 first performs preprocessing on the first reply corpus and the third reply corpus, including steps such as word segmentation, stop word removal and stem extraction, and converts the preprocessed text into a vector. Then, the first similarity between the third reply corpus and the first reply corpus is calculated by the cosine similarity function. In the process of calculating the second similarity, the display device 200 performs preprocessing on the second reply corpus, converts the preprocessed text into a vector, and then calculates the second similarity between the third reply corpus and the second reply corpus by the cosine similarity function.

[0091] The display device 200 adjusts the model parameters of the target language model based on the contrastive learning loss function, and trains the target language model to generate a third response corpus that is closer to the first response corpus and farther away from the second response corpus by minimizing the contrastive learning loss.

[0092] Among them, the contrastive learning loss function is expressed by the following formula:

[0093]

[0094] Among them, L represents the contrastive learning loss, z+ is the representation of the positive sample (i.e., the vector of the first reply corpus), z - is the representation of negative samples (i.e., the vector of the second reply corpus), z represents the representation of other samples (i.e., the vector of the third reply corpus), sim(z + ,z) represents the first similarity, sim(z - ,z) represents the second similarity.

[0095] The display device 200 trains the target language model based on the contrastive learning loss function to learn the difference features between the first response corpus and the second response corpus, and generates model parameters that increase the first similarity and reduce the second similarity. By adjusting the model parameters of the target language model, iterative training is performed on the target language model to improve the ability of the target language model to generate high-accuracy response corpora.

[0096] S400: Input the third reply corpus into the quality assessment model to obtain a third score, and update the learning state mark of the sample data in the training data set based on the assessment difference.

[0097] In some embodiments, the display device 200 obtains the third reply corpus output by the target language model, and inputs the third reply corpus into the quality assessment model. The quality assessment model scores the third reply corpus according to the evaluation index and outputs a third score. The display device 200 calculates the difference between the first score and the third score to calculate the evaluation difference. The evaluation difference is used to characterize the score difference between the third reply corpus and the first reply corpus. The display device 200 updates the learning state mark of the sample data to a learning success state or a learning failure state according to the value of the evaluation difference and the first preset difference threshold. Among them, the learning success state is used to characterize that the evaluation difference is less than or equal to the first preset difference threshold, and the learning failure state is used to characterize that the evaluation difference is greater than the first preset difference threshold.

[0098] Exemplarily, the first preset difference threshold is 10, the first score is 87 points, the third score is 82 points, and the evaluation difference is 5 points. Then, the learning status mark of the sample data corresponding to the first response corpus in the training data set is updated to a successful learning status to indicate that the sample data has been successfully learned by the target language model.

[0099] Exemplarily, the first preset difference threshold is 10, the first score is 92 points, the third score is 70 points, and the evaluation difference is 22 points. The learning status mark of the sample data corresponding to the first response corpus in the training data set is updated to a learning failure state to indicate that the sample data has not been successfully learned by the target language model. In the process of iterative training of the target language model, the training data set containing the sample data needs to be input into the target language model again.

[0100] In some embodiments, in order to increase the effectiveness of the training data set, the display device 200 updates the second reply corpus of the sample data according to the learning state mark of the sample data. Fig. 9 As shown, after the display device 200 executes updating the learning status mark of the sample data in the training data set based on the evaluation difference, it determines whether to update the second reply corpus, wherein, if the learning status mark of the sample data in the training data set is a learning failure state, the content of the third reply corpus is used as the second reply corpus of the sample data to update the training data set; if the learning status mark of the sample data in the training data set is a learning success state, the second reply corpus of the sample data is not modified.

[0101] Exemplarily, the first preset difference threshold is 10, the first score is 95 points, the third score is 75 points, the evaluation difference is 20 points, and the learning state mark of the sample data corresponding to the first response corpus in the training data set is updated to a learning failure state, indicating that the third response corpus generated by the target language model based on the question corpus in the sample data has a low accuracy rate. The display device 200 uses the content of the third response corpus as the second response corpus of the sample data to update the training data set. When the display device 200 iteratively trains the target language model using the updated training data set, the target language model generates a response corpus with a low similarity to the second response corpus based on the contrastive learning loss function, thereby improving the data validity of the training data set and improving the training effect of the target language model.

[0102] For example, the first preset difference threshold is 10, the first score is 90 points, the third score is 88 points, the evaluation difference is 2 points, and the learning state mark of the sample data corresponding to the first response corpus in the training data set is updated to the learning success state, indicating that the third response corpus generated by the target language model based on the question corpus in the sample data has a high accuracy rate. The second response corpus of the sample data is not modified.

[0103] S500: Input the question corpus of the test data set into the target language model, and mark the training state of the target model based on the fourth response corpus output by the target language model.

[0104] In some embodiments, the display device 200 uses the test data set to perform detection on the target language model to mark the training state of the target language model. The training state includes a training output state and an iterative optimization state, wherein the training output state is used to indicate that the target language model does not need to continue iterative optimization, and the iterative optimization state is used to indicate that the target language model needs to be iteratively trained again. Fig.10As shown, the display device 200 inputs the question corpus of the test data set into the target language model to obtain the fourth reply corpus, and inputs the fourth reply corpus into the quality assessment model to obtain the fourth score; then calculates the difference between the standard score and the fourth score to calculate the test difference; and marks the test status of the fourth reply corpus according to the test difference. Among them, the standard score is the first score of the first reply corpus in the test data set, and the test status includes a passed test status and a failed test status.

[0105] The display device 200 marks the test status of the fourth reply corpus according to the test difference between the fourth score and the first score. If the test difference is less than or equal to the second preset difference threshold, the fourth reply corpus is marked as having passed the test; if the test difference is greater than the second preset difference threshold, the fourth reply corpus is marked as having failed the test.

[0106] Exemplarily, the second preset difference threshold is 10, the first score is 93 points, the fourth score is 90 points, the test difference is 3 points, and the display device 200 marks the fourth reply corpus as passing the test to indicate that the fourth reply corpus output by the target language model based on the question corpus is highly similar to the first reply corpus, and the fourth reply corpus is a high-accuracy reply corpus.

[0107] Exemplarily, the second preset difference threshold is 10, the first score is 93 points, the fourth score is 73 points, the test difference is 20 points, and the display device 200 marks the fourth reply corpus as failing the test to indicate that the fourth reply corpus output by the target language model based on the question corpus has a low similarity to the first reply corpus, and the fourth reply corpus is a low accuracy reply corpus.

[0108] In some embodiments, the display device 200 marks the training status of the target language model according to the number of fourth reply corpora that pass the test status. Fig.11 As shown, the display device 200 obtains the number of fourth reply corpora marked as passed the test status; if the number of fourth reply corpora is greater than or equal to the preset qualified number threshold, the training status of the target language model is marked as the training output status; if the number of fourth reply corpora is less than the preset qualified number threshold, the training status of the target language model is marked as the iterative optimization status.

[0109] Exemplarily, the preset qualified quantity threshold is 200, and the display device 200 obtains 260 fourth response corpora marked as passed the test status, then the training status of the target language model is marked as the training output status, to indicate that the number of high-accuracy response corpora generated by the target language model reaches the preset standard, and no further iterative training is required.

[0110] Exemplarily, the preset qualified quantity threshold is 200, and the display device 200 obtains 120 fourth response corpora marked as passed the test status, then the training status of the target language model is marked as an iterative optimization status, to indicate that the number of high-accuracy response corpora generated by the target language model has not reached the preset annotation, and it is necessary to continue to perform iterative training on the target language model using the training data set.

[0111] In some embodiments, the training state of the target language model is an iterative optimization state, and the display device 200 updates the training data set based on the corpus optimization rule, and performs iterative training on the target language model using the updated training data set. The corpus optimization rule includes deleting sample data in the training data set whose learning state is successful.

[0112] Exemplarily, the learning status of sample data A, sample data B and sample data C in the training data set is marked as a successful learning status, and the learning status of sample data D, sample data E and sample data F is marked as a failed learning status. The display device 200 deletes sample data A, sample data B and sample data C in the training data set, and retains sample data D, sample data E and sample data F to update the training data set.

[0113] By deleting the sample data whose learning status is marked as a successful learning status in the training data set, the data utilization rate of the training data set can be improved. In the process of iterative training of the target language model by the display device 200, the target language model can learn the sample data in a step-by-step manner according to the difficulty of the sample data, that is, the sample data with low learning difficulty is learned first, and the sample data with high learning difficulty is learned iteratively for multiple times, so that the target language model can gradually improve the accuracy of the output reply corpus.

[0114] In some embodiments, the display device 200 adds new sample data to the training data set to expand the training data set. Fig.12 As shown, the display device 200 obtains the first sample quantity of sample data in the training data set (S1201); then calculates the difference between the second preset quantity and the first sample quantity to calculate the number of newly added samples (S1202); and obtains newly added sample data of the newly added sample quantity from the question and answer data set (S1203); and adds the newly added sample data to the training data set to update the training data set (S1204). Among them, the first sample quantity is used to represent the number of sample data in the training data set after deleting the sample data that has been successfully learned, and the newly added sample data is the sample data whose learning state is marked as the unlearned state.

[0115] Exemplarily, the second preset number is 120,000, and the number of sample data in the training data set after deleting the successfully learned sample data (i.e., the first sample number) is 100,000, and the number of newly added samples is calculated to be 20,000. The display device 200 obtains 20,000 sample data with the learning status marked as unlearned from the question and answer data set, and adds these sample data as newly added sample data to the training data set to update the training data set. The number of samples in the updated training data set is 120,000.

[0116] The display device 200 expands the training data set by supplementing new sample data from the candidate data, so that the training data set contains sample data in different fields and scenarios, as well as more language features. Using the expanded training data set to train the target language model can improve the generalization ability and robustness of the target language model, reduce the risk of overfitting of the target language model, and thus enable the target language model to output a response corpus with higher accuracy.

[0117] In some embodiments, the training state of the target language model is a training output state, and the display device 200 ends the training of the target language model.

[0118] As the display device 200 increases the number of times the target language model is trained, the performance improvement of the target language model will gradually tend to saturation, that is, the target language model has reached a relatively stable training output state. At this time, if the target language model continues to be iteratively trained, the training cost and difficulty will increase, while the performance improvement of the target language model will decrease. Therefore, when the training state of the target language model is the training output state, the display device 200 ends the training of the target language model.

[0119] Based on the above display device 200, some embodiments of the present application further provide an iterative optimization method of a language model, comprising the following steps:

[0120] Acquire a question-and-answer dataset, the question-and-answer dataset comprising a first preset number of sample data, the sample data comprising a question corpus, a first reply corpus, a second reply corpus and a learning status mark, the first reply corpus is provided with a first score generated based on a quality assessment model, the second reply corpus is provided with a second score generated based on the quality assessment model; the first score is higher than the second score; the learning status mark comprises a learning success state, a learning failure state and a non-learning state; the quality assessment model is a deep learning model that performs standardized assessment on a text corpus;

[0121] Generate a training data set and a test data set based on the question-answer data set, wherein the training data set includes a second preset amount of the sample data, and the test data set includes a third preset amount of the sample data;

[0122] Using the training data set to train a target language model, and obtaining a third response corpus output by the target language model, wherein the third response corpus is a response language generated based on the question corpus in the training data set;

[0123] Inputting the third reply corpus into the quality assessment model to obtain a third score, and updating the learning state mark of the sample data in the training data set based on the assessment difference, wherein the assessment difference is the difference between the first score and the third score;

[0124] Inputting the question corpus of the test data set into the target language model, and marking the training state of the target language model based on the fourth response corpus output by the target language model, wherein the training state includes a training output state and an iterative optimization state;

[0125] If the training state of the target language model is an iterative optimization state, the training data set is updated based on a corpus optimization rule, and the corpus optimization rule includes deleting sample data in the training data set whose learning state is marked as a learning success state.

[0126] As can be seen from the above technical solutions, some embodiments of the present application provide an iterative optimization method for a display device and a language model, the method can obtain a question-answering data set, generate a training data set and a test data set, and use the training data set to perform training on a target language model; and use a quality assessment model to evaluate the third reply corpus to update the learning status mark of the sample data in the training data set; and input the question corpus of the test data set into the target language model, and mark the training status of the target language model based on the output fourth reply corpus; if the training status is an iterative optimization state, then delete the sample data in the learning success state, and retain the sample data in the learning failure state to update the training data set. The method updates the training data set by deleting the learned sample data, and performs iterative training on the target language model by contrast learning to improve the accuracy of the reply output by the target language model.

[0127] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0128] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0130] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; The controller is configured as: Acquire a question-and-answer dataset, the question-and-answer dataset comprising a first preset number of sample data, the sample data comprising a question corpus, a first reply corpus, a second reply corpus, and a learning state mark, the first reply corpus being provided with a first score generated based on a quality assessment model, and the second reply corpus being provided with a second score generated based on the quality assessment model; The first score is higher than the second score; The learning status mark includes a learning success state, a learning failure state and a non-learning state; The quality assessment model is a deep learning model that performs standardized assessment on text corpus; Generate a training data set and a test data set based on the question-answer data set, wherein the training data set includes a second preset amount of the sample data, and the test data set includes a third preset amount of the sample data; Using the training data set to train a target language model, and obtaining a third response corpus output by the target language model, wherein the third response corpus is a response language generated based on the question corpus in the training data set; Inputting the third reply corpus into the quality assessment model to obtain a third score, and updating the learning state mark of the sample data in the training data set based on the assessment difference, wherein the assessment difference is the difference between the first score and the third score; Inputting the question corpus of the test data set into the target language model, and marking the training state of the target language model based on the fourth response corpus output by the target language model, wherein the training state includes a training output state and an iterative optimization state; If the training state of the target language model is an iterative optimization state, the training data set is updated based on a corpus optimization rule, and the corpus optimization rule includes deleting sample data in the training data set whose learning state is marked as a learning success state.

2. The display device according to claim 1, characterized in that The controller is also configured to: If the training state of the target language model is a training output state, the training of the target language model is terminated.

3. The display device according to claim 1, characterized in that The controller executes updating of the training data set based on the corpus optimization rule, and is further configured to: Acquire a first sample quantity of sample data in the training data set, where the first sample quantity is used to represent the sample quantity of the training data set after deleting sample data in a learning success state; Calculating the difference between the second preset number and the first sample number to calculate the number of newly added samples; Acquire the newly added sample data of the newly added sample quantity from the question-answer data set, the newly added sample data being the sample data whose learning state is marked as the unlearned state; The newly added sample data is added to the training data set to update the training data set.

4. The display device according to claim 1, characterized in that After the controller updates the learning state mark of the sample data in the training data set based on the evaluation difference, the controller is further configured to: If the learning state of the sample data in the training data set is marked as a learning failure state, the content of the third response corpus is used as the second response corpus of the sample data to update the training data set, and the learning failure state is used to characterize that the evaluation difference is greater than a first preset difference threshold; If the learning status of the sample data in the training data set is marked as a learning success status, the second response corpus of the sample data is not modified, and the learning success status is used to indicate that the evaluation difference is less than or equal to a first preset difference threshold.

5. The display device according to claim 1, characterized in that The controller performs marking of the training state of the target language model based on the fourth response corpus output by the target language model, and is further configured to: Inputting the fourth response corpus into the quality assessment model to obtain a fourth score; Calculating a difference between a standard score and the fourth score to calculate a test difference, wherein the standard score is a first score of the first response corpus in the test data set; Marking a test status of the fourth response corpus according to the test difference, wherein the test status includes a passed test status and a failed test status; Obtaining the number of fourth reply corpora marked as the passing test status; If the number of the fourth reply corpus is greater than or equal to a preset qualified number threshold, marking the training state of the target language model as a training output state; If the number of the fourth reply corpus is less than a preset qualified number threshold, the training state of the target language model is marked as an iterative optimization state.

6. The display device according to claim 5, characterized in that The controller executes the test state of marking the fourth reply corpus according to the test difference, and is further configured to: If the test difference is less than or equal to a second preset difference threshold, marking the fourth reply corpus as passing the test; If the test difference is greater than a second preset difference threshold, the fourth reply corpus is marked as failing the test.

7. The display device according to claim 1, characterized in that The controller is also configured to: Calculating a first similarity between the third reply corpus and the first reply corpus using a cosine similarity function, and calculating a second similarity between the third reply corpus and the second reply corpus; The model parameters of the target language model are adjusted based on the contrastive learning loss function, and the adjusted model parameters are used to increase the first similarity and reduce the second similarity.

8. The display device according to claim 1, characterized in that Before the controller executes the acquisition of the question-answering dataset, it is also configured as follows: Obtaining the question corpus; Input the question corpus into a first language model to obtain the first reply corpus; and input the question corpus into a second language model to obtain the second reply corpus; wherein the first language model is a large language model trained based on a first training set, and the second language model is a large language model trained based on a second training set, and the data volume of the first training set is greater than the data volume of the second training set; Adding a learning state mark of an unlearned state to the question corpus; An association relationship is established among the question corpus, the first reply corpus, the second reply corpus and the learning state mark to construct the sample data.

9. The display device according to claim 8, characterized in that The controller is also configured to: Inputting the first reply corpus into the quality assessment model to obtain a first score, and inputting the second reply corpus into the quality assessment model to obtain a second score; Calculating a difference between the first score and the second score to calculate a corpus difference; If the corpus difference is greater than or equal to a third preset difference threshold, setting the first score for the first reply corpus and setting the second score for the second reply corpus; If the corpus difference is less than a third preset difference threshold, the sample data corresponding to the first reply corpus and the second reply corpus are deleted.

10. A method for iterative optimization of a language model, characterized in that: Applied to the display device according to any one of claims 1 to 9, the method comprising: Acquire a question-and-answer dataset, the question-and-answer dataset comprising a first preset number of sample data, the sample data comprising a question corpus, a first reply corpus, a second reply corpus and a learning status mark, the first reply corpus is provided with a first score generated based on a quality assessment model, the second reply corpus is provided with a second score generated based on the quality assessment model; the first score is higher than the second score; the learning status mark comprises a learning success state, a learning failure state and a non-learning state; the quality assessment model is a deep learning model that performs standardized assessment on a text corpus; Generate a training data set and a test data set based on the question-answer data set, wherein the training data set includes a second preset amount of the sample data, and the test data set includes a third preset amount of the sample data; Using the training data set to train a target language model, and obtaining a third response corpus output by the target language model, wherein the third response corpus is a response language generated based on the question corpus in the training data set; Inputting the third reply corpus into the quality assessment model to obtain a third score, and updating the learning state mark of the sample data in the training data set based on the assessment difference, wherein the assessment difference is the difference between the first score and the third score; Inputting the question corpus of the test data set into the target language model, and marking the training state of the target language model based on the fourth response corpus output by the target language model, wherein the training state includes a training output state and an iterative optimization state; If the training state of the target language model is an iterative optimization state, the training data set is updated based on a corpus optimization rule, and the corpus optimization rule includes deleting sample data in the training data set whose learning state is marked as a learning success state.

Citation Information

Cited By

  • Training data synthesis method of voice assistant model, voice assistant system and computer equipment

    CN120783729A

  • Information processing system, information processing method, and program

    JP7949425B1