Chinese opera theater generation system based on Transform architecture

Abstract

The invention discloses a Chinese opera drama generation system based on a Transform architecture, and the system comprises a data preprocessing module which collects and arranges a corpus containing opera drama, and converts the corpus into a training format which can be accepted by GPT-2 (Generative Pre-train Transform 2); the model fine tuning module is used for performing fine tuning based on a HugingFace GPT-2 pre-training model after the data is prepared; the text generation module is used for analyzing the input story by using a Transform encoder and generating a text conforming to the Chinese opera style; the style control module is used for controlling artistry and diversity of the generated text by adjusting the generation temperature and the sampling strategy; the evaluation optimization module is used for optimizing the model performance and improving the generation quality through manual and automatic evaluation; and the application and extension module is used for expanding and applying the obtained data to a plurality of fields. The invention aims to apply the Transform architecture to the automatic generation of the Chinese opera drama, can automatically generate the drama conforming to the opera art law, provides an auxiliary tool for opera creation, and promotes the inheritance and development of the opera art.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a Chinese opera repertoire generation system based on the Transformer architecture. Background Art

[0002] As a treasure of the traditional Chinese culture, Chinese opera has a long history. It not only contains profound cultural heritage, but also has rich and diverse artistic expression forms and unique artistic charm, and is an important part of the traditional Chinese culture.

[0003] However, with the development of the times, traditional opera creation faces problems such as difficulties in inheritance and lack of innovation. The creation of traditional opera repertoires depends on professional opera creators. Creators need rich historical knowledge and artistic accomplishment, and at the same time possess profound literary accomplishment and a profound understanding of opera art. The creation process is complex and time-consuming, and is restricted by the personal style and experience of the creators, making it difficult to meet the needs of modern society for the innovation and diversification of opera culture.

[0004] In recent years, artificial intelligence technology has made breakthrough progress in the field of natural language processing. Using machine learning and deep learning methods for text generation is no longer a fantasy. However, the current generation technology for Chinese opera repertoires is still in the exploration stage. Most of the existing repertoire generation methods are based on simple rules or statistical models, and there are problems such as poor plot logic, language style not conforming to the characteristics of opera, and lack of in-depth understanding of opera knowledge and cultural connotations, making it difficult to generate opera repertoires with high artistry and coherence.

[0005] As an important model in the field of deep learning, the Transformer architecture has achieved remarkable results in natural language processing tasks with its powerful language generation and understanding capabilities. Especially the extensive application of deep learning technology in the field of natural language processing provides new possibilities for the automatic generation of opera repertoires.

[0006] The purpose of the present invention is to apply the Transformer architecture to the automatic generation of Chinese opera repertoires to achieve the rapid and efficient creation of opera repertoires. Summary of the Invention

[0007] In view of this, the present invention provides a Chinese opera repertoire generation system based on the Transformer architecture.

[0008] To achieve the above object, the present invention provides the following technical solutions, mainly including:

[0009] The data preprocessing module collects and organizes the corpus containing Chinese opera repertoires and converts it into a training format acceptable to GPT-2 (Generative Pre-trained Transformer 2).

[0010] The model fine-tuning module, after the data is ready, we fine-tune based on the pre-trained model of HuggingFace GPT-2.

[0011] The text generation module uses the Transformer encoder to parse the input story and generate text in the style of Chinese opera.

[0012] The style control module controls the artistry and diversity of the generated text by adjusting the generation temperature and sampling strategy.

[0013] The evaluation and optimization module optimizes the model performance and improves the generation quality through manual and automated evaluations.

[0014] The application and extension module extends the obtained data and applies it to multiple fields.

[0015] The Chinese opera repertoire generation system based on the Transformer architecture as described above is further preferably that the data preprocessing module includes:

[0016] Word segmentation and annotation, segment the Chinese opera repertoire and annotate the key elements to ensure the integrity of the story structure and semantics.

[0017] Data cleaning, clean the text, remove meaningless characters, duplicate content and low-quality text to ensure the high quality of the data.

[0018] Sample formatting, define each training sample as two parts: "story input" and "repertoire output" to ensure that the model can correctly learn the correspondence between the input and the output.

[0019] The Chinese opera repertoire generation system based on the Transformer architecture as described above is further preferably that the model fine-tuning module includes:

[0020] Load the pre-trained model, use the GPT-2 model provided by HuggingFace to load the pre-trained weights, which have saved a large amount of general language knowledge, and adapt it to the Chinese opera text generation task through fine-tuning.

[0021] Optimization techniques, use a custom loss function, focusing on the rhythm, sentence pattern and semantic consistency of the generated text. Especially the rhythm and artistry of Chinese opera text require a specially designed loss function to control.

[0022] The Chinese opera repertoire generation system based on the Transformer architecture as described above is further preferably that the text generation module includes:

[0023] Semantic understanding: The model parses the input story description through the Transformer encoder. The Transformer encoder can obtain context information from both left and right directions, helping the model accurately understand the semantics of the elements in the story;

[0024] Text generation: Based on the input story, GPT-2 will gradually generate text in the opera style.

[0025] The Chinese opera repertoire generation system based on the Transformer architecture as described above is further preferably that in the style control module: A lower temperature value will make the generated text more conservative and coherent, while a higher temperature value will increase the diversity and creativity of the text.

[0026] The Chinese opera repertoire generation system based on the Transformer architecture as described above is further preferably that the evaluation and optimization module includes:

[0027] Manual evaluation: Professionals evaluate the generated text and give scores from multiple dimensions such as artistry, coherence, and cultural compliance;

[0028] Automatic evaluation: Use automatic evaluation metrics such as BLEU (Bilingual Evaluation Understudy) and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) to measure the similarity between the generated text and the manually written reference text, ensuring the improvement of the language quality of the generated text;

[0029] Continuous optimization: According to the evaluation results, iteratively optimize the training data, model structure, and hyperparameters to improve the generation quality.

[0030] The Chinese opera repertoire generation system based on the Transformer architecture as described above is further preferably that the application and extension module includes:

[0031] Opera repertoire creation: As an auxiliary creation tool, it helps screenwriters quickly generate opera lines based on a simple story framework;

[0032] Opera teaching: It helps students understand and practice opera texts in different styles and provides automated creative exercises;

[0033] Cross-language extension: By training various opera types, corresponding texts are generated in different language environments to promote the digital creation of traditional arts.

[0034] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses and provides

[0035] Improve the creation efficiency: It can quickly generate a large number of opera plays with different styles and themes, providing rich creative inspiration and materials for opera creators, reducing the work burden of opera creators, lowering the creation cost, significantly shortening the creation cycle, and improving the creation efficiency.

[0036] Ensure the quality of the plays: Introduce the knowledge base in the opera field to fine-tune the Transformer model to ensure that the generated play text has high quality and meets the requirements of opera art in terms of plot, language, and cultural connotation, which helps to promote the inheritance and development of Chinese opera culture.

[0037] Enrich the opera culture: The system has good scalability and versatility, and can adapt to the generation requirements of opera plays in different regions and different opera types by continuously updating the corpus and knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative efforts.

[0039] Figure 1 It is a flowchart of a Chinese opera play generation system based on the Transformer architecture. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0041] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the ordinary meanings understood by those of ordinary skill in the art to which the present disclosure belongs. The terms "first", "second" and similar words used in the embodiments of the present disclosure do not denote any order, quantity or importance, but are only used to distinguish different components. Words such as "including" or "comprising" mean that the elements or items appearing before the word cover the elements or items listed after the word and their equivalents, without excluding other elements or items. For example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or devices. The terms "connected" or "coupled" and the like are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", etc. are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly. In addition, in the description of the present application, unless otherwise stated, the term "plurality" means two or more. The term "and / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0042] Traditional Chinese opera creation relies on the creativity and experience of artists. They manually arrange elements such as characters, locations, times, and events in the story, and finally form an opera text with rhythm and artistry. This creative process is both time-consuming and relies on the artistic talent of the creator. To improve the creation efficiency and maintain the artistry of Chinese opera, we propose a Chinese opera repertoire generation technology based on Hugging Face GPT-2. By leveraging pre-trained language models and combining specific opera corpus fine-tuning methods, it can automatically generate repertoire texts that conform to the style of Chinese opera. Hugging Face GPT-2, as a powerful pre-trained language model, has been trained on a large-scale corpus and has excellent text generation capabilities. It can generate coherent and contextually relevant texts, which is very suitable for tasks like creating opera repertoires that require high text fluency and semantic consistency. At the same time, it has high controllability. The style and diversity of the generated text can be adjusted by setting hyperparameters (such as temperature, Top-p sampling), so as to generate rhythms and sentence patterns that conform to the characteristics of Chinese opera, as well as its openness (which has been implemented in the Hugging Face's Transformer library). It is based on the Transformer architecture and uses a Transformer decoder. The multi-head self-attention mechanism and positional encoding in it greatly enhance its ability to generate long-sequence texts. At the same time, strategies such as temperature parameters and Beam Search can be used to adjust the generated text to make it more artistic and diverse.

[0043] Please refer to the appendix Figure 1 , which is a Chinese opera repertoire generation system based on the Transformer architecture disclosed in the present invention.

[0044] The present invention mainly includes:

[0045] A data preprocessing module that collects and organizes a corpus containing Chinese opera repertoires and converts it into a training format acceptable to GPT-2 (Generative Pre-trained Transformer 2);

[0046] A model fine-tuning module. After the data is prepared, we fine-tune based on the pre-trained model of HuggingFace GPT-2;

[0047] A text generation module that uses a Transformer encoder to parse the input story and generate texts that conform to the style of Chinese opera;

[0048] A style control module that controls the artistry and diversity of the generated text by adjusting the generation temperature and sampling strategy;

[0049] An evaluation and optimization module that optimizes the model performance and improves the generation quality through manual and automated evaluations;

[0050] Application and extension module, applying the obtained data to multiple fields.

[0051] To further optimize the above technical solution, the data preprocessing module includes:

[0052] Word segmentation and annotation, segmenting the traditional Chinese opera repertoire and annotating key elements to ensure the integrity of the story structure and semantics;

[0053] Data cleaning, cleaning the text, removing meaningless characters, duplicate content and low-quality text to ensure the high quality of the data;

[0054] Sample formatting, defining each training sample as two parts: "story input" and "opera script output", to ensure that the model can correctly learn the correspondence between the input and the output;

[0055] Specifically, the story input includes elements such as characters, locations, times, events, etc., and the opera script output is the corresponding traditional Chinese opera lines.

[0056] Example data format:

[0057] Input: Character: Li Cuilian, Location: Suzhou City, Time: Spring of a certain year in the Ming Dynasty, Event: Li Cuilian meets Zhang Sheng by chance and has a misunderstanding.

[0058] Output: In Suzhou City, the spring breeze is gentle. Cuilian steps out and meets Zhang Sheng. As soon as the words are spoken, the wind and clouds rise. The two have a misunderstanding and their emotions are intertwined.

[0059] To further optimize the above technical solution, the model fine-tuning module includes:

[0060] Loading a pre-trained model, using the GPT-2 model provided by HuggingFace to load the pre-trained weights, which preserves a large amount of general language knowledge, and adapting it to the traditional Chinese opera text generation task through fine-tuning;

[0061] Optimization techniques, using a custom loss function, focusing on the prosody, sentence pattern and semantic consistency of the generated text. Especially for the prosody and artistry of traditional Chinese opera texts, a specially designed loss function is needed to control them.

[0062] Specifically, for example, adding a prosody loss to the original cross-entropy loss function. First, use pypinpin to extract the syllables in the generated text, and according to the prosody requirements of Chinese poems, judge whether the generated text meets the basic prosody requirements. The rhyme rules can be checked using the finals of the pinyin. There is also a semantic consistency loss, which can use the pre-trained language model BERT to calculate the semantic similarity between the input text (story description) and the generated text. It can be achieved by calculating the cosine similarity between the two.

[0063] Optimize the text generation process through the language modeling objective (predict the next token). At each generation, the model predicts the next most likely word based on the current context and generates coherent text.

[0064] Adopt the Beam Search strategy. By retaining multiple candidate paths, it controls the coherence and diversity of the generated text. It is used in text generation tasks to find the optimal word sequence, avoiding the generation of overly single or incoherent sentences and missing the optimal solution at the same time.

[0065] To further optimize the above technical solution, the text generation module includes:

[0066] Semantic understanding. The model parses the input story description through a Transformer encoder. The Transformer encoder can obtain context information from both left and right directions, helping the model accurately understand the semantics of the elements in the story.

[0067] Text generation. Based on the input story, GPT-2 gradually generates text in the opera style.

[0068] Specifically, due to the special nature of opera, the generated text must be artistic, conforming to rhythm and cultural traditions.

[0069] To further optimize the above technical solution, in the style control module: A lower temperature value makes the generated text more conservative and coherent, while a higher temperature value increases the diversity and creativity of the text.

[0070] Specifically, use GPU to accelerate the training process and improve the training efficiency.

[0071] Learning Rate Schedule: Dynamically adjust the learning rate to prevent overfitting and ensure training stability.

[0072] During the training process, we conduct evaluations after each training epoch, adjust the hyperparameters of the model, and ensure that the model can better generate opera-style pieces.

[0073] To further optimize the above technical solution, the evaluation and optimization module includes:

[0074] Manual evaluation. Professionals evaluate the generated text and give scores from multiple dimensions such as artistry, coherence, and cultural compliance.

[0075] Automatic evaluation, using BLEU (Bilingual Evaluation Understudy) and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) as automated evaluation metrics, measures the similarity between the generated text and the manually written reference text to ensure the improvement of the generated text in terms of language quality;

[0076] Among them, BLEU is based on the precision of n-gram, that is, how many n-grams in the generated text appear in the reference text.

[0077] To avoid the inflated scores caused by generating overly short texts, BLEU introduces "brevity penalty" to penalize overly short generated texts.

[0078] The calculation formula is:

[0079]

[0080] BP: brevity penalty factor, and the calculation formula is:

[0081]

[0082] Among them, c is the length of the generated text, and r is the length of the reference text.

[0083] Pn: precision of n-gram, that is, the proportion of n-grams in the generated text that match the reference text.

[0084] Wn: weight of n-gram, usually taking uniform weights (such as Wn = 1 / N).

[0085] Among them, ROUGE evaluates the quality of the generated text by calculating the n-gram overlap degree between the generated text and the reference text.

[0086] Common ROUGE variants include:

[0087] ROUGE-N: based on n-gram overlap.

[0088] ROUGE-L: based on the overlap of the longest common subsequence (LCS).

[0089] ROUGE-W: based on the overlap of the weighted longest common subsequence.

[0090] ROUGE-S: based on the overlap of skip bigram.

[0091] Calculation formula:

[0092] Take ROUGE-N as an example:

[0093]

[0094] Count match (n-gram): The number of n-grams that match between the generated text and the reference text.

[0095] Count reference (n-gram): The number of n-grams in the reference text.

[0096] Continuously optimize, and according to the evaluation results, iteratively optimize the training data, model structure, and hyperparameters to improve the generation quality.

[0097] To further optimize the above technical solution, the application and extension module includes:

[0098] The creation of traditional Chinese opera repertoire, as an auxiliary creation tool, helps screenwriters quickly generate traditional Chinese opera lines based on a simple story framework;

[0099] Traditional Chinese opera teaching, which helps students understand and practice traditional Chinese opera texts in different styles and provides automated creation exercises;

[0100] Cross-language extension, by training various types of traditional Chinese opera, generates corresponding texts in different language environments to promote the digital creation of traditional art.

[0101] Specifically, the creation of traditional Chinese opera repertoire, traditional Chinese opera teaching, and cross-language extension have significant benefits in promoting the development of traditional Chinese opera art, improving students' abilities, and promoting cultural exchanges. These measures contribute to the inheritance and promotion of traditional culture, promote the innovation and development of traditional Chinese opera art, and also help cultivate students' creative abilities and cross-cultural communication abilities.

[0102] In this specification, each embodiment is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the description in the method part.

[0103] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A Chinese opera repertoire generation system based on the Transformer architecture, characterized by: include: The data preprocessing module collects and organizes the corpus containing opera repertoires and converts it into a training format acceptable to GPT-2 (Generative Pre-trained Transformer 2); Model fine-tuning module, after the data is prepared, we fine-tune the pre-trained model based on HuggingFace GPT-2; The text generation module uses the Transformer encoder to parse the input story and generate text that conforms to the style of opera; The style control module controls the artistry and diversity of generated text by adjusting the generation temperature and sampling strategy; Evaluation and optimization module, which optimizes model performance and improves generation quality through manual and automated evaluation; The application and extension module expands the obtained data and applies it to multiple fields.

2. The Chinese opera repertoire generation system based on the Transformer architecture according to claim 1 is characterized in that: The data preprocessing module comprises: Word segmentation and annotation: segment the opera repertoire and mark the key elements to ensure the structure and semantic integrity of the story; Data cleaning: clean the text, remove meaningless characters, duplicate content and low-quality text to ensure high data quality; Sample formatting, defining each training sample as two parts: "story input" and "track output", to ensure that the model can correctly learn the correspondence between input and output.

3. The Chinese opera repertoire generation system based on Transformer architecture according to claim 1 is characterized in that: The model fine-tuning module includes: Load the pre-trained model and use the GPT-2 model provided by HuggingFace to load the pre-trained weights, which saves a lot of general language knowledge and adapts it to the opera text generation task through fine-tuning; The optimization technique uses a custom loss function and focuses on the rhythm, sentence structure, and semantic consistency of the generated text. In particular, the rhythm and artistry of opera texts require a specially designed loss function to control them.

4. The Chinese opera repertoire generation system based on Transformer architecture according to claim 1 is characterized in that: The text generation module comprises: Semantic understanding: The model parses the input story description through the Transformer encoder. The Transformer encoder can obtain contextual information from both left and right directions to help the model accurately understand the semantics of elements in the story; Text generation, based on the input story, GPT-2 will gradually generate opera-style text.

5. The Chinese opera repertoire generation system based on Transformer architecture according to claim 1 is characterized in that: The style control module wherein: a lower temperature value makes the generated text more conservative and coherent, while a higher temperature value increases the diversity and creativity of the text.

6. The Chinese opera repertoire generation system based on Transformer architecture according to claim 1 is characterized in that: The evaluation and optimization module includes: Manual evaluation: Professionals evaluate the generated text and give scores based on multiple dimensions such as artistry, coherence, and cultural conformity; Automatic evaluation, using BLEU (Bilingual Evaluation Understudy) and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) automated evaluation indicators to measure the similarity between the generated text and the manually written reference text, ensuring the improvement of the language quality of the generated text; Continuous optimization: Based on the evaluation results, iteratively optimize the training data, model structure, and hyperparameters to improve the generation quality.

7. The Chinese opera repertoire generation system based on Transformer architecture according to claim 1 is characterized in that: The application and extension modules include: Opera repertoire creation, as an auxiliary creation tool, helps screenwriters quickly generate opera lines based on a simple story framework; Opera teaching, helping students understand and practice opera texts of different styles, and providing automated creative exercises; Cross-language expansion: by training various types of opera, generating corresponding texts in different language environments, and promoting the digital creation of traditional art.

Images