A question generation method based on natural language processing

Abstract

The present invention discloses a question generation method based on natural language processing, which belongs to the technical field of natural language processing. In order to solve the problem of low quality of question generation in the field of natural language processing in traditional technologies, the method determines the question data source corresponding to the question generation, and judges whether the question data source is a preset text type data source. If the above judgment is yes, the question data source is subjected to natural language processing to obtain natural language features, and then, when several target semantic diagram elements corresponding to the natural language features can be determined, a corresponding semantic diagram is generated according to all the target semantic diagram elements, and then a question is generated according to the semantic diagram to obtain the question corresponding to the question data source, and the question data source is converted from text mode to graphic mode to refine the abstract text expression form into an intuitive and concise graphic form. The generated question can focus on the main content expressed by the question data source, thereby improving the quality of question generation.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a question generation method based on natural language processing. Background Art

[0002] Question generation (QG) refers to the process of automatically creating relevant questions from a given data source and is a key task in the field of natural language processing (NLP). With the development of artificial intelligence (AI), question generation is becoming increasingly important in various intelligent scenarios, including but not limited to the need to automatically generate test questions in education, the need for intelligent tutoring systems to help students understand material, the need for question-answering systems to generate candidate questions, the need for search engines to generate relevant question suggestions, and the need for human-computer interaction robots to maintain the fluency of human-computer interaction conversations. The quality of generated questions directly impacts the effectiveness of these various intelligent scenarios. Traditional techniques typically use natural language processing and are based on rules, machine learning, or deep learning to generate questions. Rule-based question generation typically involves word segmentation through text preprocessing and matching the resulting tokens with question templates to generate the corresponding questions. Machine learning-based question generation typically generates questions based on tokens using statistical methods or traditional machine learning methods such as support vector machines (SVMs) and random forests. Deep learning-based question generation typically generates questions based on tokens using methods including but not limited to LSTM models and Transformer models. From the above, we can see that traditional technologies all generate corresponding questions directly based on word units. However, due to the abstract nature of word units, it is difficult to generate accurate, comprehensive, and high-quality questions, which in turn affects the intelligent effects of the above-mentioned different intelligent scenarios.

[0003] Therefore, how to improve the quality of question generation has become an urgent problem to be solved in the field of natural language processing. Summary of the Invention

[0004] In response to the problems existing in the prior art, the present invention provides a question generation method based on natural language processing, which converts the question data source from text mode to graphic mode, regenerates the question, and improves the quality of question generation.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0006] A question generation method based on natural language processing, comprising:

[0007] In response to the question generation instruction, determining the question data source corresponding to the question generation;

[0008] Determine whether the question data source is a preset text type data source. If so, perform natural language processing on the question data source to obtain natural language features; otherwise, determine the corresponding target preset question generation method and generate the question;

[0009] Determine whether the natural language feature meets a preset semantic image element condition; if so, determine a number of target semantic image elements corresponding to the natural language feature;

[0010] Generate a semantic graph corresponding to the problem data source according to all the target semantic graph elements and based on a preset semantic graph generation model;

[0011] According to the semantic graph and based on a preset graph question generation method, questions are generated to obtain questions corresponding to the question data source.

[0012] Furthermore, the problem data source is subjected to natural language processing to obtain natural language features, including:

[0013] Segmenting the problem data source to obtain a number of tokens;

[0014] Performing part-of-speech tagging on the word-unit to obtain a tagged word-unit;

[0015] Perform named entity recognition based on the annotated word to obtain a target entity;

[0016] Based on a preset semantic recognition model, semantic recognition is performed on the problem data source to obtain target semantics;

[0017] The marked word, the target entity, and the target semantics are used as natural language features corresponding to the question data source.

[0018] Furthermore, all the target semantic diagram elements include: a target scene base diagram element, a target character entity diagram element, a target dynamic element diagram element, a target atmosphere rendering diagram element, and a target perspective control diagram element;

[0019] Generating a semantic graph corresponding to the problem data source according to all the target semantic graph elements and based on a preset semantic graph generation model, including:

[0020] Generate a model based on a preset picture, and perform scene base modeling according to the target scene base diagram elements to obtain a target scene base visual expression;

[0021] Based on the target scene substrate and according to the target character entity diagram elements, target character entity modeling is performed to obtain a target character entity visual expression;

[0022] Based on the target scene base and according to the target dynamic element diagram elements, target dynamic element modeling is performed to obtain a target motion trajectory visual expression corresponding to the target dynamic element;

[0023] Based on the target scene base and according to the target atmosphere rendering diagram elements, target atmosphere rendering modeling is performed to obtain a target atmosphere visual expression;

[0024] Based on the target scene substrate and according to the target perspective control diagram elements, visual control modeling is performed to obtain a target lens angle visual expression;

[0025] Based on the semantics corresponding to the problem data source, the target scene base visual expression, the target character entity visual expression, the target motion trajectory visual expression, the target atmosphere visual expression and the target lens angle visual expression are visualized and logically expressed to obtain a semantic diagram corresponding to the problem data source.

[0026] Furthermore, before generating questions according to the semantic graph and based on a preset graph question generation method and obtaining the questions corresponding to the question data source, the method further includes:

[0027] Based on the preset CLIP graphic verification model, determine whether the semantic diagram is similar to the semantics of the problem data source. If similar, execute the step of generating questions according to the semantic diagram and based on the preset diagram question generation method to obtain the question corresponding to the problem data source.

[0028] Furthermore, according to the semantic graph and based on a preset graph question generation method, questions are generated to obtain questions corresponding to the question data source, including:

[0029] Performing image preprocessing on the semantic graph to obtain a preprocessed graph;

[0030] Extracting visual features corresponding to the preprocessed image based on a preset visual feature extraction method to obtain a plurality of visual feature texts;

[0031] According to the pre-processed image and the visual feature text, and based on a preset visual language model, different visual feature texts are associated to obtain a plurality of visual text contents;

[0032] According to the visual text content and based on a preset question generation model, questions corresponding to the visual text content are generated, and questions corresponding to the question data source are obtained.

[0033] Furthermore, according to the semantic graph and based on a preset graph question generation method, questions are generated to obtain questions corresponding to the question data source, including:

[0034] Generating questions according to the semantic graph and a preset graph question generation method to obtain a benchmark question;

[0035] Perform named entity recognition on the benchmark question to obtain a question entity;

[0036] Identify the entity type to which the problem entity belongs;

[0037] Determining a corresponding preset replacement entity set according to the entity type, the preset replacement entity set including preset replacement entities;

[0038] According to the preset replacement entity set and based on a preset replacement entity transformation method, the problem entity is transformed into the corresponding replacement entity to obtain a transformed problem;

[0039] The benchmark problem and the transformed problem are used as the problems corresponding to the problem data source.

[0040] Furthermore, after identifying the entity type to which the problem entity belongs, the method further includes:

[0041] Determine, based on the entity type, a preset correspondence between a preset entity corresponding to the problem entity and a preset knowledge graph;

[0042] According to the preset correspondence, the question entity is associated with a corresponding preset knowledge graph, wherein the preset knowledge graph includes a preset entity extended question chain, and the preset entity extended question chain includes a plurality of preset extended question templates;

[0043] Associating the question entity with the preset extended question template to obtain an extended question;

[0044] The extended question is used as the question corresponding to the question data source.

[0045] Furthermore, the method further comprises:

[0046] In the case that the question data source is not a preset text type data source, a corresponding target preset question generation method is determined to perform question generation.

[0047] Furthermore, when the question data source is not a preset text type data source, determining a corresponding target preset question generation method and performing question generation includes:

[0048] Identify the data source type corresponding to the problematic data source;

[0049] According to the data source type, a corresponding target preset question generation method is determined to generate questions.

[0050] Furthermore, according to the data source type, a corresponding target preset question generation method is determined, including at least one of the following:

[0051] In the case where the data source type is a structured data source type, determining a preset structured question generation method as a target preset question generation method;

[0052] In a case where the data source type is an image data source type, determining a preset image question generation method as a target preset question generation method;

[0053] In a case where the data source type is a dialogue data source type, the preset dialogue question generation method is determined as the target preset question generation method.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The question generation method based on natural language processing of the present invention determines the question data source corresponding to the question generation and judges whether the question data source is a preset text type data source. If the above judgment is yes, the question data source is subjected to natural language processing to obtain natural language features. Then, when several target semantic diagram elements corresponding to the natural language features can be determined, a semantic diagram corresponding to the question data source is generated according to all target semantic diagram elements. Then, based on the semantic diagram, the question is generated to obtain the question corresponding to the question data source. Therefore, when the question data source is a preset text type data source, the question data source is converted from text mode to graphic mode, so as to refine the abstract and complex text expression form into an image-based, intuitive, and concise concrete form. The graphic mode can focus on the main and important semantic content and core ideas expressed by the question data source, so that the generated questions can also focus on the main and important semantic content and core ideas expressed by the question data source, thereby improving the question generation quality under the text type data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 Schematic diagram of the flow of the question generation method based on natural language processing of the present invention;

[0057] Figure 2 A schematic diagram of the overall concept of the question generation method based on natural language processing of the present invention;

[0058] Figure 3 A schematic diagram of the process of obtaining natural language features in the present invention;

[0059] Figure 4 A schematic diagram of the process of generating a semantic graph in the present invention;

[0060] Figure 5This is a flow chart of generating questions based on the generated semantic graph in the present invention. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0062] An embodiment of the present invention provides a question generation method based on natural language processing. The method can be applied to corresponding devices such as smart phones, tablet computers, learning machines, desktop computers, smart terminals, servers, cloud platforms, etc., and can be used in scenarios including but not limited to automatically generating test questions in the education field, intelligent tutoring systems helping students understand materials, question-and-answer systems generating candidate questions, search engines generating relevant question suggestions, and human-computer interaction robots conducting human-computer dialogues.

[0063] In response to the technical problem of low quality of question generation in the field of natural language processing in traditional technologies, the present invention provides a question generation method based on natural language processing: when the data source for question generation is text, a semantic graph is first generated, and then questions are generated based on the semantic graph, so that when the data source for question generation is text, the abstract text is semantically graphed, and comprehensive and accurate questions from different angles are generated through intuitive and vivid visual semantic graphs, which can improve the quality of question generation under text type data sources.

[0064] See also Figure 1 and Figure 2 The question generation method based on natural language processing provided by the embodiment of the present invention includes the following steps:

[0065] S11. In response to the question generation instruction, determine the source of the question data corresponding to the question generation:

[0066] When question generation is initiated, a question generation instruction is generated. In response to the question generation instruction, the question data source corresponding to the question generation is determined. The question data source represents the data source of the question to be generated, that is, the type of content based on which the question is generated. The question data source includes but is not limited to a text data source, a structured data source, an image data source, or a conversation data source. Among them, a text data source represents a data source that uses text corresponding to paragraphs and sentences as the source of the question. The text data source includes but is not limited to textbook paragraphs, news articles, document text, and encyclopedia content; a structured data source represents a data source that uses structured data as the source of the question. The structured data source includes but is not limited to knowledge graph triples, database tables, and key-value pairs; an image data source represents a data source that uses images as the source of the question. The image data source includes but is not limited to images and videos; a conversation data source represents a data source that uses conversations as the source of the question. The conversation data source includes but is not limited to customer service conversation records, educational counseling conversations, and social media discussion threads.

[0067] S12. Determine whether the problem data source is a preset text type data source:

[0068] The preset text type data source represents a pre-set text type question data source capable of converting text into a semantic diagram corresponding to a drawing form. The text forms corresponding to the preset text type data source include, but are not limited to, text paragraphs and sentences corresponding to text types as the data source of the question source. The text types corresponding to the preset text type data source include, but are not limited to, narrative text types such as novels, folk tales, and historical records; metaphorical / symbolic text types such as poetry, fables, and philosophical texts; and multi-role interactive text types such as drama scripts and meeting minutes. For the preset text type data source, other types may also be provided, including, but not limited to, preset structured type data sources, preset image type data sources, and preset dialogue type data sources.

[0069] According to the above settings, it is determined whether the problem data source is a preset text type data source, that is, it is determined whether the problem data source is a data source that uses text type data as the source of the problem.

[0070] S13. If the question data source is not a preset text type data source, determine a corresponding target preset question generation method to generate the question;

[0071] S14. When the question data source is a preset text type data source, perform natural language processing on the question data source to obtain natural language features.

[0072] Explanatory note: In addition to the preset text-type data source, corresponding preset question generation methods are set for other types of data sources, as described above, including but not limited to preset structured-type data sources, preset image-type data sources, and preset dialogue-type data sources. For example, for the preset structured-type data source, a corresponding preset structured-question generation method is set; for the preset image-type data source, a corresponding preset image-question generation method is set; and for the preset dialogue-type data source, a corresponding preset dialogue-question generation method is set. This allows the question generation methods to be automatically adapted to different data source types, enabling the use of appropriate question generation methods for different question data sources, thereby improving the accuracy of questions generated for different types of question data sources.

[0073] According to the above settings, when the problem data source is not a preset text type data source, the corresponding question generation method is determined as the target preset question generation method according to the data source type to which the problem data source belongs, and when the problem data source is a preset text type data source, the problem data source is subjected to natural language processing to obtain natural language features. The natural language features represent the text features corresponding to the problem data source. Natural language features include but are not limited to named entities and semantic features. Among them, natural language processing (NLP) mainly studies how to enable computers to understand, process, generate and simulate human language capabilities. Natural language processing includes but is not limited to word segmentation, part-of-speech tagging, named entity recognition, and semantic understanding.

[0074] S15. Determine whether the natural language feature meets the preset semantic image element conditions:

[0075] Pre-set semantic diagram element conditions, that is, preset semantic diagram element conditions, which represent pre-set conditions corresponding to the semantic diagram corresponding to the text in painting form and the drawing elements required for drawing the picture. The preset semantic diagram element conditions include but are not limited to whether the natural language features express the drawing elements required for drawing the picture, including but not limited to the scene base, character entity, dynamic elements, atmosphere rendering or perspective control, that is, the picture is composed of corresponding drawing elements. Here, it is necessary to first determine whether the natural language features describe the drawing elements or more drawing elements necessary to construct the painting, so as to construct the semantic diagram of the painting form corresponding to the text.

[0076] The scene base represents the physical or abstract environmental framework that supports the narrative, including spatial structure, historical background, and cultural context. For example, if the scene base is "spatial prepositional phrase - in the forest," the semantic expression corresponds to the spatial prepositional phrase - in the forest, and the corresponding visual representation can be background texture - tree density indicating spatial depth. Whether the natural language feature meets the preset semantic diagram element conditions of the scene base dimension can be determined by determining whether it contains, including but not limited to, the expression corresponding to the spatial prepositional phrase.

[0077] A character entity represents an intelligent or non-intelligent subject with independent behavioral logic and emotional attributes. For example, if the character entity is "Character Description - Girl in Red Dress," the semantic expression corresponding to the character description - Girl in Red Dress could have a corresponding visual representation of "concrete character design + costume color reproduction." Whether the natural language features meet the pre-set semantic diagram element conditions for the character entity dimension can be determined by determining whether they include, but are not limited to, the expression corresponding to the character description.

[0078] Dynamic elements represent changes in time and space, including physical movement and state transitions. For example, if the dynamic element is an action verb (running / falling), the semantic expression corresponding to the action verb (running / falling) can be represented by a visual representation of a line of motion / blurred afterimage / frame. Whether the natural language feature meets the pre-set semantic diagram element requirements for the dynamic element dimension can be determined by determining whether it contains, including but not limited to, expressions corresponding to the action verb.

[0079] Atmosphere rendering represents the emotional field and stylized expression conveyed through visual elements. For example, if the atmosphere rendering is "Rhetorical Device - Metaphor / Exaggeration," the semantic expression corresponds to the rhetorical device - Metaphor / Exaggeration, and the corresponding visual representation can be "the fusion of surreal elements, such as a broken heart and cracked glass." Whether the natural language features meet the preset semantic diagram element conditions for the atmosphere rendering dimension can be determined by determining whether they contain, including but not limited to, the expression corresponding to the rhetorical device.

[0080] Perspective control represents the selective presentation strategy of visual information, encompassing both physical and narrative perspectives. For example, if perspective control is "personal pronouns - I / He," the semantic expression corresponding to the personal pronouns "I / He" could be visually represented as "camera angle - first person → subjective perspective." Whether the natural language features meet the pre-set semantic diagram element requirements for perspective control can be determined by determining whether they include, but are not limited to, expressions corresponding to the personal pronouns.

[0081] Based on the above settings, determine whether the natural language features meet the preset semantic diagram element conditions, for example, determine whether the natural language features contain relevant content expressions of the drawing elements corresponding to the scene base, character entity, dynamic elements, atmosphere rendering or perspective control.

[0082] S16: If the natural language feature does not meet the preset semantic image element conditions, the target semantic image elements corresponding to the natural language feature cannot be determined;

[0083] S17. When the natural language feature satisfies a preset semantic diagram element condition, determine a number of target semantic diagram elements corresponding to the natural language feature.

[0084] Explanatoryally, when the natural language feature does not meet the preset semantic diagram element conditions, it indicates that the drawing elements required for converting the text into the semantic diagram corresponding to the painting form are not met. For example, as described above, when judging whether the natural language feature contains the expression corresponding to the spatial prepositional phrase to determine whether the natural language feature meets the preset semantic diagram element conditions of the scene substrate dimension, when it is determined that the natural language feature does not contain the expression corresponding to the spatial prepositional phrase, it is determined that the natural language feature does not meet the preset semantic diagram element conditions of the scene substrate dimension, indicating that the natural language feature does not contain the painting element of the scene substrate required for painting, and the text corresponding to the problem data source cannot be converted into the semantic diagram corresponding to the painting form. The same applies to other cases. Therefore, the target semantic diagram elements corresponding to the natural language feature are not determined. For the case where the text corresponding to the problem data source cannot be converted into the semantic diagram corresponding to the painting form, a method for generating questions based on the text can be set to generate the corresponding questions. For example, existing text-based question generation methods including but not limited to template-based methods and rule-based methods can be adopted. Existing corresponding technical means can be used for reference and will not be repeated here.

[0085] Similarly, when the natural language features meet the preset semantic diagram element conditions, it indicates that the natural language features contain the corresponding painting elements required for painting, and can convert the text corresponding to the problem data source into a semantic diagram corresponding to the painting form. In this case, several target semantic diagram elements corresponding to the natural language features are specifically determined, for example, the specific information corresponding to the scene base is determined, the specific information corresponding to the character entity is determined, the specific information corresponding to the dynamic element is determined, the specific information corresponding to the atmosphere rendering is determined, or the specific information corresponding to the perspective control is determined, and other specific contents of the drawing elements.

[0086] S18. Generate a semantic graph corresponding to the problem data source according to all target semantic graph elements and based on a preset semantic graph generation model:

[0087] A pre-set semantic diagram generation model, i.e., a preset semantic diagram generation model, represents a model for drawing corresponding diagrams according to all target semantic diagram elements contained in the semantics corresponding to the text, i.e., the preset semantic diagram generation model is a processing model for organizing all target semantic diagram elements into paintings, thereby constructing a visual picture logical expression, for example, a model for generating diagrams corresponding to pictures or images according to text, for example, a model for expressing the semantics, content, and core ideas of the text with pictures or images corresponding to scene diagrams or scenario diagrams, and the preset semantic diagram generation model includes but is not limited to generative adversarial networks (GANs) and autoregressive models.

[0088] Based on the above settings, according to all target semantic diagram elements and based on the preset semantic diagram generation model, a semantic diagram corresponding to the problem data source is generated, that is, a corresponding picture or image is generated from the text, wherein the semantic diagram means that the semantic content expressed by the text corresponding to the problem data source is represented by a diagram, that is, the semantic diagram represents a diagram of a scene or scenario corresponding to the sun, people, plants, animals, rivers, roads or parks that expresses the corresponding semantic content. The semantic diagram is not a graph structure containing nodes and edges. The semantic diagram includes but is not limited to pictures, images, and stereograms, so as to refine the problem data source and perform modal conversion, and convert the problem data source from an abstract text modality to an image-based and intuitive diagram modality, so as to refine the text through the diagram to reduce the noise and interference of the text expression corresponding to the problem data source, and focus on the main and important semantic content and core ideas expressed by the problem data source, so that the generated questions can also focus on the main and important semantic content and core ideas expressed by the problem data source.

[0089] S19. Generate questions based on the semantic graph and a preset graph question generation method to obtain questions corresponding to the question data source:

[0090] A method of generating diagram questions is pre-set, i.e., a preset method of generating diagram questions. The preset method of generating diagram questions refers to a method of generating questions based on diagrams including but not limited to drawings and images. The preset method of generating diagram questions includes but is not limited to a method of generating questions using pre-trained models and structured visual understanding. The use of pre-trained models includes but is not limited to calling ready-made APIs to generate questions based on diagrams. Structured visual understanding includes but is not limited to a method of generating questions based on diagrams corresponding to a combination of visual element extraction and template filling. The above-mentioned preset method of generating diagram questions can be directly used using existing technical means and will not be repeated here.

[0091] Based on the above settings, according to the semantic diagram, and using methods including but not limited to the use of pre-trained models, structured visual understanding, etc., questions are generated to obtain the questions corresponding to the question data source, and then based on the conversion of different modes between text-diagram-text, questions are generated to obtain the questions corresponding to the question data source, and the abstract and complex text expression forms are refined into vivid, intuitive, and simple concrete forms. Through the diagram mode, the main and important semantic content and core ideas expressed by the question data source are focused on, and then the questions are generated according to the semantic diagram. As a result, the generated questions can also focus on the main and important semantic content and core ideas expressed by the question data source, thereby improving the question generation quality under text type data sources.

[0092] In an embodiment of the present invention, in response to a question generation instruction, the question data source corresponding to the question generation is determined, and whether the question data source is a preset text type data source. If the above judgment is yes, the question data source is subjected to natural language processing to obtain natural language features. When a number of target semantic diagram elements corresponding to the natural language features can be determined, a semantic diagram corresponding to the question data source is generated based on a preset semantic diagram generation model. Then, based on the semantic diagram, question generation is performed to obtain the question corresponding to the question data source. When the question data source is not a preset text type data source, a corresponding target preset question generation method is determined to perform question generation. Therefore, when the question data source is a preset text type data source, the question data source is converted from text mode to graphic mode and then back to text mode, thereby realizing the refinement of the question data source based on natural language processing and modal conversion, so as to refine the abstract and complex text expression into a vivid, intuitive, simple and concrete form, and focus on the question data source through the graphic mode. The main and important semantic content and core ideas expressed by the problem data source can then be used to generate questions based on the semantic diagram. The generated questions can also focus on the main and important semantic content and core ideas expressed by the problem data source, thereby obtaining the questions corresponding to the problem data source. Comprehensive and accurate questions from different angles can be generated through intuitive and vivid refined semantic diagrams, thereby improving the quality of question generation under text type data sources. In the case where the problem data source for question generation is non-text type, a suitable question generation method is adopted, so that suitable question generation methods are adopted for different problem data sources, which can improve the accuracy and comprehensiveness of the corresponding questions generated by each type of problem data source, thereby comprehensively improving the quality of question generation, and then improving the intelligent question generation effect in different intelligent scenarios corresponding to but not limited to automatic generation of test questions in the education field, intelligent tutoring systems to help students understand materials, question and answer systems to generate candidate questions, search engines to generate relevant question suggestions, and human-computer interaction robots to conduct human-computer dialogues.

[0093] In one embodiment, see Figure 3 , perform natural language processing on the question data source to obtain natural language features, including:

[0094] S31. Segment the question data source to obtain a number of tokens;

[0095] S32, performing part-of-speech tagging on the word unit to obtain a tagged word unit;

[0096] S33, performing named entity recognition based on the marked word to obtain the target entity;

[0097] S34, based on the preset semantic recognition model, semantic recognition is performed on the problem data source to obtain target semantics;

[0098] S35. Use the labeled word, target entity, and target semantics as the natural language features corresponding to the problem data source.

[0099] Explanatory, a semantic recognition model is pre-set, that is, a preset semantic recognition model. The preset semantic recognition model represents a model for recognizing semantics. The preset semantic recognition model includes but is not limited to a Word2Vec model, a BERT-type model, a generative pre-trained transformer (GPT), and a large language model (LLM).

[0100] According to the above settings, the question data source is segmented to split the sentence into words or subwords. The segmentation can be performed by, but not limited to, BERT's WordPiece to obtain several word units, and the word units are tagged with parts of speech, for example, to identify nouns, verbs, etc., to obtain tagged word units. Then, named entity recognition (NER) is performed based on the tagged word units. Named entity recognition (NER) is to identify and classify entities with specific meanings from text, such as identifying entities such as people, places, and time to obtain target entities. Based on a preset semantic recognition model, semantic recognition is performed on the question data source to obtain target semantics. Semantic recognition includes but is not limited to lexical semantics, syntactic semantics, contextual semantics, intentional semantics, and emotional semantics. Lexical semantics represents the meaning of the word itself, syntactic semantics represents the meaning conveyed by sentence structure, contextual semantics represents the meaning determined by the context, intentional semantics represents the speaker's purpose, and emotional semantics represents the emotional tendency of the text. Finally, the labeled words, target entities, and target semantics are used as the natural language features corresponding to the question data source to obtain features from different angles of the question data source. The features from different angles are then integrated to generate a semantic graph, which can improve the consistency between the semantic graph and the content expressed by the question data source, that is, improve the accuracy of semantic graph generation, and then improve the accuracy of question generation, thereby improving the quality of question generation under text type data sources.

[0101] In one embodiment, see Figure 4 , all target semantic graph elements include: target scene base graph elements, target character entity graph elements, target dynamic element graph elements, target atmosphere rendering graph elements, and target perspective control graph elements; based on all target semantic graph elements and based on a preset semantic graph generation model, a semantic graph corresponding to the problem data source is generated, including:

[0102] S41, generating a model based on a preset picture, and performing scene base modeling according to target scene base diagram elements to obtain a target scene base visual expression;

[0103] S42, based on the target scene base and according to the target character entity diagram elements, modeling the target character entity to obtain a visual representation of the target character entity;

[0104] S43, based on the target scene base and according to the target dynamic element diagram elements, target dynamic element modeling is performed to obtain a target motion trajectory visual expression corresponding to the target dynamic element;

[0105] S44, based on the target scene base and according to the target atmosphere rendering diagram elements, perform target atmosphere rendering modeling to obtain a target atmosphere visual expression;

[0106] S45. Based on the target scene base and according to the target perspective, visual control modeling is performed to obtain a visual expression of the target lens angle;

[0107] S46. Based on the semantics corresponding to the problem data source, the target scene base visual expression, the target character entity visual expression, the target motion trajectory visual expression, the target atmosphere visual expression and the target lens angle visual expression are visualized and logically expressed to obtain the semantic diagram corresponding to the problem data source.

[0108] In the present invention, drawing / picture generation models are pre-set, that is, preset drawing generation models. The preset drawing generation models represent models that generate corresponding drawings according to target semantic diagram elements. The preset drawing generation models include but are not limited to generative adversarial networks (GANs) and diffusion models. It should be noted that the difference between drawings and images lies in: 1) drawings have low semantic density, while images have high semantic density; 2) drawings have concise structures and prominent emphasis, while images have complex structures and rich details; 3) drawings generally refer to graphics in styles such as digital paintings and stick drawings, while images generally refer to diagrams containing photo-level details.

[0109] Based on a preset image generation model and the target scene base graphical elements, scene base modeling is performed to obtain a target scene base visual representation. The target scene base graphical elements represent the target semantic graphical elements of the scene base dimension. For example, if the target scene base graphical element is "spatial prepositional phrase - in the forest," the spatial prepositional phrase corresponding to the semantic representation, "in the forest," is converted into a corresponding visual representation through corresponding modeling, namely, "background texture - tree density represents spatial depth."

[0110] Based on the target scene foundation and the target character entity graphical elements, the target character entity is modeled to obtain a visual representation of the target character entity. The target character entity graphical elements represent the target semantic graphical elements for the character entity dimension. For example, if the target character entity graphical element is "Character Description - Girl in Red Dress," the corresponding modeling can convert the character description - girl in red dress into a corresponding visual representation of "concrete character design + costume color reproduction."

[0111] Based on the target scene base and the target dynamic element diagram elements, target dynamic element modeling is performed to obtain a visual representation of the target motion trajectory corresponding to the target dynamic element. The target dynamic element diagram elements represent the target semantic diagram elements of the dynamic element dimension. For example, if the target dynamic element diagram element is "action verb - run / fall", the action verb - run / fall is converted into the corresponding visual representation of "motion line / blurred afterimage / storyboard" through corresponding modeling.

[0112] Based on the target scene base and the target atmosphere rendering graphical elements, target atmosphere rendering modeling is performed to obtain the target atmosphere visual representation. The target atmosphere rendering graphical elements represent the target semantic graphical elements in the atmosphere rendering dimension. For example, if the target atmosphere rendering graphical element is "Rhetorical Device - Metaphor / Exaggeration", the rhetorical device - metaphor / exaggeration is converted into the corresponding visual representation through corresponding modeling, forming a fusion of surreal elements, such as a broken heart → cracked glass.

[0113] Based on the target scene substrate and the target perspective control diagram elements, visual control modeling is performed to obtain the target lens angle visual representation. The target perspective control diagram elements represent the target semantic diagram elements for the perspective control dimension. For example, if the target perspective control diagram element is "personal pronouns - I / he", the personal pronouns - I / he are converted into the corresponding visual representation of "lens angle - first person → subjective perspective" through corresponding modeling.

[0114] Based on the semantic content and logic corresponding to the problem data source, the target scene base visual expression, target character entity visual expression, target motion trajectory visual expression, target atmosphere visual expression, and target lens angle visual expression are organized into drawings with corresponding content and logic. The drawing organization includes determining the respective positions of the target scene base visual expression, target character entity visual expression, target motion trajectory visual expression, target atmosphere visual expression, and target lens angle visual expression and the relationship between them, that is, realizing the logical expression of the visual picture and obtaining the semantic diagram corresponding to the problem data source.

[0115] For example, based on a preset picture generation model, a picture corresponding to the problem data source can be sketched. Sketching usually refers to expressing the main content concisely. Sketching is used to sketch pictures corresponding to natural language features to obtain pictures corresponding to the problem data source, that is, semantic diagrams corresponding to the problem data source. This can make the pictures focus on the main and important semantic content and core ideas expressed by the problem data source, thereby abstracting the text into the intuitiveness of the pictures in the form of pictures, and grasping the key points and main content, realizing the transformation of abstract logic into concrete spatial relationships, and realizing an innovative problem generation method that integrates cross-modal thinking corresponding to text and pictures. The problem generation method of the embodiment of the present invention elevates traditional text into a visual extraction and enhancement form. Its core idea is to transform linear text into a visual cognitive space through a visual bridge, extract the key points and main content of the text, and then generate questions from a visual perspective, which can improve the accuracy and quality of question generation.

[0116] The embodiment of the present invention generates pictures corresponding to the question data source according to natural language features and based on a preset picture generation model. Therefore, by taking advantage of the concise characteristics of the pictures, it is possible not only to improve the efficiency of question generation, but also to refine abstract and complex text expressions into vivid, intuitive, and concise concrete forms. It is possible to focus on the main and important semantic content and core ideas expressed by the question data source through the graphical mode corresponding to the pictures, and then generate questions based on the semantic diagram. The generated questions can also focus on the main and important semantic content and core ideas expressed by the question data source, thereby obtaining the questions corresponding to the question data source. It is possible to generate accurate questions from different angles through intuitive, vivid, and concise refined pictures, thereby improving the question generation quality under text type data sources.

[0117] In one embodiment, before generating a question according to the semantic graph and based on a preset graph question generation method and obtaining the question corresponding to the question data source, the method further includes:

[0118] Based on the preset CLIP graphic verification model, determine whether the semantic diagram is similar to the semantics of the problem data source. If the semantics are similar, execute the steps of generating questions according to the semantic diagram and based on the preset diagram problem generation method to obtain the question corresponding to the problem data source; otherwise, report an error or regenerate the semantic diagram.

[0119] Explanatoryly, a CLIP image-text verification model is pre-set, that is, a preset CLIP image-text verification model. The preset CLIP image-text verification model represents a model that performs consistency verification on the semantic graph generated above with the semantic text corresponding to the problem data source based on the CLIP model (Contrastive Language-Image Pretraining). The CLIP model is a multimodal model. The CLIP model maps the semantic graph and the semantic text corresponding to the problem data source to the same vector space through comparative learning, thereby realizing cross-modal semantic understanding between the image and the text, and verifying the semantic consistency between the semantic graph and the problem data source.

[0120] Based on the preset CLIP image-text verification model, it is determined whether the semantic diagram is similar to the semantics of the problem data source, that is, the consistency of the semantics corresponding to the content of the semantic diagram and the content of the problem data source is judged. Since the CLIP model itself contains a text encoder such as Transformer and an image encoder such as ViT or ResNet, the CLIP model can map the semantic diagram and the semantic text of the problem data source to the same vector space through comparative learning, thereby realizing cross-modal semantic understanding between the semantic diagram and the semantic text of the problem data source, and compare the semantic diagram with the semantic text of the problem data source through similarity, such as cosine similarity, to perform consistency verification between the semantic diagram and the semantic text of the problem data source. The similarity comparison is generally judged by comparing the calculated similarity value with the corresponding similarity threshold, which will not be repeated here. Therefore, when the semantic diagram is similar to the semantics of the problem data source, it indicates that the generated semantic diagram more accurately visually expresses the content and semantics of the problem data source. When the semantic diagram is similar to the semantics of the problem data source, the steps of question generation according to the semantic diagram and the preset diagram question generation method are performed to obtain the question corresponding to the problem data source, and the corresponding question is generated. Therefore, the generated question is more consistent with the content and semantics of the problem data source, that is, the quality of the generated question is higher. When the semantic diagram is not similar to the semantics of the problem data source, it indicates that the generated semantic diagram does not accurately visually express the content and semantics of the problem data source, and an error is reported or the semantic diagram is regenerated.

[0121] In an embodiment of the present invention, a consistency check is performed between the semantic graph and the semantics of the question data source based on cross-modal semantic understanding, thereby ensuring that the semantic graph used for question generation has a high degree of similarity with the semantics of the question data source, that is, there is consistency between the semantic graph and the semantics of the question data source, so as to ensure that the semantic graph more accurately expresses the semantics of the question data source. The questions generated on this basis can also have a high degree of semantic fit with the semantics of the question data source, thereby further improving the quality of question generation under text type data sources.

[0122] In one embodiment, see Figure 5 , according to the semantic diagram and based on the preset diagram question generation method, questions are generated to obtain questions corresponding to the question data source, including:

[0123] S51, performing image preprocessing on the semantic image to obtain a preprocessed image;

[0124] S52, extracting visual features corresponding to the preprocessed image based on a preset visual feature extraction method to obtain a plurality of visual feature texts;

[0125] S53, according to the preprocessed image and the visual feature text, and based on a preset visual language model, associating different visual feature texts to obtain a plurality of visual text contents;

[0126] S54. Generate questions corresponding to the visual text content based on the preset question generation model according to the visual text content, and obtain questions corresponding to the question data source.

[0127] The preset visual feature extraction method represents a method for extracting image features from an image. The preset visual feature extraction method includes but is not limited to target detection, for example: using a YOLOv8 model or a DETR model to obtain an object list and position, visual relationship detection including traditional methods based on template matching, intelligent recognition methods based on machine learning, and deep learning algorithms, etc.

[0128] Visual language models, namely Vision-Language Models, abbreviated as VLMs, are preset visual language models that understand and generate language information related to visual content through representation learning across visual and language modalities. Preset visual language models include but are not limited to CLIP models and BLIP models (Bootstrapping Language-Image Pre-training). Among them, visual language models are multimodal models that can learn from images and texts, which can accept image and text inputs and generate text outputs. Visual language models (Visual Language Models) are models that can learn from images and texts at the same time to handle many tasks. Therefore, based on the preset visual language model, it is possible to combine the preprocessed image with the visual feature text, similar to picture description, based on the preprocessed image and combined with the visual feature text, describe the preprocessed image from different angles, learn the content contained in the preprocessed image from the preprocessed image in a comprehensive and accurate manner, and then generate comprehensive and accurate questions from different angles corresponding to the semantic image, thereby improving the quality of question generation under text type data sources.

[0129] The preset question generation model refers to a model that generates questions based on the corresponding text. The preset question generation model includes but is not limited to a Seq2Seq model and a Generative Pre-trained Transformer (GPT).

[0130] According to the above settings, the semantic graph is first subjected to image preprocessing. Image preprocessing refers to the optimization operation performed on the originally generated semantic graph before the subsequent core analysis or processing of the semantic graph. For example, the semantic graph is subjected to image preprocessing including but not limited to denoising, resolution adjustment, rotation, scaling, and cropping to obtain a preprocessed graph; then the visual features corresponding to the preprocessed graph are extracted, for example, target detection is performed to obtain the object and its corresponding position, scene classification of the semantic graph is performed, or visual relationship detection is performed to obtain a number of visual feature texts; then, according to the preprocessed graph and the visual feature text, and based on a preset visual language model, the preprocessed graph is combined with the visual feature text, and different visual feature texts are associated to obtain a number of visual text contents, and the visual text content describes the semantic graph. The content included in the graph, that is, what the semantic graph includes, is included, so as to realize the conversion from visual mode to text mode; finally, according to the visual text content and based on the preset question generation model, the question corresponding to the visual text content is generated, and the question corresponding to the problem data source is obtained, thereby realizing the conversion from the text mode of the problem data source to the visual mode through refinement, and then from the visual mode to the text mode, and generating questions. Since the text mode of the problem data source is converted to the visual mode through refinement, the noise and interference corresponding to the abstract and complex description corresponding to the text mode of the problem data source are filtered out, thereby converting it into the text mode according to the visual mode and generating questions, which can make the generated questions focus on the main and important semantic content and core ideas expressed by the problem data source, and generate comprehensive and accurate questions.

[0131] In an embodiment of the present invention, a semantic graph is preprocessed to obtain a preprocessed graph, and visual features corresponding to the preprocessed graph are extracted to obtain a number of visual feature texts. Then, according to the preprocessed graph and the visual feature texts, and based on a preset visual language model, different visual feature texts are associated to obtain a number of visual text contents. Then, according to the visual text content and based on a preset question generation model, questions corresponding to the visual text contents are generated to obtain questions corresponding to the question data source. Therefore, when the question data source is a preset text type data source, the question data source is converted from a text mode to a graphic mode, and the question data source is refined based on natural language processing and modal conversion is performed to refine the abstract and complex text expression form into a vivid and intuitive one. , a simple and concrete form, and then obtains the corresponding text mode based on the diagram mode, realizing the sequential conversion from text mode-diagram mode-text mode, and based on the above conversion process, the problem data source is refined, so as to focus on the main and important semantic content and core ideas expressed by the problem data source, reduce the noise and interference of the text expression corresponding to the problem data source, and then generate the corresponding questions. The generated questions can also focus on the main and important semantic content and core ideas expressed by the problem data source, so as to obtain the questions corresponding to the problem data source, and can generate comprehensive and accurate questions from different angles corresponding to the semantic diagram through intuitive and vivid refined semantic diagrams, thereby improving the quality of question generation under text type data sources.

[0132] In one embodiment, questions are generated based on the semantic graph and a preset graph question generation method to obtain questions corresponding to the question data source, including:

[0133] Generate questions based on the semantic graph and a preset graph question generation method to obtain benchmark questions;

[0134] Perform named entity recognition on the benchmark question to obtain the question entity;

[0135] Identify the entity type to which the problem entity belongs;

[0136] Determine a corresponding preset replacement entity set according to the entity type, where the preset replacement entity set includes the preset replacement entities;

[0137] According to a preset replacement entity set and based on a preset replacement entity transformation method, the problem entity is transformed into a corresponding replacement entity to obtain a transformation problem;

[0138] The benchmark problem and the transformation problem are used as the problems corresponding to the problem data source.

[0139] The preset replacement entity set represents the entity set that replaces the problem entity. The preset replacement entity set includes preset replacement entities. The preset replacement entities include but are not limited to synonymous replacements, near-sense replacements, antonymous replacements, superordinate or subordinate replacements, and replacement entities corresponding to the same type of replacements of the problem entity, where the same type of replacements refers to replacements of the same type.

[0140] The preset replacement entity transformation method represents the method for transforming the question entity. The preset replacement entity transformation methods include, but are not limited to, several replacement methods corresponding to synonymous replacement, near-meaning replacement, antonym replacement, superordinate or hypoordinate replacement, similar replacement, and question word replacement. Question word replacement indicates changing the focus of the question, for example, replacing "When was it released?" with "Who released it?" or "Where was it released?" Furthermore, corresponding preset replacement entity transformation methods are set for different types of question entities. For example, for question entities corresponding to nouns, the preset replacement entity transformation methods include, but are not limited to, similar replacement, superordinate or hypoordinate replacement, and for question entities corresponding to adjectives, the preset replacement entity transformation methods include, but are not limited to, synonymous replacement, near-meaning replacement, and antonym replacement.

[0141] With the above settings, questions are generated according to the semantic graph and based on the preset graph question generation method to obtain benchmark questions. The benchmark questions refer to both questions directly generated according to the semantic graph and questions that serve as a benchmark for further generating questions.

[0142] Perform named entity recognition on the benchmark question to obtain the question entity. The question entity represents the entity corresponding to the benchmark question. The "question" involved in the question entity is only used to distinguish different entities and is not used to limit different entities. Named entity recognition is as described above and will not be repeated here; and identify the entity type to which the question entity belongs. The entity type represents the category of the entity. The entity type includes but is not limited to the entity types corresponding to nouns and verbs. For example, the question entity belongs to the verb type and the question entity belongs to the noun type.

[0143] According to the entity type, a corresponding preset replacement entity set is determined, and the preset replacement entity set includes preset replacement entities, wherein corresponding preset replacement entity sets are pre-set for different entity types, for example, a corresponding preset replacement entity set is set for a noun type entity, and a corresponding preset replacement entity set is set for a verb type entity; then, according to the preset replacement entity set, and based on the replacement mode corresponding to the preset replacement entity transformation mode, including but not limited to synonymous replacement, near-sense replacement, antonym replacement, superordinate or subordinate replacement, similar replacement, and question word replacement, the question entity is transformed into a corresponding replacement entity to obtain a transformed question, wherein the several replacement entities corresponding to the transformation of the question entity into the corresponding replacement entity are a subset of the preset replacement entity set, Then, the benchmark problem and the transformed problem are used as the problems corresponding to the problem data source, so that the benchmark problem is transformed on the basis of the benchmark problem to obtain the transformed problem, thereby expanding the benchmark problem. Among them, the transformed problems include the same or similar problems, opposite problems, and the benchmark problem and the transformed problem are used as the problems corresponding to the problem data source. On the basis of focusing on the main and important semantic content and core ideas expressed by the problem data source through semantic diagrams, the dimension, scope and content of the problems corresponding to the problem data source can be expanded, thereby further improving the comprehensiveness and accuracy of the generated problems, thereby improving the problem generation quality under text type data sources, especially in the field of intelligent self-service, by transforming problems, the interaction quality and efficiency of human-computer interaction can be improved.

[0144] For example, for the generated benchmark question "What is the little boy doing in the park", synonym transformation "What is the little boy doing", proximal transformation "Can you describe the little boy's activities in the park", angle transformation "Who is doing what in the park", antonym transformation "Is there a little girl doing anything in the park", etc., according to different benchmark questions, at least one of the above transformations is performed to expand the scope and content of the question, which can further improve the comprehensiveness and accuracy of the generated questions, thereby improving the quality of question generation under text type data sources.

[0145] The embodiment of the present invention generates questions based on semantic diagrams to obtain benchmark questions, and transforms the benchmark questions to obtain transformed questions. The benchmark questions and the transformed questions are then used as questions corresponding to the question data source. On the basis of accurate generation based on the benchmark questions, comprehensive and accurate questions from different perspectives can be generated through transformations based on the benchmark questions, thereby improving the quality of question generation under text-type data sources, and further improving the intelligent effects in different intelligent scenarios corresponding to, including but not limited to, automatic generation of test questions in the education field, intelligent tutoring systems helping students understand materials, question-and-answer systems generating candidate questions, search engines generating related question suggestions, and human-computer interaction robots conducting human-computer dialogues.

[0146] In one embodiment, after identifying the entity type to which the problem entity belongs, the method further includes:

[0147] According to the entity type, determine the preset correspondence between the preset entity corresponding to the problem entity and the preset knowledge graph;

[0148] According to the preset correspondence, the question entity is associated with the corresponding preset knowledge graph, wherein the preset knowledge graph includes a preset entity extended question chain, and the preset entity extended question chain includes several preset extended question templates;

[0149] Associate the question entity with the preset extended question template to obtain the extended question;

[0150] Use extended questions as questions corresponding to the source of the question data.

[0151] Explanatory, a preset correspondence between a preset entity and a preset knowledge graph is pre-set, that is, a preset correspondence between a preset entity and a preset knowledge graph. The preset knowledge graph represents a pre-set knowledge graph associated with the preset entity. The preset knowledge graph contains concepts, attributes, content and mutual relationships with other entities related to the preset entity in the corresponding business field or application scenario, which are expressed in the form of a graph. For example, "Classmate A" and his family relationships, school relationships, and friend relationships can be organized into a corresponding knowledge graph. Moreover, for different types of entities, corresponding preset knowledge graphs are pre-set for several preset entities of each type, that is, a preset entity type-preset entity-preset knowledge graph composition correspondence, for example, preset entity type A-preset entity 1-preset knowledge graph 1; preset entity type A-preset entity 2-preset knowledge graph 2; preset entity type B-preset entity 1-preset knowledge graph 1, and so on.

[0152] The preset knowledge graph includes a preset entity extended question chain, and the preset entity extended question chain includes several preset extended question templates, wherein the preset entity extended question chain represents a question chain organized in the form of a chain with several preset extended question templates starting from the preset entity, and the preset extended question template represents a template for a pre-set extended question, that is, for the preset entity in the corresponding business scenario or application scenario, what problems will be involved, and a general question framework is generated in advance in the form of a question template. In the corresponding business scenario or application scenario, the preset entity extended question chain and the several preset extended question templates it contains need to be set by relevant personnel. Exemplarily, for the above example, for "Student A", two extended question chains can be set with "Student A" as the starting point, wherein one extended question chain involves the corresponding question templates of his family relationship, and the other extended question chain involves the corresponding question templates of his school relationship. For example, the extended question chain involving his family relationship and its corresponding question templates may be "Are Student A's parents in the countryside or in the city?", "Are Student A's parents working or doing business in the city?" In the corresponding specific business scenarios or application scenarios, the above technical concept can be adopted to organize several question templates into extended question chains in the form of chains, and the preset entity extended question chains may include but are not limited to vertical extended question chains and horizontal extended question chains. The vertical extended question chain represents A chain of questions that progress vertically layer by layer and expand in depth, a horizontally extended question chain represents a chain of questions that expand in horizontal breadth. For instance, for the above example, "Are classmate A's parents in the countryside or in the city?", "Are classmate A's parents working or doing business in the city?", this form of chain of questions that progress vertically layer by layer and expand in depth is a vertically extended question chain. If it expands from "Are classmate A's parents in the countryside or in the city?" to "Are classmate A's friends in the countryside or in the city?", it is a horizontally extended question chain composed of several questions that expand in horizontal breadth. The above example is only used to illustrate the technical conception and core idea of ​​the present technical solution, and is not used to limit the present technical solution. The same applies to other examples.

[0153] Based on the above settings, according to the entity type, the preset correspondence between the preset entity corresponding to the problem entity and the preset knowledge graph is determined. Among the several pairs of preset correspondences corresponding to the entity type, the preset entity corresponding to the problem entity and its corresponding preset knowledge graph can be determined by judging whether the problem entity and the preset entity are the same or similar; then, according to the preset knowledge graph corresponding to the preset correspondence, the problem entity is associated with the corresponding preset knowledge graph, wherein the preset knowledge graph includes a preset entity extended question chain, and the preset entity extended question chain includes several preset extended question templates. Then, the problem entity is associated with the preset extended question template in a manner including but not limited to filling to obtain a specific extended question, and finally the extended question is used as the corresponding question data source. Problem, thereby extending the benchmark problem vertically or horizontally to obtain an extended problem, and using the extended problem as the problem corresponding to the problem data source, wherein the extended problem is a problem that is expanded and deepened on the basis of the benchmark problem along its original problem direction or problem logic, and is a process of enriching and improving the scope, depth or details of the benchmark problem. The extension process focuses more on the in-depth excavation and expansion of the original problem, aiming to provide more comprehensive and in-depth problems. On the basis of focusing on the main and important semantic content and core ideas expressed by the problem data source, the depth, level, scope and content of the problems corresponding to the problem data source are further expanded, thereby further improving the comprehensiveness and accuracy of the generated problems, thereby improving the problem generation quality under text type data sources.

[0154] For example, the generated benchmark question "What is the little boy doing in the park" can be extended according to the corresponding logic. The extended questions can be "Who is the little boy in the park with?", "How did the little boy get to the park?", etc. Extending on the basis of the benchmark question can broaden the scope and content of the generated questions, and can further improve the comprehensiveness and accuracy of the generated questions, thereby improving the quality of question generation under text-type data sources. Especially in the field of intelligent self-service, by extending the questions, the quality and effect of human-computer interaction can be improved.

[0155] In an embodiment of the present invention, a question is generated based on a semantic diagram to obtain a benchmark question, and the benchmark question is extended to obtain an extended question. The extended question, the benchmark question and the transformed question are then used as questions corresponding to the question data source. On the basis of accurate generation based on the benchmark question, questions at a deeper level can be further generated by extension based on the benchmark question, thereby obtaining questions at different levels corresponding to the question data source, thereby achieving more comprehensive and accurate question generation, further improving the quality of question generation under text-type data sources, and further improving the intelligent effects in different intelligent scenarios corresponding to, including but not limited to, automatic generation of test questions in the education field, intelligent tutoring systems helping students understand materials, question-and-answer systems generating candidate questions, search engines generating related question suggestions, and human-computer interaction robots conducting human-computer dialogues.

[0156] In one embodiment, when the question data source is not a preset text type data source, determining a corresponding target preset question generation method to generate the question includes:

[0157] Identify the data source type corresponding to the problematic data source;

[0158] According to the data source type, determine the corresponding target preset question generation method to generate questions.

[0159] As described above, relative to the preset text type data source, a preset structured type data source, a preset image type data source, and a preset dialogue type data source can also be set, and for the preset structured type data source, a corresponding preset structured question generation method is set; for the preset image type data source, a corresponding preset image question generation method is set; for the preset dialogue type data source, a corresponding preset dialogue question generation method is set, so as to automatically adapt the question generation method according to different data source types, and adopt appropriate question generation methods for different question data sources, which can improve the accuracy of generating corresponding questions for different types of question data sources.

[0160] According to the above settings, when the problem data source is not a preset text type data source, the data source type corresponding to the problem data source is identified, and according to the data source type, the corresponding target preset question generation method is determined to generate the question.

[0161] Further, see Figure 2 , based on the data source type, determine the corresponding target preset question generation method, including at least one of the following:

[0162] In the case where the data source type is a structured data source type, determining the preset structured question generation method as the target preset question generation method;

[0163] In the case where the data source type is an image data source type, determining the preset image question generation method as the target preset question generation method;

[0164] When the data source type is a dialogue data source type, the preset dialogue question generation method is determined as the target preset question generation method.

[0165] Specifically, when the data source type is a structured data source type, the preset structured question generation method is determined as the target preset question generation method. Exemplarily, when the question data source is a knowledge graph triple, a database table, or key-value pair data, the question generation method corresponding to the template filling method, the graph neural network GNN method, or SQL-to-Question can be determined as the target preset question generation method. When the data source type is an image data source type, the preset image question generation method is determined as the target preset question generation method. Exemplarily, when the question data source is an image, a picture, a video, or an infrared heat map, the question generation method corresponding to visual question answering (VQA) reverse engineering, scene graph driven generation, or video timing question generation can be determined as the target preset question generation method, wherein the visual question answering (VQA) reverse engineering question generation method can be to generate questions by analyzing image features using models such as BLIP and OFA; the scene graph driven question generation method can be to generate a scene graph for detecting objects and relationships in the image and converting the scene graph into questions; the video timing question generation method can be to extract video features using 3D-CNN and generate timing-related questions.

[0166] When the data source type is a conversation data source type, the preset conversation question generation method is determined to be the target preset question generation method. For example, when the question data source is a customer service conversation record, an educational counseling conversation, or a social media discussion thread, the question generation method corresponding to context-aware generation or intent-entity joint modeling can be determined to be the target preset question generation method. The context-aware question generation method can be to generate follow-up questions based on the conversation history, and the intent-entity joint modeling can be to identify user intent, extract entities, and then generate questions. For example, identifying user intent such as asking about symptoms and extracting entities such as headaches can generate the question "Have you experienced nausea and headaches?"

[0167] According to the above settings, the question generation method is automatically adapted according to different data source types, and the appropriate question generation method is adopted for different question data sources, which can improve the accuracy of generating corresponding questions for different types of question data sources.

[0168] In an embodiment of the present invention, when the question data source is not a preset text type data source, the data source type corresponding to the question data source is identified, and according to the data source type, a corresponding target preset question generation method is determined to generate questions, so that question generation can automatically adapt to question generation according to different data source types, and adopt appropriate question generation methods for different question data sources, which can further improve the accuracy and comprehensiveness of corresponding questions generated by different types of question data sources, thereby comprehensively improving the quality of question generation, and further improving the intelligent effects in different intelligent scenarios corresponding to automatic generation of test questions in the education field, including but not limited to, intelligent tutoring systems helping students understand materials, question-answering systems generating candidate questions, search engines generating related question suggestions, and human-computer interaction robots conducting human-computer dialogues.

[0169] It should be noted that the question generation method based on natural language processing described in the above embodiments can recombine the technical features contained in different embodiments as needed to obtain a combined implementation plan, but they are all within the scope of protection required by the present invention.

[0170] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0171] The software tools, components, and models not produced by our company that appear in the embodiments of the present invention are for illustrative purposes only and do not represent actual use.

[0172] The relevant data collection in the embodiments of the present invention complies with the requirements of relevant laws and regulations, such as China's Personal Information Protection Law, GDPR (EU General Data Protection Regulation) or information security standards of other countries and regions.

[0173] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A question generation method based on natural language processing, characterized in that: include: In response to the question generation instruction, determining the question data source corresponding to the question generation; Determine whether the question data source is a preset text type data source, and if so, perform natural language processing on the question data source to obtain natural language features; Otherwise, determine the corresponding target preset question generation method and generate the question; Determining whether the natural language feature meets a preset semantic image element condition; If so, determining a number of target semantic graph elements corresponding to the natural language features; All the target semantic diagram elements include: target scene base diagram elements, target character entity diagram elements, target dynamic element diagram elements, target atmosphere rendering diagram elements and target perspective control diagram elements; Generating a semantic graph corresponding to the problem data source according to all the target semantic graph elements and based on a preset semantic graph generation model, including: Generate a model based on a preset picture, and perform scene base modeling according to the target scene base diagram elements to obtain a target scene base visual expression; Based on the target scene substrate and according to the target character entity diagram elements, target character entity modeling is performed to obtain a target character entity visual expression; Based on the target scene base and according to the target dynamic element diagram elements, target dynamic element modeling is performed to obtain a target motion trajectory visual expression corresponding to the target dynamic element; Based on the target scene base and according to the target atmosphere rendering diagram elements, target atmosphere rendering modeling is performed to obtain a target atmosphere visual expression; Based on the target scene substrate and according to the target perspective control diagram elements, visual control modeling is performed to obtain a target lens angle visual expression; Based on the semantics corresponding to the problem data source, the target scene base visual expression, the target character entity visual expression, the target motion trajectory visual expression, the target atmosphere visual expression, and the target lens angle visual expression are visualized and logically expressed to obtain a semantic diagram corresponding to the problem data source; Generate a semantic graph corresponding to the problem data source according to all the target semantic graph elements and based on a preset semantic graph generation model; According to the semantic graph and based on a preset graph question generation method, questions are generated to obtain questions corresponding to the question data source.

2. The method for generating questions based on natural language processing according to claim 1, wherein: The problem data source is subjected to natural language processing to obtain natural language features, including: Segmenting the problem data source to obtain a number of tokens; Performing part-of-speech tagging on the word-unit to obtain a tagged word-unit; Perform named entity recognition based on the annotated word to obtain a target entity; Based on a preset semantic recognition model, semantic recognition is performed on the problem data source to obtain target semantics; The marked word, the target entity, and the target semantics are used as natural language features corresponding to the question data source.

3. The method for generating questions based on natural language processing according to claim 1, wherein: Before generating a question according to the semantic graph and based on a preset graph question generation method and obtaining the question corresponding to the question data source, the method further includes: Based on the preset CLIP graphic verification model, determine whether the semantic diagram is similar to the semantics of the problem data source. If similar, execute the step of generating questions according to the semantic diagram and based on the preset diagram question generation method to obtain the question corresponding to the problem data source.

4. The method for generating questions based on natural language processing according to claim 3, wherein: According to the semantic graph and based on a preset graph question generation method, questions are generated to obtain questions corresponding to the question data source, including: Performing image preprocessing on the semantic graph to obtain a preprocessed graph; Extracting visual features corresponding to the preprocessed image based on a preset visual feature extraction method to obtain a plurality of visual feature texts; According to the pre-processed image and the visual feature text, and based on a preset visual language model, different visual feature texts are associated to obtain a plurality of visual text contents; According to the visual text content and based on a preset question generation model, questions corresponding to the visual text content are generated, and questions corresponding to the question data source are obtained.

5. The method for generating questions based on natural language processing according to claim 3, wherein: According to the semantic graph and based on a preset graph question generation method, questions are generated to obtain questions corresponding to the question data source, including: Generating questions according to the semantic graph and a preset graph question generation method to obtain a benchmark question; Perform named entity recognition on the benchmark question to obtain a question entity; Identify the entity type to which the problem entity belongs; Determining a corresponding preset replacement entity set according to the entity type, the preset replacement entity set including preset replacement entities; According to the preset replacement entity set and based on a preset replacement entity transformation method, the problem entity is transformed into the corresponding replacement entity to obtain a transformed problem; The benchmark problem and the transformed problem are used as the problems corresponding to the problem data source.

6. The method for generating questions based on natural language processing according to claim 5, wherein: After identifying the entity type to which the problem entity belongs, the method further includes: Determine, based on the entity type, a preset correspondence between a preset entity corresponding to the problem entity and a preset knowledge graph; According to the preset correspondence, the question entity is associated with a corresponding preset knowledge graph, wherein the preset knowledge graph includes a preset entity extended question chain, and the preset entity extended question chain includes a plurality of preset extended question templates; Associating the question entity with the preset extended question template to obtain an extended question; The extended question is used as the question corresponding to the question data source.

7. The method for generating questions based on natural language processing according to claim 3, wherein: The method further comprises: In the case that the question data source is not a preset text type data source, a corresponding target preset question generation method is determined to perform question generation.

8. The question generation method based on natural language processing according to claim 7, characterized in that: In the case where the question data source is not a preset text type data source, determining a corresponding target preset question generation method and performing question generation includes: Identify the data source type corresponding to the problematic data source; According to the data source type, a corresponding target preset question generation method is determined to generate questions.

9. The method for generating questions based on natural language processing according to claim 8, wherein: Determine a corresponding target preset question generation method based on the data source type, including at least one of the following: In the case where the data source type is a structured data source type, determining a preset structured question generation method as a target preset question generation method; In a case where the data source type is an image data source type, determining a preset image question generation method as a target preset question generation method; In a case where the data source type is a dialogue data source type, the preset dialogue question generation method is determined as the target preset question generation method.

Images