Question generation method based on natural language processing

Abstract

The invention discloses a problem generation method based on natural language processing, belongs to the technical field of natural language processing, and aims to solve the problem of low problem generation quality in the field of natural language processing in the traditional technology. And judging whether the problem data source is a preset text type data source, if so, carrying out natural language processing on the problem data source to obtain natural language features, and generating a corresponding semantic diagram according to all target semantic diagram elements under the condition that a plurality of target semantic diagram elements corresponding to the natural language features can be determined. According to the method, a semantic diagram is obtained, question generation is carried out according to the semantic diagram, a question corresponding to a question data source is obtained, the question data source is converted from a text mode to a graphical mode, an abstract text expression form is refined into a visual and simple graphical form, the generated question can focus on the main content expressed by the question data source, and the question generation quality is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular, to a question generation method based on natural language processing. Background Art

[0002] Question Generation (QG) refers to the process of automatically creating relevant questions from a given data source, and is an important task in the field of Natural Language Processing (NLP). With the development of artificial intelligence, different intelligent scenarios corresponding to the needs of automatically generating test questions in the education field, the needs of intelligent tutoring systems to help students understand materials, the needs of question-and-answer systems to generate candidate questions, the needs of search engines to generate relevant question suggestions, and the needs of human-computer interaction robots to maintain the fluency of human-computer conversations are booming. Question generation is becoming more and more important, and the quality of generated questions directly affects the intelligent effects of the above different intelligent scenarios. In traditional technologies, question generation is usually performed through natural language processing and based on rules, machine learning, or deep learning. Among them, rule-based question generation generally performs word segmentation through text preprocessing, and performs question template matching on the tokens obtained by word segmentation to generate corresponding questions. Machine learning-based question generation generally generates questions based on tokens and by using statistical methods or traditional machine learning such as SVM and random forests. Deep learning-based question generation generally generates corresponding questions based on tokens and by including, but not limited to, LSTM models and Transformer models. As can be seen from the above: traditional technologies all directly generate corresponding questions according to tokens, but due to the abstractness of tokens, it is difficult to generate accurate and comprehensive high-quality questions, thereby affecting the intelligent effects of the above different intelligent scenarios.

[0003] Therefore, how to improve the quality of question generation has become an urgent problem to be solved in the field of natural language processing. Summary of the Invention

[0004] Aiming at the problems existing in the prior art, the present invention provides a question generation method based on natural language processing, which converts the question data source from text modality to graphic modality and then generates questions to improve the quality of question generation.

[0005] To achieve the above technical purpose, the present invention adopts the following technical solutions: A question generation method based on natural language processing, comprising: In response to a question generation instruction, determining a question data source corresponding to question generation; Judging whether the question data source is a preset text type data source. If so, performing natural language processing on the question data source to obtain natural language features; otherwise, determining a corresponding target preset question generation method and performing question generation; Determine whether the natural language feature satisfies the preset semantic diagram element condition; if so, determine a plurality of target semantic diagram elements corresponding to the natural language feature; According to all the target semantic diagram elements and based on a preset semantic diagram generation model, generate a semantic diagram corresponding to the problem data source; According to the semantic diagram and based on a preset diagram question generation method, perform question generation to obtain questions corresponding to the problem data source.

[0006] Further, perform natural language processing on the problem data source to obtain natural language features, including: Segment the problem data source to obtain a plurality of tokens; Perform part-of-speech tagging on the tokens to obtain tagged tokens; According to the tagged tokens, perform named entity recognition to obtain target entities; Based on a preset semantic recognition model, perform semantic recognition on the problem data source to obtain target semantics; Use the tagged tokens, the target entities, and the target semantics as the natural language features corresponding to the problem data source.

[0007] Further, all the target semantic diagram elements include: target scene base diagram elements, target role entity diagram elements, target dynamic element diagram elements, target atmosphere rendering diagram elements, and target perspective control diagram elements; According to all the target semantic diagram elements and based on a preset semantic diagram generation model, generate a semantic diagram corresponding to the problem data source, including: Based on a preset drawing generation model and according to the target scene base diagram elements, perform scene base modeling to obtain a visual expression of the target scene base; Based on the target scene base and according to the target role entity diagram elements, perform target role entity modeling to obtain a visual expression of the target role entity; Based on the target scene base and according to the target dynamic element diagram elements, perform target dynamic element modeling to obtain a visual expression of the target motion trajectory corresponding to the target dynamic element; Based on the target scene base and according to the target atmosphere rendering diagram elements, perform target atmosphere rendering modeling to obtain a visual expression of the target atmosphere; Based on the target scene base and according to the target perspective control diagram elements, perform visual control modeling to obtain a visual expression of the target camera angle; Visualize the logical expression of the visual representation of the target scene base, the visual representation of the target character entity, the visual representation of the target motion trajectory, the visual representation of the target atmosphere, and the visual representation of the target camera angle based on the semantics corresponding to the problem data source, and obtain the semantic diagram corresponding to the problem data source.

[0008] Further, before generating a problem based on the semantic diagram and based on a preset diagram problem generation method to obtain the problem corresponding to the problem data source, it further includes: Based on a preset CLIP image-text verification model, determine whether the semantics of the semantic diagram are similar to the semantics of the problem data source. If they are similar, perform the step of generating a problem based on the semantic diagram and based on a preset diagram problem generation method to obtain the problem corresponding to the problem data source.

[0009] Further, generating a problem based on the semantic diagram and based on a preset diagram problem generation method to obtain the problem corresponding to the problem data source includes: Perform image preprocessing on the semantic diagram to obtain a preprocessed diagram; Extract the visual features corresponding to the preprocessed diagram based on a preset visual feature extraction method to obtain a number of visual feature texts; Based on the preprocessed diagram and the visual feature texts, and based on a preset visual language model, associate different visual feature texts to obtain a number of visual text contents; Based on the visual text contents and based on a preset problem generation model, generate the problems corresponding to the visual text contents to obtain the problems corresponding to the problem data source.

[0010] Further, generating a problem based on the semantic diagram and based on a preset diagram problem generation method to obtain the problem corresponding to the problem data source includes: Generate a reference problem based on the semantic diagram and based on a preset diagram problem generation method; Perform named entity recognition on the reference problem to obtain problem entities; Identify the entity type to which the problem entities belong; Based on the entity type, determine a corresponding preset replacement entity set, and the preset replacement entity set contains preset replacement entities; Based on the preset replacement entity set and based on a preset replacement entity transformation method, transform the problem entities into the corresponding replacement entities to obtain transformed problems; Use the reference problem and the transformed problems as the problems corresponding to the problem data source.

[0011] Further, after identifying the entity type to which the problem entity belongs, it further includes: Determine a preset correspondence between the preset entity corresponding to the problem entity and a preset knowledge graph according to the entity type; Associate the problem entity with the corresponding preset knowledge graph according to the preset correspondence, where the preset knowledge graph includes a preset entity extended problem chain, and the preset entity extended problem chain includes a number of preset extended problem templates; Associate the problem entity with the preset extended problem template to obtain an extended problem; Use the extended problem as the problem corresponding to the problem data source.

[0012] Further, the method further includes: In the case where the problem data source is not a preset text type data source, determine a corresponding target preset problem generation method to generate problems.

[0013] Further, in the case where the problem data source is not a preset text type data source, determining a corresponding target preset problem generation method to generate problems includes: Identify the data source type corresponding to the problem data source; Determine a corresponding target preset problem generation method according to the data source type to generate problems.

[0014] Further, determining a corresponding target preset problem generation method according to the data source type includes at least one of the following: In the case where the data source type is a structured data source type, determine the preset structured problem generation method as the target preset problem generation method; In the case where the data source type is an image data source type, determine the preset image problem generation method as the target preset problem generation method; In the case where the data source type is a dialogue data source type, determine the preset dialogue problem generation method as the target preset problem generation method.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The problem generation method based on natural language processing of the present invention determines the problem data source corresponding to problem generation, and determines whether the problem data source is a preset text type data source. If the above determination is yes, natural language processing is performed on the problem data source to obtain natural language features. Then, in the case where several target semantic diagram elements corresponding to the natural language features can be determined, according to all the target semantic diagram elements, a semantic diagram corresponding to the problem data source is generated. Then, according to the semantic diagram, problem generation is performed to obtain the problem corresponding to the problem data source. Thus, in the case where the problem data source is a preset text type data source, the problem data source is converted from the text modality to the diagram modality, so as to refine the abstract and complex text expression form into an image-intuitive and simple concrete form, and the main, important semantic content and core idea expressed by the problem data source can be focused through the diagram modality. Therefore, the generated problem can also focus on the main, important semantic content and core idea expressed by the problem data source, improving the problem generation quality under the text type data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic flowchart of the problem generation method based on natural language processing of the present invention; Figure 2 is a schematic overall concept diagram of the problem generation method based on natural language processing of the present invention; Figure 3 is a schematic flowchart of natural language feature acquisition in the present invention; Figure 4 is a schematic flowchart of semantic diagram generation in the present invention; Figure 5 is a schematic flowchart of generating problems according to the generated semantic diagram in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given in conjunction with the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments.

[0018] The embodiment of the present invention provides a problem generation method based on natural language processing. This method can be applied to devices corresponding to smartphones, tablets, learning machines, desktop computers, intelligent terminals, servers, cloud platforms, etc., and is used in scenarios including but not limited to automatically generating test questions in the education field, intelligent tutoring systems to help students understand materials, question answering systems to generate candidate questions, search engines to generate relevant question suggestions, and human-computer interaction robots for human-computer conversations.

[0019] In the face of the technical problem of low quality of question generation in the field of natural language processing in traditional technologies, the present invention provides a question generation method based on natural language processing: when the question data source for question generation is text, first generate a semantic diagram, and then generate questions based on the semantic diagram. Thus, when the question data source for question generation is text, the abstract text is semantically diagrammed, and comprehensive and accurate questions from different perspectives are generated through an intuitive and vivid visual semantic diagram, which can improve the quality of question generation for text type data sources.

[0020] Please refer to Figure 1 and Figure 2 , the question generation method based on natural language processing provided by the embodiments of the present invention includes the following steps: S11. In response to a question generation instruction, determine the question data source corresponding to the question generation: When starting question generation, generate a question generation instruction. In response to the question generation instruction, determine the question data source corresponding to the question generation. The question data source represents the data source of the question to be generated, that is, the type of content based on which the question is generated. The question data source includes but is not limited to text data sources, structured data sources, image data sources, or dialogue data sources. Among them, the text data source represents a data source with text including but not limited to text paragraphs and sentences as the question source, and the text data source includes but is not limited to textbook paragraphs, news articles, document texts, and encyclopedia content; the structured data source represents a data source with structured data as the question source, and the structured data source includes but is not limited to knowledge graph triples, database tables, and key-value pair data; the image data source represents a data source with images as the question source, and the image data source includes but is not limited to images and videos; the dialogue data source represents a data source with dialogues as the question source, and the dialogue data source includes but is not limited to customer service dialogue records, educational tutoring dialogues, and social media discussion threads.

[0021] S12. Determine whether the question data source is a preset text type data source: The preset text type data source represents a question data source of a text type that can convert text into a semantic diagram in a painting form as preset. The text forms corresponding to the preset text type data source include but are not limited to data of text types such as text paragraphs and sentences as the question source, and the text types corresponding to the preset text type data source include but are not limited to narrative text types such as novels, folk stories, and historical records, metaphor / symbol text types such as poems, fables, and philosophical texts, and multi-character interaction text types such as drama scripts and meeting records. For the preset text type data source, it can also be set to include but not limited to preset structured type data sources, preset image type data sources, and preset dialogue type data sources.

[0022] According to the above settings, determine whether the problem data source is a preset text type data source, that is, determine whether the problem data source is a data source that uses text type data as the problem source.

[0023] S13. In the case where the problem data source is not a preset text type data source, determine the corresponding target preset question generation method to generate questions; S14. In the case where the problem data source is a preset text type data source, perform natural language processing on the problem data source to obtain natural language features.

[0024] Explanatorily, in addition to the preset text type data source, for other types of data sources, as described above, including but not limited to the preset structured type data source, the preset image type data source, and the preset dialogue type data source, set the corresponding preset question generation methods. Exemplarily, for the preset structured type data source, set the corresponding preset structured question generation method; for the preset image type data source, set the corresponding preset image question generation method; for the preset dialogue type data source, set the corresponding preset dialogue question generation method, so as to automatically adapt the question generation method according to different data source types, realize the adoption of corresponding question generation methods for different problem data sources, and improve the accuracy of generating corresponding questions for different types of problem data sources.

[0025] According to the above settings, in the case where the problem data source is not a preset text type data source, determine the corresponding question generation method as the target preset question generation method according to the data source type to which the problem data source belongs to generate questions, and in the case where the problem data source is a preset text type data source, perform natural language processing on the problem data source to obtain natural language features. The natural language features represent the text features corresponding to the problem data source, and the natural language features include but not limited to named entities and semantic features. Among them, natural language processing (Natural Language Processing, NLP) mainly studies how to enable a computer to understand, process, generate, and simulate human language capabilities. Natural language processing includes but not limited to word segmentation, part-of-speech tagging, named entity recognition, and semantic understanding.

[0026] S15. Determine whether the natural language features meet the preset semantic diagram element conditions: Pre-set semantic diagram element conditions, that is, pre-set semantic diagram element conditions. The pre-set semantic diagram element conditions represent the conditions corresponding to the picture elements required for converting text into a painting form and for drawing pictures in the pre-set semantic diagram. The pre-set semantic diagram element conditions include, but are not limited to, whether the natural language features express the drawing elements required for drawing pictures, including but not limited to the scene base, character entities, dynamic elements, atmosphere rendering, or perspective control. That is, a picture is composed of corresponding drawing elements. Here, it is necessary to first judge whether the natural language features describe the drawing elements necessary for constructing a painting or more drawing elements, so as to be able to construct the semantic diagram of the painting form corresponding to the text.

[0027] Among them, the scene base represents the physical or abstract environmental framework that supports the narrative, including spatial structure, era background, and cultural context. For example, when the scene base is "spatial prepositional phrase - in the forest", the spatial prepositional phrase - in the forest corresponding to the semantic expression, and the corresponding visual representation can be the background texture - the tree density represents the spatial depth. It can be judged whether the natural language features meet the pre-set semantic diagram element conditions of this dimension of the scene base by judging whether the natural language features contain expressions corresponding to, including but not limited to, spatial prepositional phrases.

[0028] Character entities represent intelligent or non-intelligent agents with independent behavior logic and emotional attributes. For example, when the character entity is "description of a person - a girl in a red dress", the description of a person - a girl in a red dress corresponding to the semantic expression, and the corresponding visual representation can be "concretized character design + restoration of clothing color". It can be judged whether the natural language features meet the pre-set semantic diagram element conditions of this dimension of the character entity by judging whether the natural language features contain expressions corresponding to, including but not limited to, descriptions of people.

[0029] Dynamic elements represent the change process in the time and space dimensions, including physical movement and state migration. For example, when the dynamic element is "action verb - running / falling", the action verb - running / falling corresponding to the semantic expression, and the corresponding visual representation can be "motion line / blurred afterimage / split frame". It can be judged whether the natural language features meet the pre-set semantic diagram element conditions of this dimension of the dynamic element by judging whether the natural language features contain expressions corresponding to, including but not limited to, action verbs.

[0030] Atmosphere rendering represents the emotional field and stylized expression conveyed by visual elements. For example, when the atmosphere rendering is "rhetorical device - metaphor / hyperbole", the rhetorical device corresponding to the semantic expression - metaphor / hyperbole, the corresponding visual representation can be "integration of surreal elements such as a broken heart → glass cracks. It can be determined whether the natural language features meet the preset semantic diagram element conditions of this dimension of atmosphere rendering by judging whether the natural language features include expressions corresponding to rhetorical devices, including but not limited to.

[0031] Viewpoint control represents the selective presentation strategy of visual information, including physical viewpoints and narrative viewpoints. For example, when the viewpoint control is "personal pronoun - I / he", the personal pronoun corresponding to the semantic expression - I / he, the corresponding visual representation can be "camera angle - first person → subjective viewpoint". It can be determined whether the natural language features meet the preset semantic diagram element conditions of this dimension of viewpoint control by judging whether the natural language features include expressions corresponding to personal pronouns, including but not limited to.

[0032] Based on the above settings, it is determined whether the natural language features meet the preset semantic diagram element conditions. For example, it is judged whether the natural language features include the relevant content expressions of the drawing elements corresponding to the scene base, character entity, dynamic elements, atmosphere rendering, or viewpoint control.

[0033] S16. In the case where the natural language features do not meet the preset semantic diagram element conditions, several target semantic diagram elements corresponding to the natural language features cannot be determined; S17. In the case where the natural language features meet the preset semantic diagram element conditions, several target semantic diagram elements corresponding to the natural language features are determined.

[0034] Explanatorily, in the case where the natural language features do not meet the conditions of the preset semantic view elements, it indicates that the drawing elements required for the semantic view corresponding to converting the text into a painting form are not met. For example, as described above, in the case of judging whether the natural language features meet the conditions of the preset semantic view elements in the dimension of the scene base by judging whether the natural language features contain the expressions corresponding to the spatial prepositional phrases, when it is determined that the natural language features do not contain the expressions corresponding to the spatial prepositional phrases, it is determined that the natural language features do not meet the conditions of the preset semantic view elements in the dimension of the scene base, indicating that the natural language features do not contain the painting element of the scene base required for painting, and the text corresponding to the problem data source cannot be converted into the semantic view corresponding to the painting form. And so on for others. Thus, several target semantic view elements corresponding to the natural language features are uncertain. For the case where the text corresponding to the problem data source cannot be converted into the semantic view corresponding to the painting form, a method for generating questions based on the text can be additionally set to generate corresponding questions. For example, methods for generating questions based on text that include but are not limited to template-based methods and rule-based methods can be adopted, and existing corresponding technical means can be borrowed, which will not be elaborated here.

[0035] Similarly, in the case where the natural language features meet the conditions of the preset semantic view elements, it indicates that the natural language features contain the corresponding painting elements required for painting, and the text corresponding to the problem data source can be converted into the semantic view corresponding to the painting form. In this case, several target semantic view elements corresponding to the natural language features are specifically determined. For example, the specific information corresponding to the scene base is determined, the specific information corresponding to the role entity is determined, the specific information corresponding to the dynamic elements is determined, the specific information corresponding to the atmosphere rendering is determined, or the specific content of the drawing elements such as the specific information corresponding to the perspective control is determined.

[0036] S18. Generate the semantic view corresponding to the problem data source according to all the target semantic view elements and based on the preset semantic view generation model: A semantic view generation model is preset in advance, that is, the preset semantic view generation model. The preset semantic view generation model represents a model that draws the corresponding view according to all the target semantic view elements included in the semantics corresponding to the text. That is, the preset semantic view generation model is a processing model that organizes all the target semantic view elements into a painting to construct a visual picture logical expression. For example, a model that generates a picture or an image corresponding to the text. For example, a model that uses a picture or an image corresponding to a scene graph or a scenario graph to express the semantics, content, and core idea of the text. The preset semantic view generation model includes but is not limited to the generative adversarial network GAN and the autoregressive model.

[0037] Based on the above settings, according to all target semantic diagram elements, and based on a preset semantic diagram generation model, a semantic diagram corresponding to the problem data source is generated, that is, corresponding pictures or images are generated from the text. Among them, the semantic diagram represents the semantic content expressed by the text corresponding to the problem data source in the form of a diagram. That is, the semantic diagram represents diagrams including but not limited to scenes or scenarios corresponding to the sun, people, plants, animals, rivers, roads, or parks that express the corresponding semantic content. The semantic diagram is not a graph structure including nodes and edges. The semantic diagram includes but not limited to pictures, images, and three-dimensional diagrams. Thus, the problem data source is refined and undergoes a modality transformation, converting the problem data source from an abstract text modality to a vivid and intuitive diagram modality. Thus, the text is refined through the diagram to reduce the noise and interference in the text expression corresponding to the problem data source, focusing on the main, important semantic content and core idea expressed by the problem data source. Furthermore, the generated questions can also focus on the main, important semantic content and core idea expressed by the problem data source.

[0038] S19. According to the semantic diagram, and based on a preset diagram question generation method, question generation is performed to obtain the questions corresponding to the problem data source: The diagram question generation method is preset in advance, that is, the preset diagram question generation method. The preset diagram question generation method represents a method for generating questions according to diagrams including but not limited to pictures and images. The preset diagram question generation method includes but not limited to methods for generating questions using pre-trained models, structured visual understanding, etc. Among them, using pre-trained models includes but not limited to calling existing APIs, etc., to generate questions according to the diagram, and structured visual understanding includes but not limited to a method for generating questions according to the diagram by combining visual element extraction and template filling. The above preset diagram question generation method can be directly used by existing technical means and will not be elaborated here.

[0039] Based on the above settings, according to the semantic diagram, and using methods including but not limited to using pre-trained models, structured visual understanding, etc., question generation is performed to obtain the questions corresponding to the problem data source. Thus, question generation is carried out based on the transformation between different modalities of text - diagram - text, obtaining the questions corresponding to the problem data source, realizing the refinement of the abstract and complex text expression form into a vivid, intuitive, and concise concrete form, focusing on the main, important semantic content and core idea expressed by the problem data source through the diagram modality. Furthermore, according to the semantic diagram, question generation is performed. Thus, the generated questions can also focus on the main, important semantic content and core idea expressed by the problem data source, thereby improving the quality of question generation for text-type data sources.

[0040] In an embodiment of the present invention, in response to a question generation instruction, a question data source corresponding to the question generation is determined, and it is judged whether the question data source is a preset text type data source. If the above judgment is yes, natural language processing is performed on the question data source to obtain natural language features. When the natural language features corresponding to several target semantic diagram elements can be determined, and based on a preset semantic diagram generation model, a semantic diagram corresponding to the question data source is generated. Then, according to the semantic diagram, question generation is performed to obtain the question corresponding to the question data source. When the question data source is not a preset text type data source, a corresponding target preset question generation method is determined to perform question generation. Thus, when the question data source is a preset text type data source, the question data source is converted from a text modality to a diagram modality and then back to the text modality, realizing the refinement of the question data source based on natural language processing and modality conversion, so as to refine the abstract and complex text expression form into an intuitive, vivid, simple and concrete form. It is possible to focus on the main, important semantic content and core idea expressed by the question data source through the diagram modality. Furthermore, according to the semantic diagram, question generation is performed, and the generated question can also focus on the main, important semantic content and core idea expressed by the question data source, thereby obtaining the question corresponding to the question data source. Comprehensive and accurate questions from different perspectives can be generated through the intuitive and vivid refined semantic diagram, thus improving the quality of question generation under text type data sources. When the question data source for question generation is of non-text type, a suitable question generation method is adopted. Therefore, corresponding question generation methods are adopted for different question data sources, which can improve the accuracy and comprehensiveness of generating corresponding questions for each type of question data source, thus comprehensively improving the quality of question generation. Furthermore, the intelligent question generation effect in different intelligent scenarios including but not limited to automatically generating test questions in the education field, intelligent tutoring systems helping students understand materials, question-and-answer systems generating candidate questions, search engines generating relevant question suggestions, and human-machine interaction robots conducting human-machine conversations can be improved.

[0041] In one embodiment, please refer to Figure 3 , performing natural language processing on the question data source to obtain natural language features, including: S31. Segment the question data source to obtain a number of word tokens; S32. Perform part-of-speech tagging on the word tokens to obtain tagged word tokens; S33. According to the tagged word tokens, perform named entity recognition to obtain target entities; S34. Based on a preset semantic recognition model, perform semantic recognition on the question data source to obtain target semantics; S35. Take the tagged word tokens, target entities, and target semantics as the natural language features corresponding to the question data source.

[0042] Explanatorily, a semantic recognition model is preset, that is, a preset semantic recognition model, which represents a model for semantic recognition. The preset semantic recognition model includes, but is not limited to, Word2Vec model, BERT-like models, Generative Pre-trained Transformer (GPT), and Large Language Model (LLM).

[0043] According to the above settings, the question data source is tokenized to split the sentence into words or subwords. It can be tokenized by, but not limited to, using WordPiece of BERT to obtain a number of tokens, and then the tokens are part-of-speech tagged, for example, to identify nouns, verbs, etc., to obtain tagged tokens. Then, based on the tagged tokens, named entity recognition is performed. Named Entity Recognition (NER) is to identify and classify entities with specific meanings from the text, such as identifying entities like people, places, times, etc., to obtain target entities. And based on the preset semantic recognition model, the question data source is semantically recognized to obtain target semantics. Semantic recognition includes, but is not limited to, lexical semantics, syntactic semantics, contextual semantics, intention semantics, and emotional semantics. Lexical semantics represents the meaning of a word itself, syntactic semantics represents the meaning conveyed by the sentence structure, contextual semantics represents the meaning determined by the context, intention semantics represents the purpose of the speaker, and emotional semantics represents the emotional tendency of the text. Finally, the tagged tokens, target entities, and target semantics are used as the natural language features corresponding to the question data source, obtaining features from different perspectives of the question data source. Furthermore, the features from different perspectives are integrated to generate a semantic map, which can improve the consistency between the semantic map and the content expressed by the question data source, that is, improve the accuracy of semantic map generation, and further improve the accuracy of question generation, thereby improving the quality of question generation under text-type data sources.

[0044] In one embodiment, please refer to Figure 4 , all target semantic map elements include: target scene base map elements, target role entity map elements, target dynamic element map elements, target atmosphere rendering map elements, and target perspective control map elements; according to all target semantic map elements, and based on the preset semantic map generation model, a semantic map corresponding to the question data source is generated, including: S41. Based on the preset drawing generation model, and according to the target scene base map elements, perform scene base modeling to obtain the visual expression of the target scene base; S42. Based on the target scene base, and according to the target role entity map elements, perform target role entity modeling to obtain the visual expression of the target role entity; S43. Based on the target scenario base and according to the target dynamic element diagram elements, perform target dynamic element modeling to obtain the visual expression of the target motion trajectory corresponding to the target dynamic elements; S44. Based on the target scenario base and according to the target atmosphere rendering diagram elements, perform target atmosphere rendering modeling to obtain the target atmosphere visual expression; S45. Based on the target scenario base and according to the target perspective control diagram elements, perform visual control modeling to obtain the target camera angle visual expression; S46. Based on the semantics corresponding to the problem data source, perform visual drawing logical expression on the target scenario base visual expression, the target character entity visual expression, the target motion trajectory visual expression, the target atmosphere visual expression, and the target camera angle visual expression to obtain the semantic diagram corresponding to the problem data source.

[0045] In the present invention, a drawing generation model (Drawing / Picture Generation Models) is preset, that is, a preset drawing generation model. The preset drawing generation model represents a model for generating corresponding drawings according to the target semantic diagram elements. The preset drawing generation model includes, but is not limited to, the generative adversarial network GAN and the diffusion model Diffusion Models. It should be noted that the differences between a drawing and an image are as follows: 1) The semantic density of a drawing is low, while that of an image is high; 2) The structure of a drawing is simple and the key points are prominent, while the structure of an image is complex and the details are rich; 3) A drawing generally refers to graphics in styles such as digital painting and stick figures, while an image generally refers to illustrations with photo-level details.

[0046] Based on the preset drawing generation model and according to the target scenario base diagram elements, perform scenario base modeling to obtain the target scenario base visual expression, where the target scenario base diagram elements represent the target semantic diagram elements in the dimension of the scenario base. For example, when the target scenario base diagram elements are "spatial prepositional phrase - in the forest", the spatial prepositional phrase - in the forest corresponding to the semantic expression is converted into the corresponding visual representation "background texture - tree density represents spatial depth" through corresponding modeling.

[0047] Based on the target scenario base and according to the target character entity diagram elements, perform target character entity modeling to obtain the target character entity visual expression, where the target character entity diagram elements represent the target semantic diagram elements in the dimension of the character entity. For example, when the target character entity diagram elements are "description of a person - a girl in a red dress", the description of a person - a girl in a red dress is converted into the corresponding visual representation, which can be "concretized character design + restoration of clothing color" through corresponding modeling.

[0048] Based on the target scene base and according to the target dynamic element diagram elements, perform target dynamic element modeling to obtain the visual expression of the target motion trajectory corresponding to the target dynamic elements, where the target dynamic element diagram elements represent the target semantic diagram elements in the dimension of dynamic elements. For example, when the target dynamic element diagram elements are "action verb - running / falling", the action verb - running / falling is converted into the corresponding visual representation as "motion line / blurred afterimage / split frame" through the corresponding modeling.

[0049] Based on the target scene base and according to the target atmosphere rendering diagram elements, perform target atmosphere rendering modeling to obtain the target atmosphere visual expression, where the target atmosphere rendering diagram elements represent the target semantic diagram elements in the dimension of atmosphere rendering. For example, when the target atmosphere rendering diagram elements are "rhetorical devices - metaphor / hyperbole", the rhetorical devices - metaphor / hyperbole are converted into the corresponding visual representation as the fusion of surreal elements, such as a broken heart → glass cracks, through the corresponding modeling.

[0050] Based on the target scene base and according to the target perspective control diagram elements, perform visual control modeling to obtain the visual expression of the target camera angle, where the target perspective control diagram elements represent the target semantic diagram elements in the dimension of perspective control. For example, when the target perspective control diagram elements are "personal pronouns - I / he", the personal pronouns - I / he are converted into the corresponding visual representation as "camera angle - first person → subjective perspective" through the corresponding modeling.

[0051] Based on the content and logic corresponding to the semantics of the problem data source, organize the drawing of the corresponding content and logic of the target scene base visual expression, the target character entity visual expression, the target motion trajectory visual expression, the target atmosphere visual expression, and the target camera angle visual expression. The drawing organization includes determining the positions of the target scene base visual expression, the target character entity visual expression, the target motion trajectory visual expression, the target atmosphere visual expression, and the target camera angle visual expression and their relationships with each other, that is, realizing the logical expression of the visual drawing and obtaining the semantic diagram corresponding to the problem data source.

[0052] For example, based on a preset drawing generation model, a drawing corresponding to the problem data source can be sketched. Sketching usually refers to a concise expression of the main content. By adopting the way of sketching, a drawing corresponding to the natural language features is sketched to obtain a drawing corresponding to the problem data source, that is, a semantic diagram corresponding to the problem data source. It can make the drawing focus on the main, important semantic content and core idea expressed by the problem data source. Thus, the abstraction of the text is refined into the intuitiveness of the drawing in the form of the drawing, and the key and main content are grasped, realizing the transformation of the abstract logic into the concrete spatial relationship, and realizing an innovative problem generation method of cross-modal thinking integrating text and drawing. The problem generation method of the embodiment of the present invention upgrades the traditional text to a visual refinement enhancement form. Its core idea is to transform the linear text into a visual cognitive space through a visual bridge, refine the key and main content of the text, and then generate problems from a visual perspective, which can improve the accuracy and quality of problem generation.

[0053] In the embodiment of the present invention, by according to the natural language features and based on a preset drawing generation model, a drawing corresponding to the problem data source is generated. Thus, by virtue of the concise characteristics of the drawing, not only can the efficiency of problem generation be improved, but also the abstract and complex text expression form can be refined into an intuitive, vivid and simple concrete form. It can focus on the main, important semantic content and core idea expressed by the problem data source through the graphical modality corresponding to the drawing. Then, according to the semantic diagram, problems are generated. The generated problems can also focus on the main, important semantic content and core idea expressed by the problem data source, so as to obtain the problems corresponding to the problem data source, and accurate problems from different angles can be generated through the intuitive, vivid and simple refined drawing, thereby improving the quality of problem generation under the text type data source.

[0054] In one embodiment, before generating the problems corresponding to the problem data source according to the semantic diagram and based on a preset diagram problem generation method, it further includes: Based on a preset CLIP text-image verification model, it is judged whether the semantics of the semantic diagram are similar to those of the problem data source. In the case of semantic similarity, the step of generating the problems corresponding to the problem data source according to the semantic diagram and based on a preset diagram problem generation method is executed; otherwise, an error is reported or the semantic diagram is regenerated.

[0055] Explanatorily, a CLIP image-text verification model is preset, that is, a CLIP image-text verification model is preset. The preset CLIP image-text verification model represents a model for verifying the consistency between the generated semantic illustration and the semantic text corresponding to the problem data source based on the CLIP model (Contrastive Language-Image Pretraining). Among them, the CLIP model is a multimodal model. The CLIP model maps the semantic illustration and the semantic text corresponding to the problem data source to the same vector space through contrastive learning, realizes cross-modal semantic understanding between images and texts, and verifies the semantic consistency between the semantic illustration and the semantic text corresponding to the problem data source.

[0056] Based on the preset CLIP image-text verification model, it is judged whether the semantics of the semantic illustration and the problem data source are similar, that is, a consistency judgment is made on the semantics corresponding to the content of the semantic illustration and the content of the problem data source. Since the CLIP model itself includes a text encoder such as Transformer and an image encoder such as ViT or ResNet, the CLIP model can map the semantic illustration and the semantic text of the problem data source to the same vector space through contrastive learning, so as to realize cross-modal semantic understanding between the semantic illustration and the semantic text of the problem data source, and compare the semantic illustration and the semantic text of the problem data source through a similarity such as cosine similarity to perform consistency verification between the semantic illustration and the semantic text of the problem data source. The similarity comparison is generally judged by comparing the calculated similarity value with the corresponding similarity threshold, which will not be elaborated here. Thus, when the semantics of the semantic illustration and the problem data source are similar, it indicates that the generated semantic illustration accurately visually expresses the content and its semantics of the problem data source. When the semantics of the semantic illustration and the problem data source are similar, perform the step of generating a problem according to the semantic illustration and based on the preset illustration problem generation method to generate the problem corresponding to the problem data source, so as to generate a problem that is more in line with the content and its semantics of the problem data source, that is, the generated problem has a higher quality; while when the semantics of the semantic illustration and the problem data source are not similar, it indicates that the generated semantic illustration does not accurately visually express the content and its semantics of the problem data source, and an error is reported or the semantic illustration is regenerated.

[0057] In the embodiment of the present invention, by performing consistency verification based on cross-modal semantic understanding between the semantics of the semantic illustration and the problem data source, it is ensured that there is a high similarity between the semantic illustration used for problem generation and the semantics of the problem data source, that is, there is consistency between the semantics of the semantic illustration and the problem data source, so as to ensure that the semantic illustration accurately expresses the semantics of the problem data source. On this basis, the generated problem can also have a high degree of fit with the semantics of the problem data source, thereby further improving the quality of problem generation under text-type data sources.

[0058] In one embodiment, refer to Figure 5 , and according to the semantic diagram and based on a preset diagram question generation method, question generation is performed to obtain the questions corresponding to the question data source, including: S51. Perform image preprocessing on the semantic diagram to obtain a preprocessed diagram; S52. Based on a preset visual feature extraction method, extract the visual features corresponding to the preprocessed diagram to obtain a number of visual feature texts; S53. According to the preprocessed diagram and the visual feature texts, and based on a preset vision-language model, associate different visual feature texts to obtain a number of visual text contents; S54. According to the visual text contents, and based on a preset question generation model, generate the questions corresponding to the visual text contents to obtain the questions corresponding to the question data source.

[0059] The preset visual feature extraction method refers to the method of extracting diagram features from a diagram. The preset visual feature extraction method includes, but is not limited to, object detection, for example: using the YOLOv8 model or the DETR model to obtain the object list and positions, and visual relationship detection includes traditional methods based on template matching, intelligent recognition methods based on machine learning, and deep learning algorithms, etc.

[0060] The vision-language model, that is, Vision-Language Models, abbreviated as VLMs, is a model that understands and generates language information related to visual content through cross-modal representation learning of vision and language. The preset vision-language model includes, but is not limited to, the CLIP model, the BLIP model (Bootstrapping Language-Image Pre-training). Among them, the vision-language model is a multi-modal model that can learn from images and texts, can accept image and text inputs and generate text outputs. The vision-language model (VisualLanguageModels) is a model that can learn from images and texts simultaneously to process many tasks. Therefore, based on the preset vision-language model, according to the preprocessed diagram and the visual feature texts, the preprocessed diagram and the visual feature texts can be combined, similar to describing a picture in words. Based on the preprocessed diagram and combined with the visual feature texts, the preprocessed diagram can be described from different angles, and the content contained in the preprocessed diagram can be learned comprehensively and accurately from the preprocessed diagram, and then comprehensive and accurate questions from different angles corresponding to the semantic diagram can be generated, thereby improving the quality of question generation under the text type data source.

[0061] The preset question generation model refers to a model that generates questions according to the corresponding text. The preset question generation model includes, but is not limited to, the Seq2Seq model and the Generative Pre-trained Transformer (GPT).

[0062] According to the above settings, first, perform image preprocessing on the semantic map. Image preprocessing refers to the optimization operation performed on the originally generated semantic map before subsequent core analysis or processing of the semantic map. For example, perform image preprocessing on the semantic map including, but not limited to, denoising, resolution adjustment, rotation, scaling, and cropping to obtain a preprocessed map; then extract the visual features corresponding to the preprocessed map. For example, perform object detection to obtain the object and its corresponding position, perform scene classification of the semantic map, or perform visual relationship detection to obtain several visual feature texts; then, based on the preprocessed map and the visual feature texts, and based on the preset visual language model, combine the preprocessed map with the visual feature texts, associate different visual feature texts, and obtain several visual text contents. The visual text contents describe the contents included in the semantic map, that is, what the semantic map contains, so as to achieve the conversion from the visual modality to the text modality; finally, based on the visual text contents and based on the preset question generation model, generate the questions corresponding to the visual text contents, obtain the questions corresponding to the question data source, thereby realizing the refinement conversion from the text modality of the question data source to the visual modality, then from the visual modality to the text modality, and perform question generation. Since the refinement conversion from the text modality of the question data source to the visual modality filters out the noise and interference corresponding to the abstract and complex descriptions of the text modality of the question data source, the questions generated according to the conversion from the visual modality to the text modality can focus on the main, important semantic content and core idea expressed by the question data source, and generate comprehensive and accurate questions.

[0063] In an embodiment of the present invention, by preprocessing the semantic diagram to obtain a preprocessed diagram, and extracting the visual features corresponding to the preprocessed diagram to obtain a number of visual feature texts, then according to the preprocessed diagram and the visual feature texts, and based on a preset visual language model, different visual feature texts are associated to obtain a number of visual text contents. Then, according to the visual text contents, and based on a preset question generation model, questions corresponding to the visual text contents are generated to obtain the questions corresponding to the question data source. Thus, when the question data source is a preset text type data source, the question data source is converted from the text modality to the diagram modality, realizing the refinement of the question data source based on natural language processing and the modality conversion, so as to refine the abstract and complex text expression form into an intuitive, vivid and concise concrete form. Then, based on the diagram modality, the corresponding text modality is obtained, realizing the sequential conversion between the text modality - diagram modality - text modality. And based on the above conversion process, the question data source is refined, thereby focusing on the main, important semantic content and core idea expressed by the question data source, reducing the noise and interference of the text expression corresponding to the question data source, and then generating corresponding questions. The generated questions can also focus on the main, important semantic content and core idea expressed by the question data source, so as to obtain the questions corresponding to the question data source, and can generate comprehensive and accurate questions from different angles corresponding to the semantic diagram through the intuitive and vivid refined semantic diagram, thereby improving the quality of question generation under the text type data source.

[0064] In one embodiment, according to the semantic diagram, and based on a preset diagram question generation method, questions are generated to obtain the questions corresponding to the question data source, including: According to the semantic diagram, and based on a preset diagram question generation method, questions are generated to obtain benchmark questions; Perform named entity recognition on the benchmark questions to obtain question entities; Identify the entity type to which the question entity belongs; According to the entity type, determine the corresponding preset replacement entity set, and the preset replacement entity set includes preset replacement entities; According to the preset replacement entity set, and based on a preset replacement entity transformation method, transform the question entity into the corresponding replacement entity to obtain transformed questions; Take the benchmark questions and the transformed questions as the questions corresponding to the question data source.

[0065] The preset replacement entity set represents the entity set for replacing the question entity, and the preset replacement entity set includes preset replacement entities. The preset replacement entities include, but are not limited to, the replacement entities corresponding to the synonymous replacement, near-synonymous replacement, antonymous replacement, upper or lower replacement, and same-kind replacement of the question entity, where the same-kind replacement represents the replacement of the same type.

[0066] The preset replacement entity transformation method represents the method of transforming a problem entity. The preset replacement entity transformation method includes, but is not limited to, synonym replacement, near-synonym replacement, antonym replacement, upper or lower-level replacement, same-category replacement, and several replacement methods corresponding to interrogative-word replacement. Among them, interrogative-word replacement means changing the focus of the question. For example, "When will it be released?" is replaced by "Who released it?" or "Where was it released?". And for different types of problem entities, corresponding preset replacement entity transformation methods are set. For example, for the problem entity corresponding to a noun, methods including but not limited to same-category replacement, upper or lower-level replacement are carried out. For the problem entity corresponding to an adjective, methods including but not limited to synonym replacement, near-synonym replacement, antonym replacement are carried out.

[0067] With the above settings, based on the semantic diagram and the preset diagram question generation method, question generation is carried out to obtain a reference question. The reference question not only represents the question directly generated according to the semantic diagram but also represents the question serving as the basis for further question generation.

[0068] Named entity recognition is performed on the reference question to obtain a problem entity. The problem entity represents the entity corresponding to the reference question. The "problem" involved in the problem entity is only used to distinguish different entities and is not used to define different entities. Named entity recognition is as described above and will not be elaborated here; and the entity type to which the problem entity belongs is identified. The entity type represents the category of the entity. The entity type includes, but is not limited to, the entity types corresponding to nouns and verbs. For example, the problem entity belongs to the verb type or the problem entity belongs to the noun type.

[0069] According to the entity type, determine the corresponding preset replacement entity set. The preset replacement entity set contains preset replacement entities. For different entity types, corresponding preset replacement entity sets are preset. For example, for entity types of nouns, corresponding preset replacement entity sets are set, and for entity types of verbs, corresponding preset replacement entity sets are set. Then, according to the preset replacement entity set and based on the replacement methods corresponding to the preset replacement entity transformation methods, including but not limited to synonym replacement, near-synonym replacement, antonym replacement, upper or lower position replacement, similar category replacement, and interrogative word replacement, transform the problem entity into the corresponding replacement entity to obtain a transformed problem. The several replacement entities corresponding to transforming the problem entity into the corresponding replacement entity are subsets of the preset replacement entity set. Then, use the benchmark problem and the transformed problem as the problems corresponding to the problem data source, so as to, on the basis of the benchmark problem, transform the benchmark problem to obtain a transformed problem, realizing the expansion of the benchmark problem, where the transformed problem includes the same or similar problems and opposite problems. And using the benchmark problem and the transformed problem as the problems corresponding to the problem data source can, on the basis of focusing on the main, important semantic content and core idea expressed by the problem data source through the semantic diagram, expand the dimensions, scope, and content of the problems corresponding to the problem data source, thereby further improving the comprehensiveness and accuracy of the generated problems, and thus improving the quality of problem generation under the text type data source. Especially in the field of intelligent self-service, by transforming the problem, the interaction quality and efficiency of human-computer interaction can be improved.

[0070] Exemplarily, for the generated benchmark problem "What is the little boy doing in the park", the synonym transformation is "What is the little boy doing", the near-synonym transformation is "Can you describe the activities of the little boy in the park", the angle conversion is "Who is doing what in the park", the antonym conversion is "What is the little girl doing in the park", etc. According to different benchmark problems, perform at least one of the above transformations, so as to expand the scope and content of the problem, and further improve the comprehensiveness and accuracy of the generated problems, thereby improving the quality of problem generation under the text type data source.

[0071] In the embodiments of the present invention, by generating problems according to the semantic diagram to obtain a benchmark problem, transforming the benchmark problem to obtain a transformed problem, and then using the benchmark problem and the transformed problem as the problems corresponding to the problem data source, it is possible to, on the basis of accurately generating the benchmark problem, and through the transformation based on the benchmark problem, generate comprehensive and accurate problems from different angles, thereby improving the quality of problem generation under the text type data source, and further improving the intelligent effects in different intelligent scenarios, including but not limited to automatically generating test questions in the education field, intelligent tutoring systems helping students understand materials, question-and-answer systems generating candidate questions, search engines generating relevant question suggestions, and human-computer interaction robots conducting human-computer conversations.

[0072] In one embodiment, after identifying the entity type to which the problem entity belongs, the following steps are further included: Determine the preset correspondence between the preset entity corresponding to the problem entity and the preset knowledge graph according to the entity type; Associate the problem entity with the corresponding preset knowledge graph according to the preset correspondence, where the preset knowledge graph includes a preset entity extension problem chain, and the preset entity extension problem chain includes a number of preset extension problem templates; Associate the problem entity with the preset extension problem template to obtain an extended problem; Use the extended problem as the problem corresponding to the problem data source.

[0073] Explanatorily, the preset correspondence between the preset entity and the preset knowledge graph is set in advance, that is, the preset correspondence between the preset entity and the preset knowledge graph. The preset knowledge graph represents the knowledge graph preset and associated with the preset entity. The preset knowledge graph includes the graphical expression of concepts, attributes, content related to the preset entity and the mutual relationships with other entities in the corresponding business domain or application scenario. For example, the family relationships, school relationships, and friend relationships of "Student A" can be organized into the corresponding knowledge graph. And for different types of entities, for several preset entities of each type, the corresponding preset knowledge graphs are preset in advance, that is, the corresponding relationship of preset entity type - preset entity - preset knowledge graph is formed. For example, preset entity type A - preset entity 1 - preset knowledge graph 1; preset entity type A - preset entity 2 - preset knowledge graph 2; preset entity type B - preset entity 1 - preset knowledge graph 1, and so on for others.

[0074] The preset knowledge graph contains a preset entity extension problem chain, and the preset entity extension problem chain contains a number of preset extension problem templates. Among them, the preset entity extension problem chain represents a problem chain in which a number of preset extension problem templates starting from the preset entity are organized in a chain form. The preset extension problem template represents a template for preset extension problems, that is, in the corresponding business scenario or application scenario of the preset entity, which problems will be involved, and a general problem framework is generated in the form of a problem template in advance. In the corresponding business scenario or application scenario, the preset entity extension problem chain and the number of preset extension problem templates it contains need to be set by relevant personnel. Exemplarily, for the above example, for "Student A", two extension problem chains can be set starting from "Student A". Among them, one extension problem chain involves the corresponding problem templates of his family relationship, and the other extension problem chain involves the corresponding problem templates of his school relationship. For example, the extension problem chain involving his family relationship and its corresponding problem templates can be "Are Student A's parents in the countryside or in the city?" and "Do Student A's parents work or do business in the city?". In the corresponding specific business scenario or application scenario, the above technical concept can be adopted to organize a number of problem templates into an extension problem chain in the form of a chain. Moreover, the preset entity extension problem chain can include, but is not limited to, a vertical extension problem chain and a horizontal extension problem chain. The vertical extension problem chain represents a chain composed of a number of problems that progress layer by layer and expand deeply in a vertical direction. The horizontal extension problem chain represents a chain composed of a number of problems that expand in a horizontal breadth. For example, for the above example, "Are Student A's parents in the countryside or in the city?" and "Do Student A's parents work or do business in the city?", this form of chain composed of a number of problems that progress layer by layer and expand deeply in a vertical direction is a vertical extension problem chain. If it expands from "Are Student A's parents in the countryside or in the city?" to "Are Student A's friends in the countryside or in the city?", it is a horizontal extension problem chain composed of a number of problems that expand in a horizontal breadth. The above example is only used to illustrate the technical concept and core idea of the present technical solution, and is not used to limit the present technical solution, and so on for others.

[0075] Based on the above settings, according to the entity type, determine the preset correspondence between the preset entity corresponding to the problem entity and the preset knowledge graph. Among the several pairs of preset correspondences corresponding to the entity type, it is possible to determine the preset entity corresponding to the problem entity and its corresponding preset knowledge graph by judging whether the problem entity is the same as or similar to the preset entity; then, according to the preset knowledge graph corresponding to the preset correspondence, associate the problem entity with the corresponding preset knowledge graph. The preset knowledge graph contains a preset entity extended problem chain, and the preset entity extended problem chain contains several preset extended problem templates. Then, associate the problem entity with the preset extended problem template by means including but not limited to filling to obtain specific extended problems. Finally, use the extended problems as the problems corresponding to the problem data source, thereby extending the benchmark problem longitudinally or horizontally to obtain extended problems, and using the extended problems as the problems corresponding to the problem data source. The extended problems are problems that are extended and deepened on the basis of the benchmark problem along its original problem direction or problem logic, and are a process of enriching and improving the scope, depth, or details of the benchmark problem. The extension process pays more attention to the in-depth excavation and expansion of the original problem, aiming to provide more comprehensive and in-depth problems. On the basis of focusing on the main, important semantic content and core idea expressed by the problem data source, further expand the depth, level, scope, and content of the problems corresponding to the problem data source, thereby further improving the comprehensiveness and accuracy of the generated problems, and thus improving the quality of problem generation under the text type data source.

[0076] Exemplarily, for the generated benchmark problem "What is the little boy doing in the park", extending it according to the corresponding logic, the extended problems can be "Who is the little boy in the park with", "How did the little boy come to the park", etc. Extending on the basis of the benchmark problem can broaden the scope and content of the generated problems, further improve the comprehensiveness and accuracy of the generated problems, and thus improve the quality of problem generation under the text type data source. Especially in the field of intelligent self-service, through extended problems, the interaction quality and effect of human-computer interaction can be improved.

[0077] In an embodiment of the present invention, by generating questions according to a semantic diagram, a benchmark question is obtained, and the benchmark question is extended to obtain an extended question. Then, the extended question, the benchmark question, and the transformed question are used as the questions corresponding to the question data source, which can be based on the accurate generation of the benchmark question and further generate questions at a deeper level through the extension based on the benchmark question, so as to obtain questions at different levels corresponding to the question data source, thereby achieving more comprehensive and accurate question generation, further improving the quality of question generation for text type data sources, and further improving the intelligent effects in different intelligent scenarios including but not limited to automatically generating test questions in the education field, intelligent tutoring systems to help students understand materials, question-and-answer systems to generate candidate questions, search engines to generate relevant question suggestions, and human-computer interaction robots to conduct human-computer conversations.

[0078] In one embodiment, when the question data source is not a preset text type data source, a corresponding target preset question generation method is determined to generate questions, including: Identifying the data source type corresponding to the question data source; According to the data source type, determining a corresponding target preset question generation method to generate questions.

[0079] As described above, relative to the preset text type data source, it is also possible to set, including but not limited to, a preset structured type data source, a preset image type data source, and a preset dialogue type data source. Moreover, for the preset structured type data source, a corresponding preset structured question generation method is set; for the preset image type data source, a corresponding preset image question generation method is set; for the preset dialogue type data source, a corresponding preset dialogue question generation method is set, so as to automatically adapt the question generation method according to different data source types, implement an appropriate question generation method for different question data sources, and improve the accuracy of generating corresponding questions for different types of question data sources.

[0080] According to the above settings, when the question data source is not a preset text type data source, the data source type corresponding to the question data source is identified, and according to the data source type, a corresponding target preset question generation method is determined to generate questions.

[0081] Further, referring to Figure 2 , determining a corresponding target preset question generation method according to the data source type includes at least one of the following: When the data source type is a structured data source type, determining the preset structured question generation method as the target preset question generation method; When the data source type is an image data source type, determining the preset image question generation method as the target preset question generation method; When the data source type is the dialogue data source type, determine the preset dialogue question generation method as the target preset question generation method.

[0082] Specifically, when the data source type is the structured data source type, determine the preset structured question generation method as the target preset question generation method. Exemplarily, when the question data source is a knowledge graph triple, a database table, or key-value pair data, the question generation methods corresponding to template filling, graph neural network (GNN) method, or SQL-to-Question can be determined as the target preset question generation method. When the data source type is the image data source type, determine the preset image question generation method as the target preset question generation method. Exemplarily, when the question data source is an image, a drawing, a video, or an infrared thermal image, the question generation methods corresponding to visual question answering (VQA) reverse engineering, scene graph-driven generation, or video temporal question generation can be determined as the target preset question generation method. Among them, the VQA reverse engineering question generation method can be to analyze image features using models such as BLIP and OFA to generate questions; the scene graph-driven question generation method can be to detect objects and relationships in the image to generate a scene graph and convert the scene graph into a question; the video temporal question generation method can be to use 3D-CNN to extract video features and generate temporally related questions.

[0083] When the data source type is the dialogue data source type, determine the preset dialogue question generation method as the target preset question generation method. Exemplarily, when the question data source is a customer service dialogue record, an educational tutoring dialogue, or a social media discussion thread, the question generation methods corresponding to context-aware generation or intent-entity joint modeling can be determined as the target preset question generation method. Among them, the context-aware question generation method can be to generate follow-up questions based on the dialogue history, and intent-entity joint modeling can be to identify the user's intent, extract entities, and then generate questions. For example, identify the user's intent such as asking about symptoms and extract entities such as headache to generate the question "Have you experienced nausea and headache?".

[0084] According to the above settings, the question generation method can be automatically adapted according to different data source types, realizing the adoption of an appropriate question generation method for different question data sources, and improving the accuracy of generating corresponding questions for different types of question data sources.

[0085] In the embodiments of the present invention, when the problem data source is not a preset text type data source, the data source type corresponding to the problem data source is identified, and according to the data source type, the corresponding target preset problem generation method is determined to generate problems. Thus, problem generation can automatically adapt to different data source types for problem generation, realizing the adoption of corresponding problem generation methods for different problem data sources, which can further improve the accuracy and comprehensiveness of generating corresponding problems for different types of problem data sources, thereby comprehensively improving the quality of problem generation, and further improving the intelligent effects in different intelligent scenarios including but not limited to automatically generating test questions in the education field, intelligent tutoring systems helping students understand materials, question and answer systems generating candidate questions, search engines generating relevant question suggestions, and human-machine interaction robots conducting human-machine conversations.

[0086] It should be noted that for the problem generation method based on natural language processing described in each of the above embodiments, the technical features included in different embodiments can be recombined as needed to obtain the combined implementation scheme, but all are within the protection scope required by the present invention.

[0087] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, abbreviated as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, abbreviated as EPROM), programmable read-only memory (Programmable Red-Only Memory, abbreviated as PROM), read-only memory (Read-Only Memory, abbreviated as ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements in the process Figure 1 a process or multiple processes and / or blocks Figure 1The functions specified in one or more boxes.

[0088] In the embodiments of the present invention, the software tools, components, or models of non-our company are only introduced by way of example and do not represent actual use.

[0089] The relevant data collection in the embodiments of the present invention complies with the requirements of relevant laws and regulations, such as the Personal Information Protection Law of China, GDPR (General Data Protection Regulation of the European Union), or the information security standards of other countries and regions.

[0090] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A question generation method based on natural language processing, characterized in that, Including: In response to a question generation instruction, determining a question data source corresponding to the question generation; Judging whether the question data source is a preset text type data source, and if so, performing natural language processing on the question data source to obtain natural language features; Otherwise, determining a corresponding target preset question generation method to perform question generation; Judging whether the natural language features meet the preset semantic diagram element conditions; if so, determining a plurality of target semantic diagram elements corresponding to the natural language features; According to all the target semantic diagram elements and based on a preset semantic diagram generation model, generating a semantic diagram corresponding to the question data source; According to the semantic diagram and based on a preset diagram question generation method, performing question generation to obtain a question corresponding to the question data source.

2. The problem generation method based on natural language processing according to claim 1, wherein Performing natural language processing on the question data source to obtain natural language features, including: Performing word segmentation on the question data source to obtain a plurality of word elements; Performing part-of-speech tagging on the word elements to obtain tagged word elements; According to the tagged word elements, performing named entity recognition to obtain target entities; Based on a preset semantic recognition model, performing semantic recognition on the question data source to obtain a target semantics; Taking the tagged word elements, the target entities, and the target semantics as the natural language features corresponding to the question data source.

3. The problem generation method based on natural language processing according to claim 1, characterized in that, All the target semantic diagram elements include: a target scene base diagram element, a target role entity diagram element, a target dynamic element diagram element, a target atmosphere rendering diagram element, and a target perspective control diagram element; According to all the target semantic diagram elements and based on a preset semantic diagram generation model, generating a semantic diagram corresponding to the question data source, including: Based on a preset drawing generation model and according to the target scene base diagram element, performing scene base modeling to obtain a target scene base visual expression; Based on the target scene base and according to the target role entity diagram element, performing target role entity modeling to obtain a target role entity visual expression; Based on the target scene base and according to the target dynamic element diagram element, performing target dynamic element modeling to obtain a target motion trajectory visual expression corresponding to the target dynamic element; Based on the target scene base and according to the target atmosphere rendering diagram element, performing target atmosphere rendering modeling to obtain a target atmosphere visual expression; Based on the target scene base and according to the target perspective control diagram element, performing visual control modeling to obtain a target camera angle visual expression; Based on the semantics corresponding to the question data source, performing visual drawing logical expression on the target scene base visual expression, the target role entity visual expression, the target motion trajectory visual expression, the target atmosphere visual expression, and the target camera angle visual expression to obtain a semantic diagram corresponding to the question data source.

4. A method for generating questions based on natural language processing according to any one of claims 1-3, characterized in that, Before performing question generation according to the semantic diagram and based on a preset diagram question generation method to obtain a question corresponding to the question data source, further including: Based on a preset CLIP image-text verification model, determine whether the semantic diagram is similar to the semantics of the problem data source. If they are similar, perform the step of generating a problem based on the semantic diagram and a preset diagram problem generation method to obtain the problem corresponding to the problem data source.

5. The problem generation method based on natural language processing according to claim 4, characterized in that, Generating a problem based on the semantic diagram and a preset diagram problem generation method to obtain the problem corresponding to the problem data source includes: Performing image preprocessing on the semantic diagram to obtain a preprocessed diagram; Extracting the visual features corresponding to the preprocessed diagram based on a preset visual feature extraction method to obtain a number of visual feature texts; Associating different visual feature texts based on the preprocessed diagram and the visual feature texts and a preset vision-language model to obtain a number of visual text contents; Generating the problem corresponding to the visual text content based on the visual text content and a preset problem generation model to obtain the problem corresponding to the problem data source.

6. The problem generation method based on natural language processing according to claim 4, wherein Generating a problem based on the semantic diagram and a preset diagram problem generation method to obtain the problem corresponding to the problem data source includes: Generating a baseline problem based on the semantic diagram and a preset diagram problem generation method; Performing named entity recognition on the baseline problem to obtain problem entities; Identifying the entity type to which the problem entities belong; Determining a corresponding preset replacement entity set according to the entity type, where the preset replacement entity set contains preset replacement entities; Transforming the problem entities into the corresponding replacement entities based on the preset replacement entity set and a preset replacement entity transformation method to obtain transformed problems; Regarding the baseline problem and the transformed problems as the problems corresponding to the problem data source.

7. The problem generation method based on natural language processing according to claim 6, characterized in that After identifying the entity type to which the problem entities belong, it further includes: Determining a preset correspondence between the preset entity corresponding to the problem entity and a preset knowledge graph according to the entity type; Associating the problem entities with the corresponding preset knowledge graph according to the preset correspondence, where the preset knowledge graph contains a preset entity extension problem chain, and the preset entity extension problem chain contains a number of preset extension problem templates; Associating the problem entities with the preset extension problem templates to obtain extension problems; Regarding the extension problems as the problems corresponding to the problem data source.

8. The problem generation method based on natural language processing according to claim 4, wherein The method further includes: When the problem data source is not a preset text type data source, determining a corresponding target preset problem generation method to generate problems.

9. The problem generation method based on natural language processing according to claim 8, characterized in that, When the problem data source is not a preset text type data source, determining a corresponding target preset problem generation method to generate problems includes: Identifying the data source type corresponding to the problem data source; Determining a corresponding target preset problem generation method according to the data source type to generate problems.

10. A method for generating questions based on natural language processing according to claim 9, wherein Determining a corresponding target preset problem generation method according to the data source type includes at least one of the following: In the case where the data source type is a structured data source type, determine the preset structured question generation method as the target preset question generation method; In the case where the data source type is an image data source type, determine the preset image question generation method as the target preset question generation method; In the case where the data source type is a dialogue data source type, determine the preset dialogue question generation method as the target preset question generation method.

Images