Method, device, electronic device and storage medium for generating a topic explanation video

Abstract

The present application discloses a method, device, electronic device and storage medium for generating a topic explanation video, which relates to the technical field of computer graphics and can flexibly generate explanation videos for different types of topics automatically without setting different explanation templates for different types of topics in advance. The method includes: in response to a topic explanation request, determining a target topic to be explained; based on a pre-constructed knowledge graph template, querying the topic type to which the target topic belongs, obtaining an original matrix and multiple preset animation steps matching the topic type; using the original matrix to generate a corresponding animation matrix for each preset animation step, and integrating multiple animation matrices corresponding to multiple preset animation steps to obtain an animation demonstration video; generating a corresponding animation explanation audio for each preset animation step, and integrating the animation explanation audio corresponding to each preset animation step with the animation demonstration video to obtain a topic explanation video for the target topic.

Description

Technical Field

[0001] The present application relates to the technical field of computer graphics, and in particular to a method, device, electronic device and storage medium for generating a topic explanation video. Background Art

[0002] In the mathematics education system, calculation problems, as a key link in the students' learning process, have always been regarded as the focus and difficulty of teaching. Learning calculation problems not only helps students to deeply understand the basic concepts of mathematics, such as number recognition and four arithmetic operations, but also effectively improves their calculation speed and accuracy, further exercises their attention and persistence, and lays a solid foundation for students' subsequent mathematics learning and even all-round development. In recent years, the teaching of calculation problems is no longer limited to solving problems, but pays more attention to guiding students to understand the essence of the problems, master problem-solving strategies, and form the ability to think independently and solve problems.

[0003] In the related technology, considering that the teacher's explanation in class cannot meet the learning needs of all students in time, and the thinking logic of calculation problems cannot be intuitively understood through teaching materials or textbooks after class, there are currently some automatic explanation tools, which try to solve the difficulties of students in learning calculation problems through technical means. However, the applicant realizes that the automatic explanation tool can only generate explanation content for fixed types of questions with animation generation templates, and the coverage rate of question types is not high. In addition, some tools provide pure text analysis, which is not only difficult to meet the diverse needs of students, but also has a single explanation format, lacks vividness and interactivity, resulting in the problem-solving process of complex calculation problems becoming obscure and difficult for students to obtain effective learning support. Summary of the invention

[0004] In view of this, the present application provides a method, device, electronic device and storage medium for generating a question explanation video. The main purpose is to solve the problem that the current question type coverage is not high and some tools provide pure text analysis, which not only fails to meet the diverse needs of students, but also has a single explanation format and lacks vividness and interactivity, resulting in the problem-solving process of complex calculation problems becoming obscure and difficult for students to obtain effective learning support.

[0005] According to the first aspect of the present application, a method for generating a topic explanation video is provided, the method comprising:

[0006] In response to the topic explanation request, determining a target topic to be explained;

[0007] Based on the pre-built knowledge graph template, query the question type to which the target question belongs, and obtain an original matrix and multiple preset animation steps matching the question type, wherein the original matrix is ​​used to indicate the number of rows and columns occupied by the calculation results of the questions belonging to the question type in the animation graphic space;

[0008] Using the original matrix to generate a corresponding animation matrix for each of the preset animation steps, and integrating multiple animation matrices corresponding to the multiple preset animation steps to obtain an animation demonstration video;

[0009] Generate a corresponding animation explanation audio for each of the preset animation steps, and integrate the animation explanation audio corresponding to each of the preset animation steps with the animation demonstration video to obtain a question explanation video for the target question.

[0010] Optionally, in response to the topic explanation request, determining a target topic to be explained includes:

[0011] Receiving the question explanation request, and obtaining the question formula according to the question input method indicated by the question explanation request;

[0012] Input the question formula into the question determination model, and obtain the determination result output by the question determination model for the question formula;

[0013] When the judgment result indicates that the question formula is a calculation question, taking the question formula as the target question;

[0014] Wherein, when the judgment result indicates that the question formula is a non-calculation question, a re-input instruction is pushed to the terminal that initiated the question explanation request, so that the user holding the terminal re-initiates the question explanation request.

[0015] Optionally, obtaining the question formula according to the question input mode indicated by the question explanation request includes:

[0016] Identify the topic input method indicated by the topic explanation request;

[0017] When the question input mode indicates a photograph input, obtaining a question picture collected by a terminal held by a user who initiates the question explanation request, extracting a question text from the question picture, and converting the question text into the question formula;

[0018] When the question input mode indicates voice input, obtaining the voice content input by the user who initiated the question explanation request, recognizing the voice content as text information, and converting the text information into the question formula;

[0019] When the input method indicates gesture input, the image content currently captured by the terminal held by the user who initiated the question explanation request is obtained, the target area pointed by the user's finger is selected in the image content, the text content of the target area is identified, and the text content is converted into the question formula.

[0020] Optionally, querying the topic type to which the target topic belongs based on a pre-built knowledge graph template, and obtaining an original matrix and a plurality of preset animation steps matching the topic type include:

[0021] Extracting topic features from the target topic, wherein the topic features include one or more of operation symbols, variable information, constant information, and function information;

[0022] Matching the topic feature with a plurality of nodes included in the knowledge graph template, and searching for a target node matching the topic feature among the plurality of nodes;

[0023] Taking the topic type identified by the target node as the topic type to which the target topic belongs, and obtaining the original matrix associated with the topic type;

[0024] Obtain at least one candidate problem-solving path associated with the target node in the knowledge graph template, and determine a candidate problem-solving path that matches the question feature from the at least one candidate problem-solving path as a target problem-solving path;

[0025] Reading the path information of the target problem-solving path to obtain the plurality of preset animation steps;

[0026] When the target node matching the topic feature is not found among the multiple nodes, an abnormal feedback recipient is determined, an abnormal reminder carrying the target topic is generated, and the abnormal reminder is pushed to the abnormal feedback recipient.

[0027] Optionally, the step of using the original matrix to generate a corresponding animation matrix for each preset animation step includes:

[0028] For each of the preset animation steps, read the matrix position information, animation supplementary information, and animation behavior information associated with the preset animation step in the knowledge graph template;

[0029] Extracting at least one topic character matching the matrix position information from the target topic, acquiring a plurality of predefined reference characters, and determining a corresponding target reference character for each of the topic characters in the plurality of reference characters;

[0030] Referring to the matrix position information, using the target reference character corresponding to each of the question characters, assigning values ​​to the matrix digits at the corresponding positions in the original matrix to obtain an intermediate matrix;

[0031] Adjusting the intermediate matrix according to the animation supplementary information to obtain the adjusted intermediate matrix;

[0032] The adjusted intermediate matrix is ​​rendered according to the animation behavior information to obtain an animation matrix corresponding to the preset animation step.

[0033] Optionally, generating a corresponding animation explanation audio for each of the preset animation steps, and integrating the animation explanation audio corresponding to each of the preset animation steps with the animation demonstration video to obtain a topic explanation video of the target topic, includes:

[0034] Extracting step detail information from each of the preset animation steps, and extracting topic detail information from the animation matrix corresponding to each of the preset animation steps;

[0035] Integrate the step detail information and the title detail information of each of the preset animation steps to form step introduction information of each of the preset animation steps;

[0036] Performing voice conversion on the step introduction information of each of the preset animation steps to obtain an initial explanation audio corresponding to each of the preset animation steps, and adjusting the initial explanation audio corresponding to each of the preset animation steps according to preset voice parameters to obtain an animation explanation audio corresponding to each of the preset animation steps;

[0037] The animation demonstration video is segmented to determine the video segment corresponding to each of the preset animation steps, and the video segment corresponding to each of the preset animation steps is aligned with the corresponding animation explanation audio on the time axis, and the playback speed of each of the video segments is adjusted according to the preset playback speed to obtain an initial explanation video;

[0038] Determine the elements to be marked in the initial explanation video, mark the elements to be marked in the initial explanation video with preset key marks, and embed preset interactive elements in the initial explanation video to obtain the topic explanation video of the target topic.

[0039] Optionally, the method further comprises:

[0040] Acquire a plurality of preset step types, segment the topic explanation video according to the plurality of step types to obtain a plurality of step segments, determine a matching step segment for each of the step types in the plurality of step segments and establish an association relationship, store the associated plurality of step segments in a database, and display prompt terms related to each of the step types when providing explanations for the target topic, and when any prompt term is detected to be triggered, play the step segment associated with the step type corresponding to the triggered prompt term; and / or,

[0041] Collect and cache the opinion information of user feedback, and push at least one opinion information currently cached to the abnormal feedback recipient every preset time period, so that the abnormal feedback recipient can refer to at least one opinion information currently cached to update the knowledge graph template.

[0042] According to a second aspect of the present application, a device for generating a topic explanation video is provided, the device comprising:

[0043] A determination module, for determining a target topic to be explained in response to a topic explanation request;

[0044] A query module, used to query the type of question to which the target question belongs based on a pre-built knowledge graph template, and obtain an original matrix and a plurality of preset animation steps matching the type of question, wherein the original matrix is ​​used to indicate the number of rows and columns occupied by the calculation results of the question belonging to the type of question in the animation graphic space;

[0045] A generating module, configured to generate a corresponding animation matrix for each of the preset animation steps using the original matrix, and to integrate multiple animation matrices corresponding to the multiple preset animation steps to obtain an animation demonstration video;

[0046] The integration module is used to generate a corresponding animation explanation audio for each of the preset animation steps, and integrate the animation explanation audio corresponding to each of the preset animation steps with the animation demonstration video to obtain a question explanation video for the target question.

[0047] Optionally, the determination module is used to receive the question explanation request, obtain the question formula according to the question input method indicated by the question explanation request; input the question formula into the question judgment model, and obtain the judgment result output by the question judgment model for the question formula; when the judgment result indicates that the question formula is a calculation question, use the question formula as the target question; wherein, when the judgment result indicates that the question formula is a non-calculation question, push a re-input instruction to the terminal that initiated the question explanation request, so that the user holding the terminal re-initiates the question explanation request.

[0048] Optionally, the determination module is used to identify the question input method indicated by the question explanation request; when the question input method indicates photo input, obtain the question picture collected by the user who initiates the question explanation request based on the terminal held by the user, extract the question text from the question picture, and convert the question text into the question formula; when the question input method indicates voice input, obtain the voice content input by the user who initiates the question explanation request, recognize the voice content as text information, and convert the text information into the question formula; when the input method indicates gesture input, obtain the image content currently captured by the terminal held by the user who initiates the question explanation request, select the target area pointed by the user's finger in the image content, identify the text content of the target area, and convert the text content into the question formula.

[0049] Optionally, the query module is used to extract question features from the target question, wherein the question features include one or more of operation symbols, variable information, constant information, and function information; match the question features with multiple nodes included in the knowledge graph template, and query the target node matching the question features among the multiple nodes; use the question type identified by the target node as the question type to which the target question belongs, and obtain the original matrix associated with the question type; obtain at least one candidate solution path associated with the target node in the knowledge graph template, and determine the candidate solution path matching the question features among the at least one candidate solution path as the target solution path; read the path information of the target solution path to obtain the multiple preset animation steps; wherein, when the target node matching the question features is not found among the multiple nodes, determine an abnormal feedback recipient, generate an abnormal reminder carrying the target question, and push the abnormal reminder to the abnormal feedback recipient.

[0050] Optionally, the generation module is used to read the matrix position information, animation supplementary information and animation behavior information associated with each of the preset animation steps in the knowledge graph template; extract at least one title character matching the matrix position information in the target title, and obtain a plurality of predefined reference characters, and determine a corresponding target reference character for each of the title characters in the plurality of reference characters; refer to the matrix position information, use the target reference character corresponding to each of the title characters, assign values ​​to the matrix digits at corresponding positions in the original matrix, and obtain an intermediate matrix; adjust the intermediate matrix according to the animation supplementary information to obtain the adjusted intermediate matrix; render the adjusted intermediate matrix according to the animation behavior information to obtain the animation matrix corresponding to the preset animation step.

[0051] Optionally, the integration module is used to extract step detail information from each of the preset animation steps, and to extract topic detail information from the animation matrix corresponding to each of the preset animation steps; integrate the step detail information and topic detail information of each of the preset animation steps to form step introduction information of each of the preset animation steps; perform voice conversion on the step introduction information of each of the preset animation steps to obtain an initial explanation audio corresponding to each of the preset animation steps, and adjust the initial explanation audio corresponding to each of the preset animation steps according to preset voice parameters to obtain an animation explanation audio corresponding to each of the preset animation steps; segment the animation demonstration video to determine a video clip corresponding to each of the preset animation steps, and align the video clip corresponding to each of the preset animation steps with its corresponding animation explanation audio on the timeline, and adjust the playback speed of each of the video clips according to a preset playback speed to obtain an initial explanation video; determine the elements to be marked in the initial explanation video, mark the elements to be marked in the initial explanation video using a preset emphasis mark, and embed a preset interactive element in the initial explanation video to obtain a topic explanation video of the target topic.

[0052] Optionally, the device further comprises:

[0053] A storage module is used to obtain a plurality of preset step types, segment the topic explanation video according to the plurality of step types to obtain a plurality of step segments, determine a matching step segment for each of the step types in the plurality of step segments and establish an association relationship, store the plurality of associated step segments in a database, and display a prompt entry related to each of the step types when providing explanations for the target topic, and when any prompt entry is detected to be triggered, play a step segment associated with the step type corresponding to the triggered prompt entry; and / or,

[0054] The feedback module is used to collect and cache the opinion information of user feedback, and push at least one opinion information currently cached to the abnormal feedback recipient every preset time period, so that the abnormal feedback recipient can refer to the at least one opinion information currently cached to update the knowledge graph template.

[0055] According to a third aspect of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0056] According to a fourth aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0057] By means of the above technical scheme, the present application provides a method, device, electronic device and storage medium for generating a question explanation video. The present application responds to a question explanation request, determines a target question to be explained, queries the question type to which the target question belongs based on a pre-built knowledge graph template, obtains an original matrix and multiple preset animation steps matching the question type, generates a corresponding animation matrix for each preset animation step using the original matrix, integrates multiple animation matrices corresponding to multiple preset animation steps to obtain an animation demonstration video, generates a corresponding animation explanation audio for each preset animation step, and integrates the animation explanation audio corresponding to each preset animation step with the animation demonstration video to obtain a question explanation video of the target question. By setting an original matrix that can accurately indicate the number of rows and columns occupied by calculation results belonging to different question types in the animation graphic space, a knowledge graph template with breadth and depth is combined with the original matrix, so that the automatic question explanation tool can flexibly generate explanation videos for different types of questions automatically, without having to set different explanation templates for different types of questions in advance. While improving the question type coverage and meeting the diverse needs of students, the students' learning process is made vivid and interactive, so that students can easily understand and master problem-solving skills when watching the animation demonstration video.

[0058] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0060] Figure 1A A schematic diagram of a process flow of a method for generating a topic explanation video provided in an embodiment of the present application is shown;

[0061] Figure 1B A schematic diagram showing a method for generating a topic explanation video provided in an embodiment of the present application is shown;

[0062] Figure 2A A schematic diagram of a flow chart of another method for generating a topic explanation video provided in an embodiment of the present application is shown;

[0063] Figure 2BA schematic diagram showing a method for generating a topic explanation video provided in an embodiment of the present application is shown;

[0064] Figure 2C A schematic diagram showing a method for generating a topic explanation video provided in an embodiment of the present application is shown;

[0065] Figure 2D A schematic diagram showing a method for generating a topic explanation video provided in an embodiment of the present application is shown;

[0066] Figure 2E A schematic diagram of a flow chart of a method for generating a topic explanation video provided in an embodiment of the present application is shown;

[0067] Figure 3 A schematic diagram of the structure of a device for generating a topic explanation video provided in an embodiment of the present application is shown;

[0068] Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0069] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0070] The present application embodiment provides a method for generating a topic explanation video, such as Figure 1A As shown, the method includes:

[0071] 101. In response to a topic explanation request, determine a target topic to be explained.

[0072] The technical solution of the embodiment of the present application can be applied to the automatic question explanation tool. The automatic question explanation tool is an intelligent learning auxiliary tool that integrates computer technology and educational concepts, and is intended to provide students with efficient and accurate question explanation services in an automated and intelligent manner. The automatic question explanation tool can be installed on different platforms, such as online education platforms, learning management systems, mobile learning applications, etc., so as to receive and process question explanation requests received by these platforms.

[0073] The topic explanation request is initiated by the user, and the user can initiate it in a variety of ways, such as clicking the "Explain the topic" button on the application interface, or requesting to explain a specific topic through voice input, or requesting to explain a specific topic by taking a photo, etc. When the platform connected to the automatic topic explanation tool receives the topic explanation request initiated by the user, the automatic topic explanation tool will determine the specific topic that the user wants to explain, and use the specific topic as the target topic to be explained, thereby accurately capturing user needs, providing users with personalized topic explanation services, and improving learning efficiency.

[0074] For example, assuming that the user inputs the question "33+44=?" by taking a photo, the automatic explanation tool will determine this question as the target question to be explained.

[0075] 102. Based on the pre-built knowledge graph template, query the question type to which the target question belongs, and obtain the original matrix and multiple preset animation steps that match the question type.

[0076] In an embodiment of the present application, the automatic question-explaining tool identifies the question type of the target question based on a pre-built knowledge graph template. The knowledge graph template is a complex network including multiple nodes constructed using various question types and their corresponding problem-solving steps. Using the knowledge graph template, the automatic question-explaining tool can quickly and accurately determine the question type to which the question belongs.

[0077] After determining the question type, the automatic explanation tool will search for the corresponding original matrix and preset animation steps in the knowledge graph template according to the question type. The original matrix is ​​a matrix used to indicate the number of rows and columns occupied by the question calculation result in the animation graphic space, which determines the basic framework of the animation demonstration. The preset animation steps are a series of animation steps pre-designed according to the question type to show the problem-solving process.

[0078] Continuing with the target question "33+44=?" as an example, through the knowledge graph template, the automatic question-teaching tool determines that the question type corresponding to "33+44=?" is "vertical addition", and the original matrix obtained can be Figure 1B In this way, the original matrix represents "33+44=?" The final calculation result will occupy 5 rows and 3 columns in the animation graphic space. In this way, using the knowledge graph template, the automatic explanation tool can automatically identify the question type and obtain the corresponding problem-solving resources. While improving the efficiency and accuracy of the problem-solving explanation, it can also ensure a higher coverage of the question type and meet the diverse learning needs of students.

[0079] 103. Generate a corresponding animation matrix for each preset animation step using the original matrix, and integrate multiple animation matrices corresponding to multiple preset animation steps to obtain an animation demonstration video.

[0080] In the embodiment of the present application, the automatic presentation tool generates a corresponding animation matrix for each preset animation step using the original matrix. The animation matrix is ​​an extension or deformation of the original matrix, and is used to display the calculation results and state changes of each step in the animation.

[0081] After generating the animation matrix, the automatic explanation tool will integrate multiple animation matrices corresponding to multiple preset animation steps, and form a complete animation demonstration video through frame-by-frame rendering and animation transition effects. The animation demonstration video shows the entire process from question input to the final answer, including the calculation and reasoning of each step. Through the animation demonstration video, the automatic explanation tool can show the problem-solving process in an intuitive and vivid way, helping students better understand and master problem-solving skills.

[0082] 104. Generate a corresponding animation explanation audio for each preset animation step, and integrate the animation explanation audio corresponding to each preset animation step with the animation demonstration video to obtain a question explanation video of the target question.

[0083] In an embodiment of the present application, the automatic explanation tool will generate corresponding animation explanation audio for each preset animation step. The explanation audio is a detailed explanation of each preset animation step, aiming to help students understand the reasoning and calculation process of each step.

[0084] After generating the audio, the automatic explanation tool will integrate the audio with the animation demonstration video, and form a complete question explanation video by playing the audio and video content synchronously. The question explanation video contains both the intuitive effect of the animation demonstration and the detailed audio content, which can provide students with all-round learning support and help them better understand and master the problem-solving skills.

[0085] The method provided in the embodiment of the present application determines, in response to a topic explanation request, a target topic to be explained, queries the topic type to which the target topic belongs based on a pre-constructed knowledge graph template, obtains an original matrix and multiple preset animation steps matching the topic type, generates a corresponding animation matrix for each preset animation step using the original matrix, integrates multiple animation matrices corresponding to multiple preset animation steps to obtain an animation demonstration video, generates a corresponding animation explanation audio for each preset animation step, and integrates the animation explanation audio corresponding to each preset animation step with the animation demonstration video to obtain a topic explanation video of the target topic, and combines a knowledge graph template with breadth and depth with the original matrix by setting an original matrix that can accurately indicate the number of rows and columns occupied by calculation results belonging to different topic types in the animation graphic space, so that the automatic topic explanation tool can flexibly generate explanation videos for different types of topics automatically without setting different explanation templates for different types of topics in advance, while improving the topic coverage and meeting the diverse needs of students, making the students' learning process vivid and interactive, so that students can easily understand and master problem-solving skills when watching the animation demonstration video.

[0086] Further, as a refinement and expansion of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, the embodiment of the present application provides another method for generating a topic explanation video, such as Figure 2A As shown, the method includes:

[0087] 201. Receive a question explanation request, and obtain a question formula according to a question input method indicated by the question explanation request.

[0088] The technical solution of the embodiment of the present application can be applied to the automatic question explanation tool. The automatic question explanation tool is an intelligent learning auxiliary tool that integrates computer technology and educational concepts, and is intended to provide students with efficient and accurate question explanation services in an automated and intelligent manner. The automatic question explanation tool can be installed on different platforms, such as online education platforms, learning management systems, mobile learning applications, etc., so as to receive and process question explanation requests received by these platforms.

[0089] The question explanation request is initiated by the user, and the user can initiate it in a variety of ways, such as clicking the "Explain the question" button on the application interface. The automatic question explanation tool allows the user to input the question to be explained in a variety of different ways. Therefore, after receiving the question explanation request, the automatic question explanation tool will identify the question input method selected by the user, and obtain the question formula based on the question input method. Among them, the question input method can be camera input, voice input or gesture input. The following describes the process of determining the question formula according to these three input methods:

[0090] 1. Photo input:

[0091] When the question input method indicates to input by taking a photo, the automatic question explanation tool obtains the question picture collected by the user who initiated the question explanation request based on the terminal held by the user, extracts the question text from the question picture, and converts the question text into the question formula. In actual application, if the user chooses to input by taking a photo, the automatic question explanation tool will use OCR (Optical Character Recognition) technology to extract the question text from the question picture taken by the user's terminal, and further convert these texts into question formulas. OCR technology determines the shape of characters by detecting the light and dark patterns in the picture, and then uses character recognition methods to translate these shapes into computer-readable text, thereby realizing the conversion from picture to text.

[0092] 2. Voice input:

[0093] When the question input method indicates voice input, the automatic question explanation tool obtains the voice content input by the user who initiates the question explanation request, recognizes the voice content as text information, and converts the text information into the question formula. In actual application, if the user chooses voice input, the automatic question explanation tool will use ASR (Automatic Speech Recognition) technology to recognize the user's voice content as text information and further convert it into the question formula. ASR technology automatically transcribes human oral speech into text through computers, greatly improving the convenience and efficiency of information input.

[0094] 3. Gesture input:

[0095] When the input method indicates gesture input, the automatic explanation tool obtains the image content currently captured by the terminal held by the user who initiated the question explanation request, selects the target area pointed by the user's finger in the image content, identifies the text content in the target area, and converts the text content into the question formula. In actual application, if the user chooses gesture input, the automatic explanation tool will automatically select the area pointed by the user's finger through the fingertip question-checking technology developed by the smart hardware, identify the question text therein, and convert it into the question formula. This method combines image recognition and gesture recognition technology to provide users with a more intuitive and natural interactive experience.

[0096] Through the above process, users can use a variety of different methods to input the questions to be explained, which greatly enriches the user's way of inputting questions, allowing users to choose the most convenient and efficient input method according to their actual situation and preferences. At the same time, through the use of advanced technologies such as OCR, ASR, and fingertip query, the automatic explanation tool can accurately and quickly identify and convert the question information entered by the user, providing a solid foundation for subsequent question explanations. For example, when a user encounters a complex math problem, manual input is inconvenient and takes a lot of time. The user can choose to take a photo to input the question, upload the question picture to the automatic explanation tool, and the automatic explanation tool will automatically identify and convert the question formula so as to provide the user with explanation services in the future.

[0097] 202. Input the question formula into the question judgment model, and obtain the judgment result output by the question judgment model for the question formula. When the judgment result indicates that the question formula is a calculation question, execute the following steps 203 to 208; when the judgment result indicates that the question formula is a non-calculation question, execute the following step 209.

[0098] In an embodiment of the present application, before explaining the formula of the question, the automatic explanation tool will perform a judgment operation on the formula of the question to determine whether it is a solvable calculation problem, thereby preventing the user from wasting time on incorrect formulas of the question.

[0099] Specifically, the automatic question-giving tool will input the question formula into the question-giving model, which is a pre-trained tool based on algorithms and data analysis, and is used to identify whether the input question formula is a correctly structured calculation formula. The question-giving model will analyze the question formula and output the judgment result. The automatic question-giving tool obtains the judgment result output by the question-giving model for the question formula. When the judgment result indicates that the question formula is a calculation question, the post-logic is performed, that is, the following steps 203 to 208 are executed; when the judgment result indicates that the question formula is a non-calculation question, the automatic question-giving tool will automatically trigger a feedback mechanism, which will immediately inform the user that their input is incorrect, that is, the input question formula does not meet the standard of the calculation question, may contain non-numerical operation elements, or belongs to a question that does not involve numerical operations and cannot be solved by mathematical formulas, such as multiple-choice questions, fill-in-the-blank questions, or text-based discussion questions, etc. The automatic question-giving tool will display a clear error message through the user interface, prompting the user to recheck and enter the correct calculation question formula, that is, execute the following step 209.

[0100] In this way, by providing immediate feedback on non-calculation questions, the automatic lecture tool can prevent users from wasting time on the wrong question types and guide them to quickly correct input errors, thereby using the automatic lecture tool more efficiently for learning and practicing calculation questions.

[0101] 203. When the judgment result indicates that the question formula is a calculation question, the question formula is used as the target question.

[0102] In the embodiment of the present application, when the judgment result indicates that the question formula is a calculation question, the automatic questioning tool will use the question formula as the target question. The question formula is a mathematical expression or a set of mathematical operations, including numbers, variables, operators, etc.

[0103] 204. Based on the pre-built knowledge graph template, query the question type to which the target question belongs, and obtain the original matrix and multiple preset animation steps matching the question type.

[0104] In an embodiment of the present application, the automatic lecture tool uses a pre-built knowledge graph template to query the type of the target question, and obtains the original matrix and multiple preset animation steps that match the type. Among them, the knowledge graph template is a complex network including multiple nodes constructed using various question types and their corresponding problem-solving steps, which can help the automatic lecture tool identify the type of question and the required knowledge points; each node in the knowledge graph template represents a specific question type or related knowledge point, and each node not only exists independently, but also is connected to other nodes through association relationships, forming a rich knowledge network; each node is associated with at least one candidate problem-solving path, which represents different methods or strategies that may be used to solve specific types of questions. The original matrix is ​​a data structure used to guide animation display, which defines the layout of elements in the animation graphic space. The preset animation steps are a series of pre-designed animation actions used to show the problem-solving steps to the user during the problem-solving process.

[0105] Specifically, the automatic lecture tool will first extract the topic features from the target topic. The topic features include one or more of operation symbols, variable information, constant information, and function information. Then, the automatic lecture tool will match the topic features with multiple nodes in the knowledge graph template to find the target node that matches the topic features. Since each node corresponds to a topic type, once the target node is found, the automatic lecture tool can use the topic type identified by the target node as the topic type to which the target topic belongs, and obtain the original matrix associated with the topic type. The original matrix is ​​used to indicate the number of rows and columns required to display the calculation results of the topic in the animated graphic space. Taking the target topic as "33+44=?" as an example, through the knowledge graph template, the automatic lecture tool determines that the topic type corresponding to "33+44=?" is "vertical addition", then the obtained original matrix can be Figure 1B In this form, the original matrix represents "33+44=?" The final calculation result will occupy 5 rows and 3 columns in the animation graphics space.

[0106] Next, the automatic lecture tool will also find at least one candidate solution path associated with the target node in the knowledge graph template. As can be seen from the above content, the knowledge graph template in the embodiment of the present application not only contains various mathematical knowledge points and their relationships, but also records the problem-solving logic and steps between these knowledge points. Therefore, the automatic lecture tool will explore all possible problem-solving paths along the connection of the target node in the knowledge graph template. The problem-solving path is like a bridge connecting different knowledge points, providing users with a variety of problem-solving solutions from the starting point to the end point.

[0107] Subsequently, the automatic explanation tool will select a candidate solution path that matches the characteristics of the question from at least one candidate solution path as the target solution path based on the characteristics of the question. During the selection process, the automatic explanation tool will fully consider the specific operations, variable relationships, function types and other factors of the target question to ensure that the selected target solution path can accurately reflect the solution logic and steps of the question, and ensure that a solution that not only meets the requirements of the target question but is also easy to understand and operate is selected.

[0108] After determining the target problem-solving path, the automatic explanation tool will read the path information of the target problem-solving path and obtain multiple preset animation steps. Multiple preset animation steps are designed by the automatic explanation tool to more intuitively display the problem-solving process. They closely combine each step of the problem-solving operation, calculation results and graphical display in the form of animation, allowing users to easily follow the guidance and gradually complete the problem-solving.

[0109] However, if the target node that matches the characteristics of the question cannot be found among multiple nodes, the automatic lecture tool will trigger the exception handling mechanism. The exception handling mechanism will first determine the recipient of the exception feedback, which can be the administrator or technical support team of the automatic lecture tool. Then, the automatic lecture tool will generate an exception reminder with the target question, which records in detail the question information, the query process, and the reason why the matching node could not be found. Finally, the automatic lecture tool will push this exception reminder to the recipient of the exception feedback so that they can find the problem and deal with it in time, ensuring that the automatic lecture tool can give timely feedback when faced with unsolvable problems, avoiding confusion for users.

[0110] 205. Generate a corresponding animation matrix for each preset animation step using the original matrix.

[0111] In the embodiment of the present application, after the original matrix and multiple preset animation steps are determined, the automatic presentation tool will use the original matrix to generate a corresponding animation matrix for each preset animation step, vividly and intuitively displaying each step of the mathematical operation.

[0112] Specifically, for each preset animation step, the automatic lecture tool reads the matrix position information, animation supplementary information, and animation behavior information associated with the preset animation step in the knowledge graph template. Among them, the matrix position information refers to the specific position in the animation step that needs to be modified or displayed in the original matrix. These positions are usually expressed in the form of coordinates, such as [0, 2], etc., which indicate the rows and columns in the original matrix. During the animation production process, the automatic lecture tool will locate the elements that need to be modified or displayed in the original matrix based on this position information, such as numbers, operators, or special marks. The accuracy of this position information is the basis for ensuring the consistency of the animation and mathematical operation process.

[0113] Animation supplementary information is a further explanation and supplement to the animation steps. It provides additional information on how to modify or display the matrix specifically. This information may include special marks that need to be added, numerical ranges that need to be modified, animation effects that need to be emphasized, etc. The existence of animation supplementary information makes the animation display more flexible and rich, and can be personalized according to different mathematical operation types and step requirements. For example, when showing vertical addition, animation supplementary information may indicate that dividing lines should be added in certain positions to distinguish different digits; or during the calculation process, the numbers or operators being operated are highlighted to guide the user's attention.

[0114] Animation behavior information refers to the specific form and action of the animation steps, which determines the presentation method and dynamic effect of the animation. Animation behavior information can include the animation playback speed, color change, shape transformation, dynamic trajectory, etc. During the animation matrix generation process, the automatic lecture tool will render the adjusted intermediate matrix according to the animation behavior information to obtain the final animation effect presented to the user. The selection and setting of animation behavior information is crucial to enhancing the attractiveness and educational effect of animation, making the animation display more vivid and interesting, and enhancing the user's learning experience and interest.

[0115] After determining the matrix position information, animation supplementary information and animation behavior information, since the matrix position information indicates which positions in the original matrix need to be modified or displayed, the automatic presentation tool will extract at least one topic character that matches the matrix position information in the target topic, and obtain a plurality of predefined reference characters, and determine the corresponding target reference character for each topic character in the plurality of reference characters. Among them, the reference character is a pre-set character used to refer to various symbols. In the actual application process, the characters below 0 are defined to refer to the auxiliary animation demonstration digits, and 0 and above are natural numbers. Specifically, the character "-10" is used to refer to the placeholder, the character "-9" is used to refer to the dividing line, the character "-8" is used to refer to the plus sign, the character "-7" is used to refer to the minus sign, the character "-6" is used to refer to the multiplication sign, the character "-5" is used to refer to the division sign, the character "-4" is used to refer to the digit display, the character "-3" is used to refer to the decimal point, the characters "-2" and "-1" are reserved for backup, and the characters "0-9" refer to natural numbers. After determining the corresponding target reference character for each question character, we can refer to the matrix position information and use the target reference character corresponding to each question character to assign values ​​to the matrix digits at the corresponding positions in the original matrix to obtain the intermediate matrix. The intermediate matrix already has the basic framework for animation display.

[0116] Next, the automatic teaching tool will adjust the intermediate matrix according to the animation supplementary information, such as modifying the value, to obtain the adjusted intermediate matrix.

[0117] Finally, the automatic teaching tool renders the adjusted intermediate matrix according to the animation behavior information to obtain the animation matrix corresponding to the preset animation steps. This process involves the specific forms of animation, such as color, shape, dynamic effects, etc., which together constitute the animation matrix finally presented to the user. The animation matrix not only accurately reflects each step of the mathematical operation, but also greatly enhances the user's learning experience and interest through vivid animation effects.

[0118] Continuing with the target question "33+44=?" as an example, in actual application, the generation of the animation matrix corresponding to the first preset animation step can be realized based on the following data sample:

[0119] Type: drawBit

[0120] Loc: [0, 2]

[0121] Txt: "pcs"

[0122] Among them, the Type field indicates that the type of the animation step is "drawBit", that is, "drawing digits". "Digits" refer to the positions where numbers are located in vertical calculations, such as the units digit, tens digit, etc. The animation step of the drawBit type is used to draw these digits in the animation matrix so that users can clearly see how the numbers are arranged and calculated in the vertical form.

[0123] The Loc field provides the specific position information where the digits need to be drawn in the animation step. It is a two-dimensional coordinate that indicates the row and column in the animation matrix. In the above example, [0, 2] means that the digit will be drawn at the third position in the first row (the row index starts counting from 0) (the column index also starts counting from 0). This position information is crucial for ensuring the correct arrangement of the digits in the animation matrix.

[0124] The Txt field provides the text representation of the digit to be drawn. In this example, "units" represents the units digit, that is, the lowest digit in the vertical form. This text information will be drawn at the specified position in the animation matrix so that users can identify the digit represented by this number.

[0125] In summary, the above data sample describes a specific animation step, which requires the automatic problem-solving tool to draw a text marker representing the units digit at the third position in the first row to form an animation matrix, helping users to more intuitively understand the principle and steps of vertical calculation.

[0126] Continuing with the example where the target question is "33 + 44 =?", the final generated animation matrix for it can be Figure 2B in the form of, where "-10" represents a placeholder, "-8" represents a plus sign, "-4" represents digit display, "-9" represents a dividing line, and "3", "4", "7" represent natural numbers. In this way, after rendering this animation matrix, the Figure 2C shown animation effect can be obtained.

[0127] 206. Integrate multiple animation matrices corresponding to multiple preset animation steps to obtain an animation demonstration video.

[0128] In the embodiments of the present application, the automatic problem-solving tool will integrate the animation matrices corresponding to multiple preset animation steps, aiming to dynamically generate a complete animation demonstration video. Specifically, the automatic problem-solving tool will use computer graphics technology to generate animation frames, and through the continuous playback of the frames, form a smooth animation demonstration, including animation effects such as the writing of numbers, the transformation of the calculation process, the display of the calculation result, carry and borrow, etc. Among them, computer graphics technology is the key to realizing the animation demonstration video, which covers multiple aspects such as two-dimensional to three-dimensional graphics generation, rendering, transformation, and processing, and can create various complex and realistic animation effects according to requirements.

[0129] Specifically, the automatic presentation tool will first load and integrate the corresponding animation matrices one by one according to the order of the preset animation steps. Then, using the rendering technology in computer graphics, these animation matrices are converted into continuous animation frames. During the rendering process, the automatic presentation tool will pay attention to the processing of details, such as the simulation of the writing trajectory of the numbers and the smooth transition of the calculation process, to ensure the fluency and authenticity of the animation demonstration.

[0130] Taking vertical addition as an example, the automatic teaching tool will integrate the key steps such as the writing process of numbers, the calculation steps of addition, the processing of carry and the display of the final result into the animation demonstration video. In the animation, users can see how the numbers are written stroke by stroke, as well as the changes and results of each step in the addition calculation process, which not only helps users understand the principles and steps of vertical addition, but also stimulates their interest and motivation in learning.

[0131] 207. Generate corresponding animation explanation audio for each preset animation step, and integrate the animation explanation audio corresponding to each preset animation step with the animation demonstration video to obtain a question explanation video of the target question.

[0132] In an embodiment of the present application, the automatic explanation tool generates corresponding animation explanation audio for each preset animation step, and integrates the animation explanation audio corresponding to each preset animation step with the animation demonstration video to obtain the topic explanation video of the target topic. Among them, the automatic explanation tool first extracts the step detail information in each preset animation step, and captures the topic detail information in the animation matrix corresponding to each preset animation step. After integration, this information forms detailed step introduction information, which provides a solid foundation for subsequent voice conversion. The automatic explanation tool uses text-to-speech technology to perform voice conversion on the step introduction information of each preset animation step, obtains the initial explanation audio corresponding to each preset animation step, and adjusts the initial explanation audio corresponding to each preset animation step according to the preset voice parameters, such as the preset voice parameters such as speech speed and intonation, to obtain the animation explanation audio corresponding to each preset animation step, and ensures that the explanation audio of each animation step is both natural and smooth.

[0133] Next, the automatic explanation tool segments the animation demonstration video and determines the video clip corresponding to each preset animation step, ensuring that each video clip corresponds to the corresponding animation step. Then, the video clip corresponding to each preset animation step is aligned with its corresponding animation explanation audio on the timeline, so that each video clip is merged with its exclusive animation explanation audio through precise timeline alignment technology; at the same time, the automatic explanation tool also adjusts the video clip according to the preset playback speed to obtain an initial explanation video with a moderate rhythm and easy to understand.

[0134] Finally, based on the generation of the initial explanation video, the automatic explanation tool further refines the video content to enhance its educational value and interactivity. Specifically, the automatic explanation tool first reviews the initial explanation video and identifies those elements that are crucial to understanding the problem-solving process, which are defined as elements to be marked. The elements to be marked can be key mathematical formulas, turning points in the problem-solving steps, or places where errors are prone to occur. The automatic explanation tool then uses preset key marks to mark the elements to be marked in the initial explanation video to achieve visual emphasis. Key marks can be highlights, borders, animation effects, or other visual elements that can attract the audience's attention. By highlighting these key elements, it is intended to help learners lock in important information more quickly, thereby deepening their understanding and memory of the problem-solving process. In addition, in addition to visual key marks, the automatic explanation tool also embeds preset interactive elements in the initial explanation video. Preset interactive elements can include question prompts, multiple-choice questions, drag-and-drop fill-in-the-blanks, or other forms of user interaction. By embedding these interactive elements, learners are encouraged to actively participate in the process of watching the video, consolidate their knowledge through practical operations, and test their understanding.

[0135] To sum up, in the above process, text-to-speech technology is used to extract text from the steps of the animation demonstration and automatically synthesize the explanation audio, including a detailed explanation of each calculation step and an elaboration of the mathematical principles; the audio is synthesized using text-to-speech technology to ensure the natural and smooth explanation; further, the video is edited and optimized, including adjusting the playback speed, adding key marks and interactive elements, and synchronously integrating the animation demonstration and audio explanation to form a complete calculation problem explanation video to improve the teaching effect.

[0136] 208. Use the question explanation video to provide the explanation of the target question to the user who initiates the question explanation request.

[0137] In an embodiment of the present application, after a question explanation video is generated, the automatic question explanation tool will use the question explanation video to provide explanations of the target questions to the user who initiated the question explanation request, so that the user no longer needs to rely on traditional paper textbooks or face-to-face tutoring, but can obtain high-quality question explanation videos anytime and anywhere through an online platform, which not only saves the user's time and energy, but also enables educational resources to transcend geographical restrictions and be shared by more users.

[0138] In another optional implementation scheme, in order to meet the diverse needs of students when watching the topic explanation video, the automatic topic explanation tool will split the synthesized topic explanation video into multiple steps and store them separately, and provide a dedicated explanation video player at the front end to classify the type of each step, such as playing carry animation, digital animation, filling in numbers, playing explanation audio, etc., so that students can choose different steps to watch by themselves, and allow students to interact while watching the video, such as pausing the video at key steps for practice, etc. Specifically, the automatic topic explanation tool will obtain multiple preset step types, divide the topic explanation video according to multiple step types, and obtain multiple step segments to ensure that each step matches its corresponding step type. Next, the automatic topic explanation tool will determine the matching step segment for each step type in the multiple step segments and establish an association relationship, and store the multiple step segments after association in the database so that they can be accurately called in the subsequent playback process. In this way, when providing explanations for the target topic, the automatic topic explanation tool will retrieve the step segments related to the topic from the database and play them in a preset order. Continuing with the target question "33+44=?" as an example, in the actual application process, the storage of a step segment can be implemented based on the following data sample, so that by running the data sample, the playback of the corresponding step segment can be implemented:

[0139] Type: drawBit

[0140] Loc: [0, 2]

[0141] Txt: "pcs"

[0142] Voice: "This is an integer addition equation. You can use the vertical form to calculate it."

[0143] KnowledgePoint: "Digital Alignment"

[0144] The Type field indicates that the type of animation step is "drawBit", which means "draw digits". "Digits" refer to the position of numbers in vertical calculations, such as ones, tens, etc. The drawBit type animation step is used to draw these digits in the animation matrix so that users can clearly see how the numbers are arranged and calculated in the vertical calculation.

[0145] The Loc field provides the specific location information of the digits to be drawn in the animation step. It is a two-dimensional coordinate that specifies the row and column in the animation matrix. In the above example, [0, 2] means that the digit will be drawn in the third position (column index also starts counting from 0) of the first row (row index starts counting from 0). This position information is crucial to ensure the correct arrangement of the digits in the animation matrix.

[0146] The Txt field provides a text representation of the digit to be drawn. In this example, "units" represents the units place, which is the lowest place value of the number in the vertical form. This text information will be drawn at the specified position in the animation matrix so that users can identify the place value represented by this number.

[0147] The Voice field provides voiceover information related to the animation step. In the above example, the voiceover explains to the user that the current operation is an integer addition formula and can be calculated using the vertical form, which helps to enhance the user's understanding and memory of the animation content.

[0148] The KnowledgePoint field indicates the knowledge points or concepts involved in the animation step. In the above example, "aligning place values" is an important principle in vertical calculation, which requires that the digits in each place value must be aligned when performing addition or subtraction operations, helping users to establish correct mathematical concepts and thinking patterns during the learning process.

[0149] In addition, to further enhance the learning experience of students, the automatic problem-solving tool also provides a hint词条 function. The automatic problem-solving tool will display hint词条 related to each step type, and when any hint词条 is detected to be triggered, it will play the step segment associated with the step type corresponding to the triggered hint词条, thereby helping students to understand and master relevant knowledge more deeply.

[0150] Continuing with the example where the target problem is "33 + 44 =?", as Figure 2D shown, the explanatory video for "33 + 44 =?" will be played in the disassembled order. The first step is the place value explanation. Users can pause the animation and adjust the playback speed during the playback to facilitate better learning of the current calculation problem. After the playback is completed, users can choose to watch again or exit the explanation.

[0151] In another optional implementation, the automatic problem-solving tool will record information such as the user's viewing behavior, interaction, and satisfaction with the video in real time, that is, collect the opinion information feedback by the user. This opinion information is cached and pushed to the abnormal feedback recipient, such as the teaching team or the technical department, at preset time intervals. They can refer to these feedback opinions to make targeted updates and optimizations to the knowledge graph template to ensure that the teaching content is consistent with the changes in the education syllabus and continuously improve the quality and teaching effect of the video. In addition, the abnormal feedback recipient will continuously maintain and upgrade technologies such as OCR technology and ASR technology to improve the accuracy and efficiency of the automatic problem-solving tool.

[0152] Through the above technical solution, the knowledge graph template with extensiveness and depth is combined with the original matrix, so that the automatic explanation tool can flexibly generate explanation videos for different types of questions, without setting different explanation templates for different types of questions in advance. It can realize the production of automatic explanation videos for all types of math calculation questions in primary school, with higher coverage, greatly improving the accessibility of educational resources and teaching efficiency. At the same time, an easy-to-use explanation video player is also provided on the front end, making the students more interactive in the learning process and easier to understand the principles of calculation questions.

[0153] 209. When the judgment result indicates that the formula of the question is a non-calculation question, a re-input instruction is pushed to the terminal that initiated the question explanation request, so that the user holding the terminal re-initiates the question explanation request.

[0154] In the embodiment of the present application, when the judgment result indicates that the formula of the question is a non-calculation question, the automatic explanation tool will automatically trigger a feedback mechanism, which will immediately inform the user that their input is incorrect, that is, the input question formula does not meet the standard of the calculation question, may contain non-numerical operation elements, or is a question that does not involve numerical operations and cannot be solved by mathematical formulas, such as multiple-choice questions, fill-in-the-blank questions, or text-based discussion questions. The automatic explanation tool will push a re-entry instruction to the terminal that initiated the question explanation request through the user interface, prompting the user to re-check and enter the correct calculation formula.

[0155] For example, if a user tries to enter a quiz about a historical event as a question formula in the automatic question-answering tool, the question-judgment model will recognize that this is a non-calculation question. Subsequently, the automatic question-answering tool will immediately display a message on the user interface, such as "The question you entered is not a calculation question, please check and re-enter it." In this way, the user can immediately realize the error and take appropriate corrective measures.

[0156] In summary, the logical process of the technical solution in this application is summarized as follows: Figure 2E, obtain the question formula input by the user using the corresponding input method, which can be specifically input by taking a photo, voice input, and gesture input. Determine whether the question formula is a calculation question. If so, match the question type of the question formula in the knowledge graph template. If not, remind the user to re-photograph the question. Among them, when matching the question type of the question formula, if no match is found, it is determined that the question formula does not conform to the template strategy, and the person in charge of the knowledge graph template needs to be informed so that the person in charge can make corrections; and if a match is found, generate an animation demonstration video for the question formula according to the steps, continue to generate animation explanation audio for the animation demonstration video, merge the animation demonstration video and the animation explanation audio, obtain the question explanation video and store it in the library. When explaining the question formula to the user, use a dedicated player to play the question explanation video and interact with the user, and collect user feedback. When the opinion indicates a bad review, it is pushed to the person in charge of the knowledge graph template so that the person in charge can make corrections, and when the opinion indicates a good review, end the current process.

[0157] The method provided in the embodiment of the present application determines, in response to a topic explanation request, a target topic to be explained, queries the topic type to which the target topic belongs based on a pre-constructed knowledge graph template, obtains an original matrix and multiple preset animation steps matching the topic type, generates a corresponding animation matrix for each preset animation step using the original matrix, integrates multiple animation matrices corresponding to multiple preset animation steps to obtain an animation demonstration video, generates a corresponding animation explanation audio for each preset animation step, and integrates the animation explanation audio corresponding to each preset animation step with the animation demonstration video to obtain a topic explanation video of the target topic, and combines a knowledge graph template with breadth and depth with the original matrix by setting an original matrix that can accurately indicate the number of rows and columns occupied by calculation results belonging to different topic types in the animation graphic space, so that the automatic topic explanation tool can flexibly generate explanation videos for different types of topics automatically without setting different explanation templates for different types of topics in advance, while improving the topic coverage and meeting the diverse needs of students, making the students' learning process vivid and interactive, so that students can easily understand and master problem-solving skills when watching the animation demonstration video.

[0158] Further, as a specific implementation of the method described in FIG. 1 , the present application embodiment provides a device for generating a topic explanation video, such as Figure 3 As shown, the device includes: a determination module 301, a query module 302, a generation module 303 and an integration module 304.

[0159] A determination module 301 is used to determine a target topic to be explained in response to a topic explanation request;

[0160] A query module 302 is used to query the type of question to which the target question belongs based on a pre-built knowledge graph template, and obtain an original matrix and a plurality of preset animation steps matching the type of question, wherein the original matrix is ​​used to indicate the number of rows and columns occupied by the calculation result of the question belonging to the type of question in the animation graphic space;

[0161] A generating module 303 is used to generate a corresponding animation matrix for each of the preset animation steps using the original matrix, and to integrate multiple animation matrices corresponding to the multiple preset animation steps to obtain an animation demonstration video;

[0162] The integration module 304 is used to generate a corresponding animation explanation audio for each of the preset animation steps, and integrate the animation explanation audio corresponding to each of the preset animation steps with the animation demonstration video to obtain a topic explanation video of the target topic.

[0163] In a specific application scenario, the determination module 301 is used to receive the question explanation request, obtain the question formula according to the question input method indicated by the question explanation request; input the question formula into the question judgment model, and obtain the judgment result output by the question judgment model for the question formula; when the judgment result indicates that the question formula is a calculation question, use the question formula as the target question; wherein, when the judgment result indicates that the question formula is a non-calculation question, push a re-input instruction to the terminal that initiated the question explanation request, so that the user holding the terminal re-initiates the question explanation request.

[0164] In a specific application scenario, the determination module 301 is used to identify the question input method indicated by the question explanation request; when the question input method indicates photo input, obtain the question picture collected by the user who initiates the question explanation request based on the terminal held by the user, extract the question text from the question picture, and convert the question text into the question formula; when the question input method indicates voice input, obtain the voice content input by the user who initiates the question explanation request, recognize the voice content as text information, and convert the text information into the question formula; when the input method indicates gesture input, obtain the image content currently captured by the terminal held by the user who initiates the question explanation request, select the target area pointed by the user's finger in the image content, recognize the text content of the target area, and convert the text content into the question formula.

[0165] In a specific application scenario, the query module 302 is used to extract question features from the target question, wherein the question features include one or more of operation symbols, variable information, constant information, and function information; match the question features with multiple nodes included in the knowledge graph template, and query the target node matching the question features among the multiple nodes; use the question type identified by the target node as the question type to which the target question belongs, and obtain the original matrix associated with the question type; obtain at least one candidate solution path associated with the target node in the knowledge graph template, and determine the candidate solution path matching the question features among the at least one candidate solution path as the target solution path; read the path information of the target solution path to obtain the multiple preset animation steps; wherein, when the target node matching the question features is not found among the multiple nodes, determine an abnormal feedback recipient, generate an abnormal reminder carrying the target question, and push the abnormal reminder to the abnormal feedback recipient.

[0166] In a specific application scenario, the generation module 303 is used to read the matrix position information, animation supplementary information and animation behavior information associated with each preset animation step in the knowledge graph template; extract at least one title character matching the matrix position information in the target title, and obtain a plurality of predefined reference characters, and determine a corresponding target reference character for each title character in the plurality of reference characters; refer to the matrix position information, use the target reference character corresponding to each title character, assign values ​​to the matrix digits at the corresponding positions in the original matrix, and obtain an intermediate matrix; adjust the intermediate matrix according to the animation supplementary information to obtain the adjusted intermediate matrix; render the adjusted intermediate matrix according to the animation behavior information to obtain the animation matrix corresponding to the preset animation step.

[0167] In a specific application scenario, the integration module 304 is used to extract step detail information from each of the preset animation steps, and to extract topic detail information from the animation matrix corresponding to each of the preset animation steps; integrate the step detail information and topic detail information of each of the preset animation steps to form step introduction information of each of the preset animation steps; perform voice conversion on the step introduction information of each of the preset animation steps to obtain the initial explanation audio corresponding to each of the preset animation steps, and adjust the initial explanation audio corresponding to each of the preset animation steps according to preset voice parameters to obtain the animation explanation audio corresponding to each of the preset animation steps; segment the animation demonstration video to determine the video clip corresponding to each of the preset animation steps, and align the video clip corresponding to each of the preset animation steps with its corresponding animation explanation audio on the timeline, and adjust the playback speed of each of the video clips according to a preset playback speed to obtain the initial explanation video; determine the elements to be marked in the initial explanation video, mark the elements to be marked in the initial explanation video using a preset emphasis mark, and embed a preset interactive element in the initial explanation video to obtain the topic explanation video of the target topic.

[0168] In a specific application scenario, the device also includes:

[0169] A storage module is used to obtain a plurality of preset step types, segment the topic explanation video according to the plurality of step types to obtain a plurality of step segments, determine a matching step segment for each of the step types in the plurality of step segments and establish an association relationship, store the plurality of associated step segments in a database, and display a prompt entry related to each of the step types when providing explanations for the target topic, and when any prompt entry is detected to be triggered, play a step segment associated with the step type corresponding to the triggered prompt entry; and / or,

[0170] The feedback module is used to collect and cache the opinion information of user feedback, and push at least one opinion information currently cached to the abnormal feedback recipient every preset time period, so that the abnormal feedback recipient can refer to the at least one opinion information currently cached to update the knowledge graph template.

[0171] The device provided in the embodiment of the present application determines a target topic to be explained in response to a topic explanation request, queries the topic type to which the target topic belongs based on a pre-constructed knowledge graph template, obtains an original matrix and multiple preset animation steps matching the topic type, generates a corresponding animation matrix for each preset animation step using the original matrix, integrates multiple animation matrices corresponding to multiple preset animation steps to obtain an animation demonstration video, generates a corresponding animation explanation audio for each preset animation step, and integrates the animation explanation audio corresponding to each preset animation step with the animation demonstration video to obtain a topic explanation video of the target topic, and combines a knowledge graph template with breadth and depth with the original matrix by setting an original matrix that can accurately indicate the number of rows and columns occupied by calculation results belonging to different topic types in the animation graphic space, so that the automatic topic explanation tool can flexibly generate explanation videos for different types of topics automatically without setting different explanation templates for different types of topics in advance, while improving the topic coverage and meeting the diverse needs of students, making the students' learning process vivid and interactive, so that students can easily understand and master problem-solving skills when watching the animation demonstration video.

[0172] It should be noted that for other corresponding descriptions of the functional units involved in the apparatus for generating a topic explanation video provided in the embodiment of the present application, reference can be made to Figures 1A to 1B and FIG. 2A to FIG. 2E The corresponding description in will not be repeated here.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0174] The above embodiments and the technical features in the embodiments can be arbitrarily combined with each other. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

[0176] In an exemplary embodiment, see Figure 4, and also provides an electronic device, which includes a bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device, wherein each functional unit can communicate with each other through the bus. The memory stores a computer program, and the processor is used to execute the program stored in the memory and execute the method for generating a question explanation video in the above embodiment.

[0177] A storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for generating a topic explanation video.

[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by hardware, or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0179] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present application.

[0180] Those skilled in the art will appreciate that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0181] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0182] The above disclosure only discloses several specific implementation scenarios of the present application. However, the present application is not limited thereto, and any changes that can be conceived by technicians in this field should fall within the scope of protection of the present application.

Claims

1. A method for generating a topic explanation video, characterized in that: include: In response to the topic explanation request, determining a target topic to be explained; Based on the pre-built knowledge graph template, query the question type to which the target question belongs, and obtain an original matrix and multiple preset animation steps matching the question type, wherein the original matrix is ​​used to indicate the number of rows and columns occupied by the calculation results of the questions belonging to the question type in the animation graphic space; Using the original matrix to generate a corresponding animation matrix for each of the preset animation steps, and integrating multiple animation matrices corresponding to the multiple preset animation steps to obtain an animation demonstration video; Generate a corresponding animation explanation audio for each of the preset animation steps, and integrate the animation explanation audio corresponding to each of the preset animation steps with the animation demonstration video to obtain a question explanation video for the target question.

2. The method according to claim 1, characterized in that The step of determining, in response to the topic explanation request, a target topic to be explained includes: Receiving the question explanation request, and obtaining the question formula according to the question input method indicated by the question explanation request; Input the question formula into the question determination model, and obtain the determination result output by the question determination model for the question formula; When the judgment result indicates that the question formula is a calculation question, taking the question formula as the target question; Wherein, when the judgment result indicates that the question formula is a non-calculation question, a re-input instruction is pushed to the terminal that initiated the question explanation request, so that the user holding the terminal re-initiates the question explanation request.

3. The method according to claim 2, characterized in that The step of obtaining the question formula according to the question input method indicated by the question explanation request includes: Identify the topic input method indicated by the topic explanation request; When the question input mode indicates a photograph input, obtaining a question picture collected by a terminal held by a user who initiates the question explanation request, extracting a question text from the question picture, and converting the question text into the question formula; When the question input mode indicates voice input, obtaining the voice content input by the user who initiated the question explanation request, recognizing the voice content as text information, and converting the text information into the question formula; When the input method indicates gesture input, the image content currently captured by the terminal held by the user who initiated the question explanation request is obtained, the target area pointed by the user's finger is selected in the image content, the text content of the target area is identified, and the text content is converted into the question formula.

4. The method according to claim 1, characterized in that The method of querying the topic type to which the target topic belongs based on the pre-built knowledge graph template and obtaining the original matrix and multiple preset animation steps matching the topic type includes: Extracting topic features from the target topic, wherein the topic features include one or more of operation symbols, variable information, constant information, and function information; Matching the topic feature with a plurality of nodes included in the knowledge graph template, and searching for a target node matching the topic feature among the plurality of nodes; Taking the topic type identified by the target node as the topic type to which the target topic belongs, and obtaining the original matrix associated with the topic type; Obtain at least one candidate problem-solving path associated with the target node in the knowledge graph template, and determine a candidate problem-solving path that matches the question feature from the at least one candidate problem-solving path as a target problem-solving path; Reading the path information of the target problem-solving path to obtain the plurality of preset animation steps; When the target node matching the topic feature is not found among the multiple nodes, an abnormal feedback recipient is determined, an abnormal reminder carrying the target topic is generated, and the abnormal reminder is pushed to the abnormal feedback recipient.

5. The method according to claim 1, characterized in that The using the original matrix to generate a corresponding animation matrix for each preset animation step includes: For each of the preset animation steps, read the matrix position information, animation supplementary information, and animation behavior information associated with the preset animation step in the knowledge graph template; Extracting at least one topic character matching the matrix position information from the target topic, acquiring a plurality of predefined reference characters, and determining a corresponding target reference character for each of the topic characters in the plurality of reference characters; Referring to the matrix position information, using the target reference character corresponding to each of the question characters, assigning values ​​to the matrix digits at the corresponding positions in the original matrix to obtain an intermediate matrix; Adjusting the intermediate matrix according to the animation supplementary information to obtain the adjusted intermediate matrix; The adjusted intermediate matrix is ​​rendered according to the animation behavior information to obtain an animation matrix corresponding to the preset animation step.

6. The method according to claim 1, characterized in that The step of generating a corresponding animation explanation audio for each of the preset animation steps, and integrating the animation explanation audio corresponding to each of the preset animation steps with the animation demonstration video to obtain a topic explanation video for the target topic includes: Extracting step detail information from each of the preset animation steps, and extracting topic detail information from the animation matrix corresponding to each of the preset animation steps; Integrate the step detail information and the title detail information of each of the preset animation steps to form step introduction information of each of the preset animation steps; Performing voice conversion on the step introduction information of each of the preset animation steps to obtain an initial explanation audio corresponding to each of the preset animation steps, and adjusting the initial explanation audio corresponding to each of the preset animation steps according to preset voice parameters to obtain an animation explanation audio corresponding to each of the preset animation steps; The animation demonstration video is segmented to determine the video segment corresponding to each of the preset animation steps, and the video segment corresponding to each of the preset animation steps is aligned with the corresponding animation explanation audio on the time axis, and the playback speed of each of the video segments is adjusted according to the preset playback speed to obtain an initial explanation video; Determine the elements to be marked in the initial explanation video, mark the elements to be marked in the initial explanation video with preset key marks, and embed preset interactive elements in the initial explanation video to obtain the topic explanation video of the target topic.

7. The method according to claim 1, characterized in that The method further comprises: Acquire a plurality of preset step types, segment the topic explanation video according to the plurality of step types to obtain a plurality of step segments, determine a matching step segment for each of the step types in the plurality of step segments and establish an association relationship, store the associated plurality of step segments in a database, and display prompt terms related to each of the step types when providing explanations for the target topic, and when any prompt term is detected to be triggered, play the step segment associated with the step type corresponding to the triggered prompt term; and / or, Collect and cache the opinion information of user feedback, and push at least one opinion information currently cached to the abnormal feedback recipient every preset time period, so that the abnormal feedback recipient can refer to at least one opinion information currently cached to update the knowledge graph template.

8. A device for generating a topic explanation video, characterized in that: include: A determination module, for determining a target topic to be explained in response to a topic explanation request; A query module, used to query the type of question to which the target question belongs based on a pre-built knowledge graph template, and obtain an original matrix and a plurality of preset animation steps matching the type of question, wherein the original matrix is ​​used to indicate the number of rows and columns occupied by the calculation results of the question belonging to the type of question in the animation graphic space; A generating module, configured to generate a corresponding animation matrix for each of the preset animation steps using the original matrix, and to integrate multiple animation matrices corresponding to the multiple preset animation steps to obtain an animation demonstration video; The integration module is used to generate a corresponding animation explanation audio for each of the preset animation steps, and integrate the animation explanation audio corresponding to each of the preset animation steps with the animation demonstration video to obtain a question explanation video for the target question.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images