Method, device and equipment for generating and processing explanation audio and video

Abstract

The invention provides an explanation audio and video generation processing method, device and equipment, and relates to the video processing technology, and the method comprises the steps: obtaining a to-be-analyzed request; wherein the to-be-analyzed request is used for requesting to generate the explanation audio and video, the to-be-analyzed request comprises to-be-processed information, and the to-be-processed information is an arithmetic expression. And performing data analysis on the to-be-processed information, and determining an arithmetic structure and a numerical range. Determining at least one target explanation option template corresponding to the arithmetic structure and the numerical range in a plurality of preset explanation option templates; wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the explanation rule information is determined according to an arithmetic structure and a numerical range. And according to the at least one target explanation option template and the to-be-processed information, generating an explanation audio / video, and displaying the explanation audio / video. According to the method provided by the invention, the technical problem that the efficiency of explaining the generation mode of the audio and video is relatively low is solved.

Description

Technical Field

[0001] The present application relates to video processing technology, and in particular to a method, device and equipment for generating and processing audio and video of explanations. Background Art

[0002] At present, the development of artificial intelligence technology has brought many changes to the education industry. For example, the use of virtual digital human images to replicate the image, voice, and movements of teachers to create a virtual image that students are more interested in. By inputting good scripts, courseware videos can be quickly generated. AI digital human teachers can set up artificial intelligence recognition and interactive systems to answer students' questions in a timely and accurate manner; and can be used to provide students with interactive learning experiences. For example, by introducing vivid story scenarios and visual knowledge presentation, explanation videos can allow children to understand knowledge in real scenarios, master knowledge to form self-learning paths, and achieve autonomous exploratory learning.

[0003] In the prior art, the explanation video is generated by recording the video in advance according to the topic and playing it directly when the student selects it.

[0004] However, in the prior art, since the video is recorded in advance according to the question, the method of generating the explanation video is fixed, while the actual problems may be complex and changeable. For example, there are many kinds of variations of multiplication and division in elementary school mathematics. In this case, it is necessary to record multiple videos in advance according to the various variations, and each question needs to record a corresponding video. The video recording workload is large and the storage volume is large. Therefore, the existing video generation method lacks flexibility and is inefficient. Summary of the invention

[0005] The present application provides a method, device and equipment for generating and processing audio and video of explanations, so as to solve the technical problem of low efficiency in the generation method of audio and video of explanations.

[0006] In a first aspect, the present application provides a method for generating and processing audio and video of explanations, comprising:

[0007] Obtaining a request to be parsed; wherein the request to be parsed is used to request the generation of audio and video explanations, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed;

[0008] Performing data analysis on the information to be processed to determine an arithmetic structure and a numerical range;

[0009] Determine at least one target explanation option template corresponding to both the arithmetic structure and the numerical range from among a plurality of preset explanation option templates; wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the explanation rule information is determined according to the arithmetic structure and the numerical range;

[0010] Generate explanation audio and video according to the at least one target explanation option template and the information to be processed, and display the explanation audio and video.

[0011] Further, determining at least one target explanation option template corresponding to both the arithmetic structure and the numerical range from among the preset multiple explanation option templates comprises:

[0012] According to the arithmetic structure, a plurality of processing steps are sequentially determined;

[0013] For each processing step, at least one target explanation option template corresponding to both the arithmetic structure of the processing step and the numerical range is determined from a plurality of preset explanation option templates.

[0014] Further, generating explanation audio and video according to the at least one target explanation option template and the information to be processed includes:

[0015] Determine at least one explanation option template combination of the information to be processed according to the information to be processed and at least one target explanation option template of each processing step; wherein the explanation option template combination includes at least one target explanation option template;

[0016] Determine a target option template combination according to at least one explanation option template combination;

[0017] Generate explanation audio and video based on the target option template combination.

[0018] Further, the determining of the target option template combination according to at least one explanation option template combination includes:

[0019] Determining the complexity of each combination of the explanation option templates;

[0020] Determine the explanation option template combination corresponding to the minimum complexity as the target option template combination of the information to be processed.

[0021] Furthermore, the determining the complexity of each combination of explanation option templates includes:

[0022] If the explanation option template combination includes multiple target explanation option templates, the sum of the complexities of the target explanation option templates is determined, and the sum is determined as the complexity of the explanation option template combination.

[0023] Furthermore, the explanation option template includes a topic analysis part, an arithmetic transformation part, and a calculation part;

[0024] The step of generating audio and video explanations according to the target option template combination includes:

[0025] If the target option template combination includes multiple target explanation option templates, the question analysis part, arithmetic transformation part, and calculation part of the first target explanation option template and the arithmetic transformation part and calculation part of other target explanation option templates are merged in sequence according to the information to be processed to generate explanation audio and video.

[0026] Furthermore, the display of the explanation audio and video includes:

[0027] For each frame of image in the explanation audio and video and the explanation audio corresponding to each frame of image, if it is determined that the explanation audio corresponding to the current frame of image has been played, the next frame of image is played.

[0028] In a second aspect, the present application provides a device for generating and processing audio and video of explanations, comprising:

[0029] An acquisition module, used for acquiring a request to be parsed; wherein the request to be parsed is used for requesting the generation of audio and video explanations, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed;

[0030] A first determination module is used to perform data analysis on the information to be processed to determine an arithmetic structure and a numerical range;

[0031] A second determination module is used to determine at least one target explanation option template corresponding to both the arithmetic structure and the numerical range from among a plurality of preset explanation option templates; wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the explanation rule information is determined according to the arithmetic structure and the numerical range;

[0032] A generating module, configured to generate audio and video of explanation according to the at least one target explanation option template and the information to be processed;

[0033] A display module is used to display the audio and video of the explanation.

[0034] Furthermore, the second determining module includes:

[0035] A first determining unit, configured to sequentially determine a plurality of processing steps according to the arithmetic structure;

[0036] The second determining unit is used to determine, for each processing step, at least one target explanation option template corresponding to both the arithmetic structure of the processing step and the numerical range from among a plurality of preset explanation option templates.

[0037] Furthermore, the generation module includes:

[0038] A third determining unit is used to determine at least one explanation option template combination of the information to be processed according to the information to be processed and at least one target explanation option template of each processing step; wherein the explanation option template combination includes at least one target explanation option template;

[0039] A fourth determining unit, configured to determine a target option template combination according to at least one explanation option template combination;

[0040] A generating unit is used to generate audio and video explanations according to the target option template combination.

[0041] Furthermore, the fourth determining unit includes:

[0042] A first determining subunit, used to determine the complexity of each combination of explanation option templates;

[0043] The second determination subunit is used to determine the explanation option template combination corresponding to the minimum complexity as the target option template combination of the information to be processed.

[0044] Furthermore, the first determining subunit is specifically configured to:

[0045] If the explanation option template combination includes multiple target explanation option templates, the sum of the complexities of the target explanation option templates is determined, and the sum is determined as the complexity of the explanation option template combination.

[0046] Furthermore, the explanation option template includes a topic analysis part, an arithmetic transformation part, and a calculation part;

[0047] The generating unit is specifically used for:

[0048] If the target option template combination includes multiple target explanation option templates, the question analysis part, arithmetic transformation part, and calculation part of the first target explanation option template and the arithmetic transformation part and calculation part of other target explanation option templates are merged in sequence according to the information to be processed to generate explanation audio and video.

[0049] Furthermore, the display module is specifically used for:

[0050] For each frame of image in the explanation audio and video and the explanation audio corresponding to each frame of image, if it is determined that the explanation audio corresponding to the current frame of image has been played, the next frame of image is played.

[0051] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0052] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0053] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0054] The present application provides a method, device and equipment for generating and processing audio and video of explanations, which obtains a request to be parsed; wherein the request to be parsed is used to request the generation of audio and video of explanations, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression. Data parsing is performed on the information to be processed to determine the arithmetic structure and the numerical range. Among the preset multiple explanation option templates, at least one target explanation option template corresponding to the arithmetic structure and the numerical range is determined; wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the explanation rule information is determined according to the arithmetic structure and the numerical range. Based on at least one target explanation option template and the information to be processed, the explanation audio and video is generated, and the explanation audio and video is displayed. In this scheme, the information to be processed is parsed, the arithmetic structure and the numerical range are determined, and at least one target explanation option template is determined according to the arithmetic structure and the numerical range, and the explanation audio and video is generated in time according to the at least one target explanation option template, and the explanation audio and video is displayed to the students. Therefore, for mathematical problems or their changing forms, mathematical problems can be analyzed flexibly and timely and the explanation option template can be called to generate explanation audio and video, avoiding the user from recording the explanation audio and video of each mathematical problem one by one, greatly saving recording time, and solving the technical problem of low efficiency in the generation method of explanation audio and video. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0056] Figure 1 A flowchart of a method for generating and processing audio and video of explanations provided in an embodiment of the present application;

[0057] Figure 2 A flowchart of another method for generating and processing audio and video of explanations provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of the structure of a device for generating and processing audio and video explanations provided in an embodiment of the present application;

[0059] Figure 4A schematic diagram of the structure of another device for generating and processing audio and video explanations provided in an embodiment of the present application;

[0060] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0061] Figure 6 A block diagram of an electronic device provided in an embodiment of the present application.

[0062] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0063] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0064] At present, the development of artificial intelligence technology has brought many changes to the education industry. For example, the use of virtual digital human images to replicate the image, voice, and movements of teachers to create a virtual image that students are more interested in. By inputting good scripts, courseware videos can be quickly generated. AI digital human teachers can set up artificial intelligence recognition and interactive systems to answer students' questions in a timely and accurate manner; and can be used to provide students with interactive learning experiences. For example, by introducing vivid story scenarios and visual knowledge presentation, explanation videos can allow children to understand knowledge in real scenarios, master knowledge to form self-learning paths, and achieve autonomous exploratory learning.

[0065] In one example, the explanation video is generated by recording a video in advance according to the topic, and playing it directly when the student chooses it; or, directly recording an audio and video template, and obtaining the parameters selected by the student during the explanation to generate a new explanation video and play it. However, in the prior art, since the video is recorded in advance according to the topic, the method of generating the explanation video is fixed, and the actual problems are complex and changeable. For example, there are many types of deformation methods for the multiplication and division of elementary school mathematics. At this time, it is necessary to record multiple videos in advance according to the various deformation methods, and each topic needs to record a corresponding video. The video recording workload is large, the storage volume is large, and it is difficult for users to quickly select the explanation video they want when choosing, which leads to a loss of interest in intelligent explanation. Therefore, the existing video generation method lacks flexibility and is inefficient.

[0066] The present application provides a method, device and equipment for generating and processing audio and video explanations, aiming to solve the above technical problems in the prior art.

[0067] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0068] Figure 1 A flowchart of a method for generating and processing an audio and video explanation provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0069] Step 101, obtaining a request to be parsed; wherein the request to be parsed is used to request generation of audio and video explanations, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed.

[0070] For example, the execution subject of this embodiment can be an electronic device, or a terminal device, or a device or equipment for generating and processing audio and video explanations, or other devices or equipment that can execute this embodiment, and there is no limitation to this. In this embodiment, the execution subject is introduced as an electronic device.

[0071] First, obtain the request to be resolved. The request to be resolved is used to request the generation of audio and video explanations, and the request to be resolved includes information to be processed. Specifically, the user inputs the information to be processed and initiates the request to be resolved. The way to input the information to be processed includes manually inputting the information to be processed, importing images of the information to be processed, or taking photos and uploading images of the information to be processed, etc., which are not limited. The electronic device obtains the request to be resolved, and then obtains the information to be processed in the request to be resolved. Among them, the information to be processed is an arithmetic expression, for example, the arithmetic expression is a mathematical expression to be calculated, such as 1+2.

[0072] Step 102: Perform data analysis on the information to be processed to determine the arithmetic structure and value range.

[0073] Exemplarily, the electronic device performs data analysis on the information to be processed through a preset expression analysis algorithm to determine the arithmetic structure and the numerical range. For example, the arithmetic structure in the mathematical expression 1+2 is the addition of two numbers, and the numerical range is within 10.

[0074] Step 103: Determine at least one target explanation option template corresponding to both the arithmetic structure and the numerical range from among the preset multiple explanation option templates; wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the explanation rule information is determined based on the arithmetic structure and the numerical range.

[0075] Exemplarily, abstract mathematical knowledge is classified to form preset explanation option templates, wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the audio and video templates include audio templates and / or video templates, wherein the audio template is a template for explanation voice, and the video template is a template for explanation animation. The explanation rule information represents the explanation method under different data processing strategies, and the data processing strategy of the explanation rule information is determined according to the arithmetic structure and the numerical range; different arithmetic structures correspond to corresponding explanation option templates, and further, the same arithmetic structure can correspond to multiple explanation option templates, and for multiple explanation option templates corresponding to the same arithmetic structure, each explanation option template corresponds to a different numerical range. The explanation process of each explanation option template includes a topic analysis part, an arithmetic transformation part, and a calculation part, wherein the topic analysis part is used to determine whether the information to be processed meets the preset applicable conditions of the explanation option template itself, and if it meets the applicable conditions, it means that the explanation option template can be applied to the information to be processed; the arithmetic transformation part is used to deform the information to be processed according to the explanation rule information of the explanation option template; and the calculation part is used to calculate the deformed information obtained by the deformation process to obtain the calculation result.

[0076] Specifically, for a single explanation option template, that is, a mathematical expression only needs to be calculated in one step through a single explanation option template to obtain the calculation result, and the preset complexity evaluation algorithm in the single explanation option template is directly called to determine the complexity, wherein the complexity evaluation consists of two parts: 1) Calculation complexity, for example, the complexity of addition of the target object within the preset numerical range is the first preset complexity, the complexity of a carry calculation is the second preset complexity, and the complexity of a multiplication within the table is the third preset complexity. 2) Transformation complexity, for example, the complexity of a multiplication distributive law expansion is the fourth preset complexity. After calculating the complexity, the result is generated according to the single explanation option template.

[0077] For the combination that requires multiple explanation option templates to get the final result, that is, the mathematical expression needs to go through multiple steps, and each step formula requires an explanation option template to calculate and get the calculation result. The specific process is: ① According to the priority information of the target object, determine the first step formula to be calculated. For the first step formula, try each explanation option template in turn to determine whether the information to be processed meets the preset applicable conditions of the explanation option template itself. If applicable, use the corresponding explanation option template to generate the next step formula, or use the corresponding explanation option template to directly generate the result; ② Repeat the attempt of ① for the next step formula; ③ Until the calculated formula has become a result, a set of template sequences are obtained according to the explanation option templates used for the formula of each step; ④ Continue to try to obtain many sets of template sequences; ⑤ Finally, adopt the template sequence with the lowest complexity as the target option template combination.

[0078] Among them, when calculating complexity, most explanation option templates can make the formula more "simple", while some explanation option templates make the formula more "complex", such as the expansion of the distributive law of multiplication, or the complement (considering 99 as 100-1). Therefore, the ordinary exhaustive method may cause the formula to expand infinitely and cannot generate an explanation in a short time. To address this problem, optimization is mainly carried out through pruning: 1) If the intermediate arithmetic exceeds twice the number of original arithmetic symbols, then this path is abandoned; 2) If there is already a solution under this subpath that has reached complexity A, and the complexity of the current solution has exceeded A, then this path is abandoned. 3) Explanation option templates such as factor decomposition will make heuristic choices based on other parts of the arithmetic. For example, an arithmetic expression includes the multiplication of two numbers, one of which is 25, and the other number will be split into two numbers based on 25, so that the split number multiplied by 25 is a positive integer that is easy to calculate, for example, a positive integer of 100, etc. There is no limit on the split. Among the preset multiple explanation option templates, determine at least one target explanation option template corresponding to both the arithmetic structure and the numerical range,

[0079] For example, if the arithmetic expression is a+b, the calculation result can be obtained by performing a one-step summation through a single explanation option template. According to the arithmetic structure and numerical range of a+b, a target explanation option template corresponding to a+b is determined in the preset multiple explanation option templates. At this time, the explanation rule information of the target explanation option template is an addition type and an explanation method of the corresponding numerical range, and the target explanation option template is an addition template of the corresponding numerical range. Specifically, according to the formula structure of the arithmetic expression a+b, the template to be selected is determined to be a plurality of explanation option templates of addition. For the plurality of explanation option templates of addition, there is a mapping relationship between each explanation option template and the numerical range of the arithmetic expression. First, the complexity of a+b is calculated, and the numerical range of a+b is determined according to the actual size of a and b. The target explanation option template corresponding to the numerical range is determined in the plurality of explanation option templates of addition, and the result is generated according to the target explanation option template.

[0080] Or, the arithmetic expression is a*b+c, and multiple target explanation option templates are required to obtain the calculation result. At this time, the explanation rule information of each target explanation option template is respectively a multiplication type explanation method, an addition type explanation method, a sequential calculation explanation method, etc., and the target explanation option template is a multiplication template, an addition template, a sequential calculation template, etc. Specifically, according to the arithmetic structure and the numerical range of a*b+c, the first step formula a*b is determined. For the first step formula a*b, each explanation option template is first tried in turn to determine whether a*b meets the preset applicable conditions of the multiplication template, and the next step formula (a*b=d)+c, that is, d+c, is generated using the multiplication template, or a*b meets the preset applicable conditions of the sequential calculation template, and the result is directly generated according to the sequential calculation template. For the next step formula d+c, each explanation option template is continued to be tried in turn, and the next step formula is generated using the applicable explanation option template, and so on, until the calculated formula has become a result, and multiple sets of template sequences are obtained according to the explanation option template used for each step formula.

[0081] In this step, based on the arithmetic structure of the arithmetic expression, multiple explanation option templates corresponding to the arithmetic structure are determined from multiple preset explanation option templates, and the multiple explanation option templates are templates corresponding to different numerical ranges. Further, based on the numerical range of the arithmetic expression, at least one target explanation option template corresponding to the numerical range is determined from multiple explanation option templates with the same arithmetic structure.

[0082] Step 104: Generate explanation audio and video according to at least one target explanation option template and the information to be processed, and display the explanation audio and video.

[0083] For example, each target explanation option template contains some video and voice clips. In order to form a holistic and coherent explanation, these clips need to be merged. Since each target explanation option template contains three-segment explanations: the question analysis part, the arithmetic transformation part, and the calculation part, when merging, according to the order of the target explanation option templates, the question analysis part, arithmetic transformation part, and calculation part of the first target explanation option template are retained, and the arithmetic transformation part and calculation part of the subsequent target explanation option templates are retained. The retained parts are merged in order to generate the explanation audio and video, and the fused explanation audio and video are displayed.

[0084] In an embodiment of the present application, a request to be parsed is obtained; wherein, the request to be parsed is used to request the generation of audio and video of explanation, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed. Data parsing is performed on the information to be processed to determine the arithmetic structure and the numerical range. Among the preset multiple explanation option templates, at least one target explanation option template corresponding to both the arithmetic structure and the numerical range is determined; wherein, the explanation option template in the preset explanation option template is an audio and video template of different types of explanation rule information, and the explanation rule information is determined according to the arithmetic structure and the numerical range. Based on at least one target explanation option template and the information to be processed, the audio and video of explanation is generated, and the audio and video of explanation is displayed. In this scheme, the information to be processed is parsed, the arithmetic structure and the numerical range are determined, and at least one target explanation option template is determined according to the arithmetic structure and the numerical range, and the audio and video of explanation is generated in time according to at least one target explanation option template, and the audio and video of explanation is displayed to students. Therefore, for mathematical problems or their changing forms, mathematical problems can be analyzed flexibly and timely and the explanation option template can be called to generate explanation audio and video, avoiding the user from recording the explanation audio and video of each mathematical problem one by one, greatly saving recording time, and solving the technical problem of low efficiency in the generation method of explanation audio and video.

[0085] Figure 2 A flowchart of another method for generating and processing audio and video for explaining provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes:

[0086] Step 201, obtaining a request to be parsed; wherein the request to be parsed is used to request generation of audio and video explanations, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed.

[0087] For example, this step can be referred to Figure 1 Step 101 in the above is not described in detail.

[0088] Step 202: Perform data analysis on the information to be processed to determine the arithmetic structure and value range.

[0089] Exemplarily, the information to be processed is the arithmetic expression to be parsed, and the arithmetic expression is parsed to determine the arithmetic structure and numerical range. The arithmetic structure refers to the operation method determined according to the mathematical symbols between the numbers in the arithmetic expression; the numerical range is the approximate range determined according to the size of the numbers in the arithmetic expression. For example, the arithmetic structure in the mathematical expression 1+2 is the addition of two numbers, and the numerical range is within 10.

[0090] Step 203: Determine multiple processing steps in sequence according to the arithmetic structure.

[0091] Exemplarily, the electronic device may determine a plurality of processing steps in sequence according to an arithmetic structure, that is, determine the formula of each step in sequence.

[0092] Step 204: For each processing step, determine at least one target explanation option template corresponding to both the arithmetic structure and the numerical range of the processing step from among a plurality of preset explanation option templates.

[0093] Exemplarily, for each processing step, the electronic device can determine, according to the arithmetic structure of the processing step, a plurality of explanation option templates corresponding to the arithmetic structure of the processing step from among the preset plurality of explanation option templates, each of which corresponds to a different numerical range. Then, according to the numerical range of the processing step, at least one target explanation option template corresponding to the numerical range of the processing step is searched from among the plurality of explanation option templates corresponding to the arithmetic structure of the processing step. For example, if the current processing step corresponds to only one target explanation option template, the calculation result can be directly output according to the one target explanation option template that meets the requirements.

[0094] If the current processing step corresponds to multiple target explanation option templates, it means that the step can have multiple calculation methods. If the first calculation method and the second calculation method both obtain calculation results and complexity, when calculating the third calculation method, it is found that the complexity of the third calculation method has exceeded the complexity of the first calculation method and the second calculation method. At this time, the third calculation method can be abandoned. At this time, the current step corresponds to two target explanation option templates, that is, two calculation branches are obtained, and each calculation branch is the formula for the next step. Similarly, for each formula of the next step, continue to determine at least one target explanation option template corresponding to the arithmetic structure and numerical range of the formula until each branch obtains a result. Alternatively, there is no need to exclude branches with higher complexity based on complexity during the calculation process, and there is no limitation on this.

[0095] Step 205: Determine at least one explanation option template combination for the information to be processed based on the information to be processed and at least one target explanation option template for each processing step; wherein the explanation option template combination includes at least one target explanation option template.

[0096] Exemplarily, each processing step includes at least one target explanation option template, and the electronic device can obtain at least one explanation option template combination of the information to be processed according to the at least one target explanation option template of each processing step. Further, if there are multiple processing steps, each explanation option template combination includes multiple target explanation option templates; if there is one processing step, each explanation option template combination includes multiple or one target explanation option template.

[0097] Step 206: Determine a target option template combination according to at least one explanation option template combination.

[0098] In one example, step 206 includes: determining the complexity of each explanation option template combination; and determining the explanation option template combination corresponding to the minimum complexity as the target option template combination of the information to be processed.

[0099] In one example, "determining the complexity of each explanation option template combination" includes: if the explanation option template combination includes multiple target explanation option templates, determining the sum of the complexities of the respective target explanation option templates, and determining the sum as the complexity of the explanation option template combination.

[0100] Exemplarily, if the explanation option template combination includes multiple target explanation option templates, the electronic device can determine the complexity of each target explanation option template according to a preset complexity evaluation algorithm, that is, determine the complexity of each processing step, and then determine the sum of the complexity of each target explanation option template as the final complexity of the explanation option template combination, so that the complexity of multiple explanation option template combinations can be obtained. The complexity of each explanation option template combination is compared to determine the minimum complexity, and the explanation option template combination corresponding to the minimum complexity is determined as the target option template combination of the information to be processed.

[0101] Step 207: Generate explanation audio and video according to the target option template combination.

[0102] In one example, step 207 includes: the explanation option template includes a topic analysis part, an arithmetic transformation part, and a calculation part; if the target option template combination includes multiple target explanation option templates, then according to the information to be processed, the topic analysis part, arithmetic transformation part, and calculation part of the first target explanation option template, and the arithmetic transformation part and calculation part of other target explanation option templates are merged in sequence to generate explanation audio and video.

[0103] Exemplarily, the target option template combination includes multiple explanation option templates, each of which contains some video and voice clips. In order to form a holistic and coherent explanation, these clips need to be merged. Since each explanation option template contains a three-segment explanation: a topic analysis part, an arithmetic transformation part, and a calculation part, when merging, according to the order of the explanation option templates, the topic analysis part, arithmetic transformation part, and calculation part of the first explanation option template are retained, and the arithmetic transformation part and calculation part of the subsequent explanation option template are retained. The retained parts are merged in order to generate the fused explanation audio and video.

[0104] Step 208: for each frame of image in the explanation audio and video and the explanation audio corresponding to each frame of image, if it is determined that the explanation audio corresponding to the current frame of image has been played, then the next frame of image is played.

[0105] For example, since the image and the explanation audio are dynamically generated, different mathematical expressions may lead to inconsistent progress of the image and the explanation audio. Therefore, for each frame of the image in the explanation audio and video and the explanation audio corresponding to each frame of the image, if the explanation time of the explanation audio corresponding to the current frame of the image is longer than the playback time of the current frame of the image, both the explanation audio voice and the current frame of the image need to be played before the next frame of the image can be played.

[0106] In an embodiment of the present application, a request to be parsed is obtained; wherein the request to be parsed is used to request the generation of an explanation audio and video, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed. Data is parsed on the information to be processed to determine the arithmetic structure and the numerical range. According to the arithmetic structure, multiple processing steps are determined in sequence. For each processing step, at least one target explanation option template corresponding to the arithmetic structure and the numerical range of the processing step is determined from multiple preset explanation option templates. According to the information to be processed and at least one target explanation option template of each processing step, at least one explanation option template combination of the information to be processed is determined; wherein the explanation option template combination includes at least one target explanation option template. According to at least one explanation option template combination, a target option template combination is determined. According to the target option template combination, an explanation audio and video is generated. For each frame image in the explanation audio and video and the explanation audio corresponding to each frame image, if it is determined that the explanation audio corresponding to the current frame image has been played, the next frame image is played. Therefore, for mathematical problems or their changing forms, mathematical problems can be analyzed flexibly and timely and the explanation option template can be called to generate explanation audio and video, avoiding the user from recording the explanation audio and video of each mathematical problem one by one, greatly saving recording time, and solving the technical problem of low efficiency in the generation method of explanation audio and video.

[0107] In one embodiment, special training of mathematical thinking can also be set in the Internet application app, such as Sudoku, clever calculation, etc. Students enter the special training interface through the special training entrance to obtain special training. The specific process and method are as follows:

[0108] 1): Students choose one of several special training options, such as arithmetic;

[0109] 2): Students choose the difficulty level of the special training (e.g. DL1 to DL20, with increasing difficulty);

[0110] 3): Students receive mathematical expressions for special training; (If students answer by themselves, the process ends; if students think the questions are difficult and need explanation, they proceed to the next step)

[0111] 4): If the student clicks the explain button, the app receives the request to be parsed;

[0112] 5): The app directly parses the current mathematical expression, determines the arithmetic structure and numerical range of the current mathematical expression, and determines at least one target explanation option template corresponding to the arithmetic structure and numerical range from a plurality of preset explanation option templates;

[0113] 6): Based on at least one target explanation option template and mathematical expression, the app generates and displays the explanation audio and video to students.

[0114] Furthermore, after the audio and video of the explanation is played, students' feedback and comments on the audio and video can be received, and special mathematical expressions related to the current audio and video can be recommended to students for training or to test the students' understanding and mastery, and targeted audio and video explanations can be recommended for intensive learning or special training. The specific process and method are as follows:

[0115] 1): If the student completes the recommended special mathematical expressions related to the current audio and video, the app receives the student's thinking training results, including the correctness of the questions, the speed and time of answering questions, the error categories, etc.;

[0116] 2): The app comprehensively evaluates students’ knowledge mastery, proficiency, and difficulty level that students can adapt to based on the results of thinking training;

[0117] 3): The app determines the audio and video explanations and special training recommendations corresponding to the students’ current level based on the students’ error categories and their knowledge mastery level;

[0118] 4): Push special training and audio and video explanations related to error categories to students.

[0119] Figure 3A schematic diagram of a structure of a device for generating and processing audio and video explanations provided in an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0120] The acquisition module 31 is used to acquire the request to be parsed; wherein the request to be parsed is used to request the generation of audio and video explanations, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed.

[0121] The first determination module 32 is used to perform data analysis on the information to be processed and determine the arithmetic structure and value range.

[0122] The second determination module 33 is used to determine at least one target explanation option template corresponding to both the arithmetic structure and the numerical range from among multiple preset explanation option templates; wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the explanation rule information is determined based on the arithmetic structure and the numerical range.

[0123] The generating module 34 is used to generate audio and video of explanation according to at least one target explanation option template and the information to be processed.

[0124] The display module 35 is used to display the audio and video of the explanation.

[0125] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0126] Figure 4 A schematic diagram of the structure of another apparatus for generating and processing audio and video explanations provided in an embodiment of the present application, Figure 3 Based on the embodiment shown, Figure 4 As shown, the second determination module 33 includes:

[0127] The first determining unit 331 is used to determine a plurality of processing steps in sequence according to the arithmetic structure.

[0128] The second determining unit 332 is used to determine, for each processing step, at least one target explanation option template corresponding to both the arithmetic structure and the numerical range of the processing step from among a plurality of preset explanation option templates.

[0129] In one example, the generating module 34 includes:

[0130] The third determining unit 341 is used to determine at least one explanation option template combination of the information to be processed according to the information to be processed and at least one target explanation option template of each processing step; wherein the explanation option template combination includes at least one target explanation option template.

[0131] The fourth determining unit 342 is used to determine a target option template combination according to at least one explanation option template combination.

[0132] The generating unit 343 is used to generate the explanation audio and video according to the target option template combination.

[0133] Further, the fourth determining unit 342 includes:

[0134] The first determining subunit 3421 is used to determine the complexity of each explanation option template combination.

[0135] The second determining subunit 3422 is used to determine the explanation option template combination corresponding to the minimum complexity as the target option template combination of the information to be processed.

[0136] In one example, the first determining subunit 3421 is specifically configured to:

[0137] If the explanation option template combination includes multiple target explanation option templates, the sum of the complexities of the target explanation option templates is determined, and the sum is determined as the complexity of the explanation option template combination.

[0138] In one example, the explanation option template includes a topic analysis part, an arithmetic transformation part, and a calculation part;

[0139] The generating unit 343 is specifically used for:

[0140] If the target option template combination includes multiple target explanation option templates, the question analysis part, arithmetic transformation part, and calculation part of the first target explanation option template and the arithmetic transformation part and calculation part of other target explanation option templates are merged in sequence according to the information to be processed to generate explanation audio and video.

[0141] In one example, the display module 35 is specifically used to:

[0142] For each frame of image in the explanation audio and video and the explanation audio corresponding to each frame of image, if it is determined that the explanation audio corresponding to the current frame of image has been played, the next frame of image is played.

[0143] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0144] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the electronic device includes: a memory 51 and a processor 52.

[0145] The memory 51 stores a computer program that can be executed on the processor 52 .

[0146] The processor 52 is configured to execute the method provided in the above embodiment.

[0147] The electronic device further includes a receiver 53 and a transmitter 54. The receiver 53 is used to receive instructions and data sent by an external device, and the transmitter 54 is used to send instructions and data to the external device.

[0148] Figure 6 This is a block diagram of an electronic device provided in an embodiment of the present application. The electronic device may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0149] The device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0150] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0151] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0152] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.

[0153] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0154] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0155] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0156] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, the sensor assembly 614 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600 and the temperature change of the device 600. The sensor assembly 614 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0157] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0158] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0159] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by the processor 620 of the device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0160] An embodiment of the present application also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method provided by the above embodiment.

[0161] An embodiment of the present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0162] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0163] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating and processing audio and video of explanation, characterized in that: include: Obtaining a request to be parsed; wherein the request to be parsed is used to request the generation of audio and video explanations, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed; Performing data analysis on the information to be processed to determine an arithmetic structure and a numerical range; Determine at least one target explanation option template corresponding to both the arithmetic structure and the numerical range from among a plurality of preset explanation option templates; wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the explanation rule information is determined according to the arithmetic structure and the numerical range; Generate explanation audio and video according to the at least one target explanation option template and the information to be processed, and display the explanation audio and video.

2. The method according to claim 1, characterized in that The step of determining at least one target explanation option template corresponding to both the arithmetic structure and the numerical range from among the preset multiple explanation option templates comprises: According to the arithmetic structure, a plurality of processing steps are sequentially determined; For each processing step, at least one target explanation option template corresponding to both the arithmetic structure of the processing step and the numerical range is determined from a plurality of preset explanation option templates.

3. The method according to claim 2, characterized in that The step of generating explanation audio and video according to the at least one target explanation option template and the information to be processed includes: Determine at least one explanation option template combination of the information to be processed according to the information to be processed and at least one target explanation option template of each processing step; wherein the explanation option template combination includes at least one target explanation option template; Determine a target option template combination according to at least one explanation option template combination; Generate explanation audio and video based on the target option template combination.

4. The method according to claim 3, characterized in that The step of determining a target option template combination according to at least one explanation option template combination comprises: Determining the complexity of each combination of the explanation option templates; Determine the explanation option template combination corresponding to the minimum complexity as the target option template combination of the information to be processed.

5. The method according to claim 4, characterized in that Determining the complexity of each combination of explanation option templates includes: If the explanation option template combination includes multiple target explanation option templates, the sum of the complexities of the target explanation option templates is determined, and the sum is determined as the complexity of the explanation option template combination.

6. The method according to claim 3, characterized in that The explanation option template includes a topic analysis part, an arithmetic transformation part, and a calculation part; The step of generating audio and video explanations according to the target option template combination includes: If the target option template combination includes multiple target explanation option templates, the question analysis part, arithmetic transformation part, and calculation part of the first target explanation option template and the arithmetic transformation part and calculation part of other target explanation option templates are merged in sequence according to the information to be processed to generate explanation audio and video.

7. The method according to any one of claims 1 to 6, characterized in that: The said display and explanation audio and video include: For each frame of image in the explanation audio and video and the explanation audio corresponding to each frame of image, if it is determined that the explanation audio corresponding to the current frame of image has been played, the next frame of image is played.

8. A device for generating and processing audio and video of explanations, characterized in that: include: An acquisition module, used for acquiring a request to be parsed; wherein the request to be parsed is used for requesting the generation of audio and video explanations, and the request to be parsed includes information to be processed, and the information to be processed is an arithmetic expression to be parsed; A first determination module is used to perform data analysis on the information to be processed to determine an arithmetic structure and a numerical range; A second determination module is used to determine at least one target explanation option template corresponding to both the arithmetic structure and the numerical range from among a plurality of preset explanation option templates; wherein the explanation option templates in the preset explanation option templates are audio and video templates of different types of explanation rule information, and the explanation rule information is determined according to the arithmetic structure and the numerical range; A generating module, configured to generate audio and video of explanation according to the at least one target explanation option template and the information to be processed; A display module is used to display the audio and video of the explanation.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images