Dictation detection method, electronic device and storage medium

Abstract

The embodiments of the present invention relate to the field of educational technology, and disclose a dictation detection method, an electronic device, and a storage medium. The method includes: during the dictation process, if the electronic device has finished reading the pronunciation of any dictation word in the dictation audio, photographing the writing page after the user writes according to the pronunciation of the dictation word; judging whether the pronunciation of the dictation word belongs to the pronunciation of the last dictation word in the dictation audio, if so, saving the writing page; if not, judging whether the user has turned the page, if so, saving the writing page; when the dictation is terminated, performing dictation detection on all the saved writing pages. The implementation of the embodiments of the present invention can intelligently detect multiple writing pages to improve the accuracy of dictation detection.

Description

Technical Field

[0001] The present invention relates to the field of educational technology, and in particular to a dictation detection method, electronic equipment and storage medium. Background Art

[0002] Currently, student users often use electronic devices such as reading machines, tutoring machines, learning machines or learning tablets to practice dictation after class. After the user completes the dictation practice, the electronic device will collect the user's writing page, identify the dictation answer written by the user on the writing page according to the dictation content broadcast by the electronic device, and detect whether the dictation answer is correct.

[0003] In practice, it is found that since the number of dictation words broadcast by the electronic device in each dictation exercise is not fixed, sometimes the dictation answers written by the user according to the dictation words in a dictation exercise do not fill a full page, and sometimes the dictation answers written by the user in a dictation exercise are written on several pages. However, after the user completes the dictation exercise, the electronic device often directly collects the last page written by the user for dictation detection, resulting in inaccurate detection. Alternatively, the user needs to manually take multiple photos and collect to complete all the pages written by the user, but this method wastes too much time and energy of the user. Summary of the invention

[0004] In view of the above-mentioned defects, the embodiments of the present invention disclose a dictation detection method, an electronic device and a storage medium, which can intelligently detect multiple written pages to improve the accuracy of dictation detection.

[0005] A first aspect of an embodiment of the present invention discloses a dictation detection method, comprising:

[0006] During the dictation process, if the electronic device has finished reading the pronunciation of any dictation word in the dictation audio, a writing page written by the user according to the pronunciation of the dictation word is photographed;

[0007] Determine whether the pronunciation of the dictated word belongs to the pronunciation of the last dictated word in the dictation audio, and if so, save the writing page;

[0008] If not, determining whether the user has turned the page, and if so, saving the written page;

[0009] When dictation is terminated, all saved written pages are checked for dictation.

[0010] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the determining whether the user performs a page turning action includes:

[0011] Determine whether all writing areas on the writing page have written content;

[0012] If there is written content in each of the writing areas, it is determined that the user has performed a page turning action.

[0013] As an optional implementation manner, in the first aspect of the embodiment of the present invention, it also includes:

[0014] If not all the writing areas are written with writing content, a plurality of frames of user images are captured at a preset frequency within a preset time period;

[0015] Extracting a first text region outline of a first frame of user images among the plurality of frames of user images, and extracting a second text region outline of a last frame of user images among the plurality of frames of user images;

[0016] Calculating the similarity between the first text area outline and the second text area outline;

[0017] If the similarity is less than a preset threshold, it is determined that the user has performed a page turning action.

[0018] As an optional implementation, in the first aspect of the embodiment of the present invention, after capturing and obtaining a plurality of frames of user images at a preset frequency within a preset time period, the method further includes:

[0019] Extracting a hand region sub-image from the user image, and judging whether the user's hand performs a preset gesture operation according to the extracted hand region sub-images of several frames;

[0020] If so, and the similarity is less than a preset threshold, it is determined that the user has performed a page turning action.

[0021] As an optional implementation, in the first aspect of the embodiment of the present invention, the step of performing dictation detection on all stored written pages includes:

[0022] Get the shooting time of each of the saved writing pages;

[0023] Sorting all the written pages according to the chronological order of the shooting moments;

[0024] According to the standard answers corresponding to the dictation words, the sorted writing pages are corrected one by one.

[0025] A second aspect of an embodiment of the present invention discloses an electronic device, including:

[0026] A photographing unit, used for photographing a writing page written by a user according to the pronunciation of any dictation word in the dictation audio after the electronic device has finished reading the pronunciation of the dictation word during the dictation process;

[0027] A first judgment unit, used to judge whether the pronunciation of the dictation word belongs to the pronunciation of the last dictation word in the dictation audio;

[0028] a saving unit, configured to save the written page when the judgment result of the first judgment unit is yes;

[0029] A second judgment unit, configured to judge whether the user has performed a page turning action when the judgment result of the first judgment unit is no;

[0030] The storage unit is further configured to store the written page when the second determination unit determines that the user has performed a page turning action;

[0031] The detection unit is used to perform dictation detection on all the saved writing pages when the dictation is terminated.

[0032] As an optional implementation manner, in a second aspect of the embodiment of the present invention, the second determination unit includes:

[0033] A judging subunit, configured to judge whether all writing areas on the writing page have written contents when the judgment result of the first judging unit is no;

[0034] The determination subunit is used to determine that the user has performed a page turning action when the determination subunit determines that each writing area has written content.

[0035] As an optional implementation, in the second aspect of the embodiment of the present invention, the shooting unit is further configured to shoot a plurality of frames of user images at a preset frequency within a preset time period when the judging subunit determines that not all the writing areas have written content;

[0036] And, the electronic device further includes:

[0037] An extraction unit is used to extract a first text area contour of a first frame of user image among the plurality of frames of user image; and to extract a second text area contour of a last frame of user image among the plurality of frames of user image;

[0038] A calculation unit, used for calculating the similarity between the first text area outline and the second text area outline;

[0039] The determination subunit is further configured to determine that the user has performed a page turning action when the similarity is less than a preset threshold.

[0040] As an optional implementation, in the second aspect of the embodiment of the present invention, the extraction unit is further configured to extract a hand region sub-image from the user image after the shooting unit obtains a plurality of frames of user images at a preset frequency within a preset time length;

[0041] The judging subunit is further used to judge whether the user's hand performs a preset gesture operation based on the hand region sub-images extracted from the plurality of frames;

[0042] The determination subunit is specifically configured to determine that the user has performed a page turning action when the determination subunit determines that the user's hand performs a preset gesture operation and the similarity is less than a preset threshold.

[0043] As an optional implementation, in the second aspect of the embodiment of the present invention, the detection unit includes:

[0044] The acquisition subunit is used to acquire the shooting time of each of all the saved writing pages when the dictation is terminated;

[0045] A sorting subunit, used for sorting all the written pages according to the chronological order of the shooting moments;

[0046] The correction subunit is used to correct the sorted writing pages one by one according to the standard answers corresponding to the dictation words.

[0047] A third aspect of an embodiment of the present invention discloses an electronic device, including:

[0048] A memory storing executable program code;

[0049] a processor coupled to the memory;

[0050] The processor calls the executable program code stored in the memory to execute a dictation detection method disclosed in the first aspect of an embodiment of the present invention.

[0051] A fourth aspect of the embodiments of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute a dictation detection method disclosed in the first aspect of the embodiments of the present invention. The computer-readable storage medium includes a ROM / RAM, a magnetic disk or an optical disk, etc.

[0052] A fifth aspect of an embodiment of the present invention discloses a computer program product. When the computer program product is run on a computer, the computer is enabled to execute part or all of the steps of any one of the methods of the first aspect.

[0053] A sixth aspect of an embodiment of the present invention discloses an application publishing platform, which is used to publish a computer program product. When the computer program product runs on a computer, the computer executes part or all of the steps of any one method of the first aspect.

[0054] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0055] In an embodiment of the present invention, during the dictation process, if the electronic device has finished reading the pronunciation of any dictated word in the dictation audio, a writing page written by the user according to the pronunciation of the dictated word is photographed, and it is determined whether the pronunciation of the dictated word belongs to the pronunciation of the last dictated word in the dictation audio. If so, the writing page is saved; if not, and it is determined that the user has turned the page, the writing page is saved, and then dictation detection is performed on all the saved writing pages when the dictation is terminated. Multiple writing pages can be intelligently detected to improve the accuracy of dictation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 is a flow chart of a dictation detection method disclosed in an embodiment of the present invention;

[0058] Figure 2 is a flow chart of another dictation detection method disclosed in an embodiment of the present invention;

[0059] Figure 3 is a structural schematic diagram of an electronic device disclosed in an embodiment of the present invention;

[0060] Figure 4 is a schematic diagram of the structure of another electronic device disclosed in an embodiment of the present invention;

[0061] Figure 5 is a structural schematic diagram of another electronic device disclosed in an embodiment of the present invention;

[0062] Figure 6 This is an example diagram of a photographing process of an electronic device photographing a written page disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present invention and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0065] In the embodiments of the present invention, the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inside", "outside", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. These terms are mainly used to better describe the present invention and its embodiments, and are not used to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation. Moreover, in addition to being used to indicate orientations or positional relationships, some of the above terms may also be used to express other meanings. For example, the term "upper" may also be used to indicate a certain dependency or connection relationship in some cases. For those of ordinary skill in the art, the specific meanings of these terms in the present invention can be understood according to specific circumstances.

[0066] In addition, the terms "install", "set", "install", "connect", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be an internal connection between two devices, elements, or components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0067] The embodiment of the present invention discloses a dictation detection method, an electronic device and a storage medium, which can intelligently detect multiple written pages to improve the accuracy of dictation detection.

[0068] The method shown in the embodiment of the present invention is applicable to electronic devices such as tutoring machines, learning machines, point reading machines, learning tablets or learning computers. The operating systems of various electronic devices include but are not limited to Android operating system, IOS operating system, Symbian operating system, Black Berry operating system, Windows Phone 8 operating system, etc., which are not limited in the embodiment of the present invention. The execution subject of the embodiment of the present invention is described by taking electronic devices as an example, and it should be understood that it should not constitute any limitation to the present invention. The following is a detailed description in conjunction with the accompanying drawings.

[0069] Embodiment 1

[0070] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a dictation detection method disclosed in an embodiment of the present invention. Figure 1 As shown, the dictation detection method may include the following steps:

[0071] 101. During the dictation process, if the electronic device has finished reading the pronunciation of any dictation word in the dictation audio, the electronic device captures a writing page after the user writes according to the pronunciation of the dictation word.

[0072] In the embodiment of the present invention, the dictation words include but are not limited to any content such as Chinese characters, Chinese phonetic letters, English phrases, English words, English letters, English phonetic symbols, chemical element names and symbols, musical solfeggios, etc., and the present invention does not make specific limitations here. The musical solfeggios refer to content such as do, re, mi, fa, so, la, si, etc.

[0073] Optionally, after the electronic device has finished reading the pronunciation of any dictated word, it can obtain a sensor signal detected by a sensor device set in the electronic device within a specified time period, and identify whether the user has finished writing according to the pronunciation of the dictated word based on the sensor signal. If the writing is completed, the writing page after the user writes according to the pronunciation of the dictated word is captured. Among them, the sensor device includes but is not limited to an infrared sensor device, an image sensor device or an ultrasonic sensor device. Through this embodiment, the moment when the user finishes writing according to the pronunciation of the dictated word can be accurately identified, making the electronic device more intelligent.

[0074] In the embodiment of the present invention, the shooting module for shooting the written page can be arranged on a side of the electronic device equipped with a display screen, and the side is equipped with a reflective device, and the mirror surface of the reflective device forms a preset angle with the lens surface of the shooting module. Figure 6 , Figure 6 FIG. 1 is an example diagram of a photographing process of an electronic device photographing a written page disclosed in an embodiment of the present invention. Figure 6As shown, the electronic device controls the shooting module to shoot the mirror image in the reflective device as a writing page as follows: the electronic device 10 in the figure can be provided with a shooting module 20, and the shooting module 20 is used to shoot to obtain the writing page; a reflecting device 30 (for example, a reflector, a prism or a convex lens, etc.) can also be provided in front of the shooting module 20, and the reflecting device 30 is used to change the optical path of the shooting module, so that the shooting module 20 shoots the carrier 40 to obtain the writing page. By using the imaging of the carrier 40 in the reflective device 30 obtained by shooting the shooting module 20 of the electronic device 10, without manually changing the placement of the electronic device 10, the shooting process can be simplified and the shooting efficiency can be improved. Among them, the carrier 40 can be specifically a textbook, a famous literary novel or a teaching material exercise book placed on the desktop, etc., which is not specifically limited in the embodiment of the present invention.

[0075] It should be noted that the dictation audio may specifically include the pronunciation of several dictation words.

[0076] Optionally, the plurality of dictation words may be determined by the electronic device according to a selection instruction received from a user input on a dictation selection interface before the electronic device enters the dictation mode.

[0077] Specifically, before executing step 101, the following steps may also be performed: when the electronic device enters the dictation mode, the electronic device displays a dictation selection interface on an electronic screen provided on the electronic device for the user to select dictation words, and receives a selection instruction input by the user on the dictation selection interface, determines a number of dictation words according to the selection instruction, and then retrieves the audio data corresponding to each dictation word, and sorts the several audio data in the order selected by the user to generate dictation audio for broadcasting during the dictation process.

[0078] Through this implementation, based on the user's dictation intention, user-personalized dictation words can be generated for the user to practice dictation, which can better meet the user's dictation needs and increase user stickiness.

[0079] Further optionally, a specific implementation of the electronic device entering the dictation mode may be that when a dictation wake-up voice input by the user is received, the electronic device displays a face recognition interface on the electronic screen for the user to perform a face recognition operation; when the user's face recognition is successful and the user performs a face recognition operation with a preset expression, the electronic device is controlled to enter the dictation mode.

[0080] The preset expression may specifically be an expression preset corresponding to the dictation mode.

[0081] Through this implementation, the fun of the dictation process started by the user can be increased, and the user can be helped to mobilize positive emotions to perform dictation learning, so as to improve learning efficiency.

[0082] For example, suppose that it is pre-set that the "sticking out tongue" expression corresponds to the point reading mode, the "single closed eyes" expression corresponds to the dictation mode, the "making a face" expression corresponds to the recitation mode, or the "puffing cheeks" expression corresponds to the intelligent question and answer mode, etc. Then, when the user's face recognition is successfully identified, and the user performs the face recognition operation with the "single closed eyes" expression, the electronic device can be controlled to enter the dictation mode. Alternatively, when the user's face recognition is successfully identified, and the user performs the face recognition operation with the "sticking out tongue" expression, the electronic device can be controlled to enter the point reading mode. In this way, interesting human-computer interaction can be performed while the electronic device mode is switched to increase user stickiness.

[0083] It is understandable that when a user performs a face recognition operation, the electronic device will obtain corresponding images or pictures for face recognition. Based on face recognition technology, facial feature information can be extracted from the images or pictures, and the user's expression can be recognized based on the facial feature information.

[0084] 102. The electronic device determines whether the pronunciation of the dictated word is the pronunciation of the last dictated word in the dictation audio. If not, execute step 103; if yes, execute step 104.

[0085] It should be noted that if the pronunciation of the dictated word is the pronunciation of the last dictated word in the dictation audio, it means that the dictation is about to end, and the currently captured writing page is the last writing page written by the user during this dictation process, so it is saved. If the pronunciation of the dictated word does not belong to the pronunciation of the last dictated word in the dictation audio, it means that the dictation is still in progress, and the user may turn the page of the current unfilled page for various reasons (such as on impulse). In order to accurately record all the writing pages written by the user during this dictation process, when it is determined that the user has turned the page, the currently captured writing page is saved as the middle page of this dictation process.

[0086] 103. The electronic device determines whether the user has turned the page. If so, execute step 104; otherwise, end this process.

[0087] 104. The electronic device saves the written page.

[0088] 105. When the dictation is terminated, the electronic device performs a dictation check on all the saved written pages.

[0089] As an optional implementation, the electronic device may save the shooting time of the written page while saving the written page. Therefore, step 105 may include the following steps S1 to S3:

[0090] S1. Obtain the shooting time of each of all the saved written pages.

[0091] S2. Sort all written pages according to the chronological order of shooting time.

[0092] S3. Correct the sorted writing pages one by one according to the standard answers corresponding to the dictation words.

[0093] Through this implementation, all written pages are sorted according to the shooting time and then corrected one by one, which can improve the efficiency of dictation detection.

[0094] As another optional implementation, step S3 may specifically include:

[0095] Extracting a number of written words formed by the user from each sorted written page in turn, obtaining a dictated word corresponding to each written word, and obtaining a standard answer corresponding to each written word, wherein the standard answer corresponds to the dictated word one by one;

[0096] If the formed character shape of any written word does not match the standard answer corresponding to the written word, mark the written word as an incorrect word;

[0097] If the formed character shape of any written word matches the standard answer corresponding to the written word, and the writing time of the written word reaches a preset threshold, the written word is marked as a correct word;

[0098] If the formed character shape of any written word matches the standard answer corresponding to the written word, and the writing time of the written word does not reach a preset threshold, the written word is marked as a mastered word.

[0099] By implementing this embodiment, the user's written words can be corrected into three words with different mastery levels, namely wrong words, correct words and mastered words according to the formed character shape and writing time. This can help the user understand the dictation test results in a hierarchical manner, so as to further consolidate the practice and thus improve the user's dictation learning effect.

[0100] It can be seen that implementation Figure 1 The described method, during the dictation process, if the electronic device has finished reading the pronunciation of any dictation word in the dictation audio, a writing page written by the user according to the pronunciation of the dictation word is photographed, and it is determined whether the pronunciation of the dictation word belongs to the pronunciation of the last dictation word in the dictation audio. If so, the writing page is saved; if not, and it is determined that the user has turned the page, the writing page is saved, and then dictation detection is performed on all the saved writing pages when the dictation is terminated. Multiple writing pages can be intelligently detected to improve the accuracy of dictation detection.

[0101] Embodiment 2

[0102] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of another dictation detection method disclosed in an embodiment of the present invention. Figure 2 As shown, the dictation detection method may include the following steps:

[0103] 201. During the dictation process, if the electronic device has finished reading the pronunciation of any dictation word in the dictation audio, the electronic device captures a writing page after the user writes according to the pronunciation of the dictation word.

[0104] 202. The electronic device determines whether the pronunciation of the dictated word is the pronunciation of the last dictated word in the dictation audio. If not, execute step 203; if yes, go to step 209.

[0105] 203 , the electronic device determines whether all writing areas on the writing page have written content. If not, execute step 204 ; if so, go to step 208 .

[0106] Among them, the writing area includes several single-word areas, and the single-word area refers to an area sufficient for the user to write a single word or a word. For example, assuming that the user is dictating new words in the language, the single-word area can be a grid on the exercise book. For another example, assuming that the user is dictating English words or letters, the single-word area can be an area on the exercise book for the user to write a word or a letter. Therefore, the writing area can be an area including one or more single-word areas, and the present invention is not specifically limited here.

[0107] 204. The electronic device captures a plurality of frames of user images at a preset frequency within a preset time period.

[0108] It should be noted that while the electronic device is executing the operation of capturing several frames of user images at a preset frequency, it may have already started reading the pronunciation of the next dictation word, or it may not have started reading the pronunciation of the next dictation word yet. The specific implementation process may depend on the actual situation.

[0109] It can be understood that the specific implementation process is related to the broadcast interval and the preset duration of the dictation audio. If the broadcast interval is shorter than the preset duration, the electronic device can start reading the pronunciation of the next dictated word while executing the operation of capturing and obtaining several frames of user images at a preset frequency; if the broadcast interval is longer than the preset duration, the electronic device may not have started reading the pronunciation of the next dictated word after completing the operation of capturing and obtaining several frames of user images at a preset frequency.

[0110] The preset duration may be 3 seconds, 2.3 seconds, or 5 seconds. The specific value may be set by the developer according to actual needs and is not specifically limited here.

[0111] As an optional implementation, before executing step 201, the electronic device may further perform the following steps: obtaining a number of dictation words that meet the current time period in the user's learning plan, and retrieving the audio data corresponding to each dictation word; obtaining the reading record data of the user's last reading and learning, and determining the user's voice characteristics according to the reading record data; generating the dictation audio with a preset speech synthesis algorithm according to the audio data, the preset broadcast interval and the voice characteristics. Among them, the preset speech synthesis algorithm may be a linear predictive coding method (LPC) or a pitch synchronization superposition method (PSOLA), etc.; the user's voice characteristics are used to describe the stored voice characteristics of the user who read aloud last time, for example, the voice characteristic identifier under emotions such as happiness, excitement, anger, etc., or the voice characteristic identifier indicating the speech speed such as fast or steady, so that a variety of voice characteristics can be obtained, thereby improving the diversity of the reading voice. Through this implementation, the voice characteristics of the user who read aloud last time can be combined to form a dictation audio with the voice characteristics, so that the diversity of the dictation audio is achieved in the dictation process, and the fun is improved to attract users to improve their learning attention.

[0112] 205. The electronic device extracts a first text region contour of a first user image among the plurality of frames of user images, and extracts a second text region contour of a last user image among the plurality of frames of user images.

[0113] In the embodiment of the present invention, the user image is an image including the user's hand area and the text area, or an image including the user's hand area, the text area and the pattern area.

[0114] It can be understood that the first text area outline may include one or more text area outlines. When there is only one complete text area in the user image (the user image does not contain a pattern area), there is only one text area outline; when the user image contains multiple text areas (the user image contains one or more pattern areas), there are correspondingly multiple text area outlines.

[0115] It should be noted that, in the process of capturing several frames of user images, assuming that the user turns the pages, then except for the first frame of user image and the last frame of user image, several user images in the middle may be captured at the same time when the user turns the pages, which means that the several user images in the middle may not be captured in a stable state. Therefore, excluding several user images in the middle and using the first frame of user image and the last frame of user image can improve the accuracy of extracting the text area contour, thereby reducing the misjudgment rate of the user turning the pages.

[0116] As an optional implementation, the electronic device may extract the outline of the first text area of ​​the first frame of the user image by performing contrast enhancement processing on the first frame of the user image among several frames of the user image, and then performing grayscale processing on the first frame of the user image after the enhancement processing, and obtaining the first text area in the first frame of the user image after the grayscale processing, and extracting the outline of the first text area as the first text area outline. Similarly, the specific implementation of the electronic device extracting the outline of the second text area of ​​the last frame of the user image among several frames of the user image will not be repeated here. Through this implementation, the accuracy of text area outline extraction can be improved.

[0117] 206. The electronic device calculates the similarity between the first text region outline and the second text region outline.

[0118] Optionally, step 206 may specifically include: the electronic device obtains the center of gravity of the first text area outline and the second text area outline, overlaps the two centers of gravity so that the two centers of gravity overlap, and then calculates the overlapping area of ​​the first text area outline and the second text area outline, and takes the quotient of the overlapping area and the area of ​​the first text area outline as the similarity, or takes the quotient of the overlapping area and the area of ​​the second text area outline as the similarity, thereby improving the efficiency and speed of the electronic device in calculating the similarity between the first text area outline and the second text area outline.

[0119] 207. The electronic device determines whether the similarity is less than a preset threshold. If so, execute step 208; otherwise, end this process.

[0120] It can be understood that the smaller the similarity is, the greater the possibility that the text area in the first frame of the user image and the text area in the last frame of the user image are not the same text page.

[0121] The preset threshold value may be preset by a developer according to actual conditions.

[0122] 208. The electronic device determines that the user has performed a page turning action.

[0123] As an optional implementation, after executing step 204, the following steps may also be performed: the electronic device extracts a hand region sub-image from the user image, and determines whether the user's hand performs a preset gesture operation based on the extracted hand region sub-images of several frames, and if so, and the similarity is less than a preset threshold, determines that the user has performed a page turning action. Through this implementation, the misjudgment rate of the page turning action can be reduced.

[0124] 209-210. For the description of steps 209-210, please refer to the detailed description of steps 104-105 in the first embodiment, which will not be repeated in the present invention.

[0125] It can be seen that with the implementation Figure 1 Compared to the described method, the implementation Figure 2 The described method can also determine whether the user has turned the page by determining whether there is written content in each writing area on the writing page, by detecting the similarity between the first frame of the user image and the last frame of the user image among several frames of captured user images, and by determining whether the user's hand performs a preset gesture operation. This can improve the accuracy of determining whether the user has turned the page, thereby further improving the accuracy of dictation detection.

[0126] Embodiment 3

[0127] See also Figure 3 , Figure 3 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. Figure 3 As shown, the electronic device may include:

[0128] The photographing unit 301 is used to photograph a writing page written by the user according to the pronunciation of any dictation word in the dictation audio if the electronic device has finished reading the pronunciation of any dictation word in the dictation audio during the dictation process.

[0129] The first judgment unit 302 is used to judge whether the pronunciation of the dictated word belongs to the pronunciation of the last dictated word in the dictation audio.

[0130] The saving unit 303 is used to save the written page when the judgment result of the first judgment unit 302 is yes.

[0131] The second judgment unit 304 is used to judge whether the user has turned the page when the judgment result of the first judgment unit 302 is no.

[0132] The saving unit 303 is further configured to save the written page when the second determining unit 304 determines that the user has performed a page turning action.

[0133] The detection unit 305 is used to perform dictation detection on all the saved written pages when the dictation is terminated.

[0134] Optionally, Figure 3 The electronic device shown may further include the following units not shown:

[0135] A display unit, for displaying a dictation selection interface on an electronic screen of the electronic device when the electronic device enters a dictation mode, so that the user can select a dictation word;

[0136] A receiving unit, used to receive a selection instruction input by a user on the dictation selection interface;

[0137] The retrieval unit is used to determine a number of dictation words according to the selection instruction, and then retrieve the audio data corresponding to each dictation word, sort the several audio data according to the order selected by the user to generate dictation audio for broadcasting during the dictation process.

[0138] Through this implementation, based on the user's dictation intention, user-personalized dictation words can be generated for the user to practice dictation, which can better meet the user's dictation needs and increase user stickiness.

[0139] Further optionally, the above-mentioned display unit is also used to display a face recognition interface on the electronic screen when receiving a dictation wake-up voice input by the user, so that the user can perform a face recognition operation;

[0140] as well as, Figure 3 The electronic device shown may also include a starting unit (not shown) for controlling the electronic device to enter a dictation mode when the face recognition of the user is successful and the user performs a face recognition operation with a preset expression.

[0141] The preset expression may specifically be an expression preset corresponding to the dictation mode.

[0142] Through this implementation, the fun of the dictation process started by the user can be increased, and the user can be helped to mobilize positive emotions to perform dictation learning, so as to improve learning efficiency.

[0143] Optionally, the above-mentioned shooting unit 301 can be specifically used in the dictation process. If the electronic device has finished reading the pronunciation of any dictated word in the dictation audio, a sensor signal detected by a sensor device set in the electronic device within a specified time period is obtained, and the user is identified according to the sensor signal whether the user has finished writing according to the pronunciation of the dictated word. If the writing is completed, the writing page after the user writes according to the pronunciation of the dictated word is obtained by shooting. Among them, the sensor device includes but is not limited to an infrared sensor device, an image sensor device or an ultrasonic sensor device. Through this embodiment, the moment when the user finishes writing according to the pronunciation of the dictated word can be accurately identified, making the electronic device more intelligent.

[0144] It can be seen that the implementation Figure 3 The electronic device shown, during the dictation process, if the electronic device has finished reading the pronunciation of any dictated word in the dictation audio, it will photograph the writing page that the user has written according to the pronunciation of the dictated word, and determine whether the pronunciation of the dictated word belongs to the pronunciation of the last dictated word in the dictation audio. If so, the writing page is saved; if not, and it is determined that the user has turned the page, the writing page is saved, and then dictation detection is performed on all the saved writing pages when the dictation is terminated. Multiple writing pages can be intelligently detected to improve the accuracy of dictation detection.

[0145] Embodiment 4

[0146] See also Figure 4 , Figure 4 is a schematic diagram of the structure of another electronic device disclosed in an embodiment of the present invention. Figure 4 The electronic equipment shown is composed of Figure 3 The electronics shown are optimized to obtain Figure 3 In comparison, Figure 4 In the electronic device shown, the second determination unit 304 may include:

[0147] The judging subunit 3041 is used to judge whether all writing areas on the writing page have written contents when the judgment result of the first judging unit 302 is no.

[0148] The determination subunit 3042 is used to determine whether the user has performed a page turning action when the determination subunit 3041 determines that each writing area has written content.

[0149] As an optional implementation, Figure 4 In the electronic device shown, the shooting unit 301 is also used to shoot a plurality of frames of user images at a preset frequency within a preset time period when the judging subunit 3041 determines that not all writing areas have written content.

[0150] as well as, Figure 4 The electronic device shown may also include:

[0151] The extraction unit 306 is used to extract the first text region outline of the first frame of the user image among the plurality of frames of the user image; and to extract the second text region outline of the last frame of the user image among the plurality of frames of the user image.

[0152] The calculation unit 307 is used to calculate the similarity between the first text region outline and the second text region outline.

[0153] The determination subunit 3042 is further configured to determine that the user has performed a page turning action when the similarity is less than a preset threshold.

[0154] Further optionally, the extraction unit 306 is further configured to extract a hand region sub-image from the user image after the shooting unit 301 shoots and obtains a plurality of frames of user images at a preset frequency within a preset time period.

[0155] The judgment subunit 3041 is further used to judge whether the user's hand performs a preset gesture operation based on the extracted hand area sub-images of several frames.

[0156] The determination subunit 3042 is specifically configured to determine that the user has performed a page turning action when the determination subunit 3041 determines that the user's hand has performed a preset gesture operation and the similarity is less than a preset threshold.

[0157] As an optional implementation, Figure 4 In the electronic device shown, the detection unit 305 includes:

[0158] The acquisition subunit 3051 is used to acquire the respective shooting times of all the saved written pages when the dictation is terminated.

[0159] The sorting subunit 3052 is used to sort all written pages according to the chronological order of shooting time.

[0160] The correction sub-unit 3053 is used to correct the sorted writing pages one by one according to the standard answers corresponding to the dictation words.

[0161] By implementing this implementation mode, all written pages are sorted according to the shooting time and then corrected one by one, which can improve the efficiency of dictation detection.

[0162] Optionally, the above-mentioned correction subunit 3053 may include the following modules not shown in the figure:

[0163] An extraction module is used to extract a number of written words written by the user from each sorted writing page in turn, obtain a dictated word corresponding to each written word, and obtain a standard answer corresponding to each written word, wherein the standard answer corresponds to the dictated word one by one;

[0164] A marking module is used to mark any written word as an incorrect word when the formed character shape of the written word does not match the standard answer corresponding to the written word; and to mark any written word as a correct word when the formed character shape of the written word matches the standard answer corresponding to the written word and the writing time of the written word reaches a preset threshold; and to mark any written word as a mastered word when the formed character shape of the written word matches the standard answer corresponding to the written word and the writing time of the written word does not reach the preset threshold.

[0165] By implementing this embodiment, the user's written words can be corrected into three words with different mastery levels, namely wrong words, correct words and mastered words according to the formed character shape and writing time. This can help the user understand the dictation test results in a hierarchical manner, so as to further consolidate the practice and thus improve the user's dictation learning effect.

[0166] Further optionally, Figure 4 The electronic device shown may further include the following units not shown:

[0167] An acquisition unit, used for acquiring a number of dictation words in a user's learning plan that meet the current time period before the electronic device enters the dictation mode; and retrieving audio data corresponding to each dictation word;

[0168] A determination unit, used to obtain the reading record data of the user's last reading learning, and determine the user's voice characteristics according to the reading record data;

[0169] The generating unit is used to generate the dictation audio by using a preset speech synthesis algorithm according to the audio data, the preset broadcast interval and the sound characteristics.

[0170] Among them, the preset speech synthesis algorithm can be a linear predictive coding method (LPC) or a pitch synchronous superposition method (PSOLA) method, etc.; the user's voice characteristics are used to describe the stored voice characteristics of the user who last performed reading aloud learning, such as voice characteristic identifiers under emotions such as happiness, excitement, and anger, or voice characteristic identifiers indicating fast or steady speaking speed, so that a variety of voice characteristics can be obtained, thereby improving the diversity of reading aloud voices.

[0171] Through this implementation, the voice characteristics of the user who performed reading aloud learning last time can be combined to form dictation audio with the voice characteristics, thereby achieving diversity in the dictation audio during the dictation process and increasing interest, so as to attract users to increase their learning attention.

[0172] As an optional implementation, Figure 4 In the electronic device shown:

[0173] The method for the extraction unit 306 to extract the first text area contour of the first frame of the user image among the plurality of frames of user images may specifically be:

[0174] The extraction unit 306 is used to perform contrast enhancement processing on the first frame of the user image among the plurality of frames of user images, and then perform grayscale processing on the first frame of the user image after the enhancement processing; and obtain the first text area in the first frame of the user image after the grayscale processing, and extract the outline of the first text area as the first text area outline. Similarly, the specific implementation method of the extraction unit 306 extracting the second text area outline of the last frame of the user image among the plurality of frames of user images will not be repeated here.

[0175] Through this implementation, the accuracy of extracting the outline of the text area can be improved.

[0176] Optionally, the calculation unit 307 may include the following modules not shown in the figure:

[0177] A superposition module, used for obtaining the center of gravity of the first text area outline and the second text area outline, and superimposing the two centers of gravity so that the two centers of gravity overlap;

[0178] The calculation module is used to calculate the overlapping area between the first text area outline and the second text area outline, and use the quotient of the overlapping area and the area of ​​the first text area outline as the similarity; or use the quotient of the overlapping area and the area of ​​the second text area outline as the similarity.

[0179] The implementation of this embodiment can improve the efficiency and speed of calculating the similarity between the first text area outline and the second text area outline.

[0180] It can be seen that with the implementation Figure 3 Compared with the electronic device shown, the implementation Figure 4 The electronic device shown can also determine whether the user has turned the page by determining whether there is writing in each writing area on the writing page, by detecting the similarity between the first frame of the user image and the last frame of the user image among several frames of captured user images, and by determining whether the user's hand performs a preset gesture operation. This can improve the accuracy of determining whether the user has turned the page, thereby further improving the accuracy of dictation detection.

[0181] Embodiment 5

[0182] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the structure of another electronic device disclosed in an embodiment of the present invention. Figure 5 As shown, the electronic device may include:

[0183] A memory 501 storing executable program codes;

[0184] a processor 502 coupled to the memory 501;

[0185] The processor 502 calls the executable program code stored in the memory 501 to execute Figure 1-2 Any dictation detection method.

[0186] It should be noted that Figure 5 The electronic device shown may also include a power supply, input buttons, speakers, microphones, screens, RF circuits, Wi-Fi modules, Bluetooth modules, sensors and other components not shown, which are not described in detail in this embodiment.

[0187] The embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute Figure 1-2 Any dictation detection method.

[0188] The embodiments of the present invention further disclose a computer program product, wherein when the computer program product is run on a computer, the computer is enabled to execute part or all of the steps of the methods in the above method embodiments.

[0189] An embodiment of the present invention further discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps of the method in the above method embodiments.

[0190] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0191] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above-mentioned processes does not mean the necessary order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0192] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above methods of various embodiments of the present invention.

[0195] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0196] Those skilled in the art can understand that some or all of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0197] A dictation detection method, an electronic device and a storage medium disclosed in an embodiment of the present invention are introduced in detail above. Specific examples are used herein to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for a person skilled in the art, according to the idea of ​​the present invention, there may be changes in the specific implementation mode and the scope of application. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A dictation detection method, characterized in that: include: During the dictation process, if the electronic device has finished reading the pronunciation of any dictation word in the dictation audio, a writing page written by the user according to the pronunciation of the dictation word is photographed; Determine whether the pronunciation of the dictated word belongs to the pronunciation of the last dictated word in the dictation audio, and if so, save the writing page; If not, determining whether the user has turned the page, and if so, saving the written page; When dictation is terminated, all saved written pages are checked for dictation; The determining whether the user has performed a page turning action includes: Determining whether each writing area on the writing page has writing content, wherein the writing area includes a plurality of single-word areas; If there is written content in each of the writing areas, it is determined that the user has performed a page turning action.

2. The method according to claim 1, characterized in that Also includes: If not all the writing areas are written with writing content, a plurality of frames of user images are captured at a preset frequency within a preset time period; Extracting a first text region outline of a first frame of user images among the plurality of frames of user images, and extracting a second text region outline of a last frame of user images among the plurality of frames of user images; Calculating the similarity between the first text area outline and the second text area outline; If the similarity is less than a preset threshold, it is determined that the user has performed a page turning action.

3. The method according to claim 2, characterized in that After obtaining a plurality of frames of user images by shooting at a preset frequency within a preset time period, the method further includes: Extracting a hand region sub-image from the user image, and judging whether the user's hand performs a preset gesture operation according to the extracted hand region sub-images of several frames; If so, and the similarity is less than a preset threshold, it is determined that the user has performed a page turning action.

4. The method according to any one of claims 1 to 3, characterized in that: The dictation detection of all the saved written pages includes: Get the shooting time of each of the saved writing pages; Sorting all the written pages according to the chronological order of the shooting moments; According to the standard answers corresponding to the dictation words, the sorted writing pages are corrected one by one.

5. An electronic device, characterized in that: include: A photographing unit is used to photograph a writing page written by a user according to the pronunciation of any dictation word in the dictation audio if the electronic device has finished reading the pronunciation of any dictation word in the dictation audio during the dictation process; A first judgment unit, used to judge whether the pronunciation of the dictation word belongs to the pronunciation of the last dictation word in the dictation audio; a saving unit, configured to save the written page when the judgment result of the first judgment unit is yes; A second judgment unit, configured to judge whether the user has performed a page turning action when the judgment result of the first judgment unit is no; The storage unit is further configured to store the written page when the second determination unit determines that the user has performed a page turning action; A detection unit, used for performing dictation detection on all saved written pages when dictation is terminated; Wherein, the second judgment unit includes: A judging subunit, used for judging whether all writing areas on the writing page have written contents when the judgment result of the first judging unit is no, the writing areas including a plurality of single-word areas; The determination subunit is used to determine that the user has performed a page turning action when the determination subunit determines that each writing area has written content.

6. The electronic device according to claim 5, characterized in that: The shooting unit is further configured to shoot a plurality of frames of user images at a preset frequency within a preset time period when the judging subunit judges that not all the writing areas have written content; And, the electronic device further includes: An extraction unit is used to extract a first text area contour of a first frame of user image among the plurality of frames of user image; and to extract a second text area contour of a last frame of user image among the plurality of frames of user image; A calculation unit, used for calculating the similarity between the first text area outline and the second text area outline; The determination subunit is further configured to determine that the user has performed a page turning action when the similarity is less than a preset threshold.

7. The electronic device according to claim 6, characterized in that: The extraction unit is further configured to extract a hand region sub-image from the user image after the shooting unit obtains a plurality of frames of the user image by shooting at a preset frequency within a preset time length; The judging subunit is further used to judge whether the user's hand performs a preset gesture operation based on the hand region sub-images extracted from the plurality of frames; The determination subunit is specifically configured to determine that the user has performed a page turning action when the determination subunit determines that the user's hand performs a preset gesture operation and the similarity is less than a preset threshold.

8. The electronic device according to any one of claims 5 to 7, characterized in that: The detection unit comprises: The acquisition subunit is used to acquire the shooting time of each of all the saved writing pages when the dictation is terminated; A sorting subunit, used for sorting all the written pages according to the chronological order of the shooting moments; The correction subunit is used to correct the sorted writing pages one by one according to the standard answers corresponding to the dictation words.

9. An electronic device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the dictation detection method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the dictation detection method according to any one of claims 1 to 4.

Images