Question positioning method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of teaching equipment, and provides a question positioning method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a page image of a book currently written by a user and related information of a writing action performed by the user; and according to the related information and the book page image, determining a target question currently written by the user. Through the setting, a user does not need to carry out operations such as manual photographing and question frame selection, and only needs to carry out normal writing in a book page, and the electronic equipment can automatically detect related information of writing actions of the user, such as information of a pen point position, a finger position, writing duration and the like; and a question currently written by the user is positioned by combining the part of information and the page image of the book, so that the user operation is simplified.

Description

Technical Field

[0001] The present application relates to the technical field of teaching equipment, and in particular to a method, device, electronic device and storage medium for locating a question. Background Art

[0002] Electronic devices such as tutoring machines or learning tablets have a wealth of learning assistance functions. For example, when a user encounters a question that he or she does not know how to answer while doing homework, he or she can use the above electronic devices to search for the corresponding answer. However, when performing the question location operation, the user is generally required to manually take a photo and select the question information, which shows that there is a problem of cumbersome user operation. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a topic locating method, device, electronic device and storage medium, which can conveniently locate the topic currently being written by the user and simplify user operations.

[0004] A first aspect of an embodiment of the present application provides a topic location method, including:

[0005] Obtain the image of the book page currently written by the user and relevant information of the writing action performed by the user;

[0006] The target topic that the user is currently writing is determined based on the relevant information and the book page image.

[0007] In the embodiment of the present application, electronic devices such as tutoring machines can obtain the image of the book page currently written by the user and the relevant information of the writing action performed by the user, and then determine the target topic currently being written by the user based on the relevant information and the book page image. With this setting, the user does not need to manually take pictures and select topics, but only needs to write normally on the book page. The electronic device will automatically detect the relevant information of the user's writing action, such as the position of the pen tip, the position of the finger, and the duration of writing, and locate the topic currently being written by the user in combination with this information and the book page image, thereby simplifying the user's operation.

[0008] In an implementation of the embodiment of the present application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; and determining the target topic currently being written by the user according to the relevant information and the book page image may include:

[0009] Determining the topic area currently being written by the user from the book page image according to the pen tip position;

[0010] Each topic in the topic area is determined as the target topic.

[0011] In an implementation of the embodiment of the present application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; and determining the target topic currently being written by the user according to the relevant information and the book page image may include:

[0012] Obtaining the position of each title frame in the book page image;

[0013] Respectively detecting the distance between the pen tip position and each of the question frame selection positions;

[0014] The question within the question frame selection position that has the smallest distance from the pen tip position is determined as the target question.

[0015] In an implementation of the embodiment of the present application, the writing pen used by the user is provided with a pen body camera, and the relevant information includes a question stem information image collected by the pen body camera; and determining the target question currently being written by the user according to the relevant information and the book page image may include:

[0016] Performing OCR recognition on the question stem information image to obtain the question stem information of the question currently being written by the user;

[0017] Search the question library stored in the background for a question that matches the question stem information as the target question.

[0018] Furthermore, searching for a question matching the question stem information from the question library stored in the background as the target question may include:

[0019] Determining, according to the book page image, a target book page currently being written by the user;

[0020] A title matching the title information is searched from various titles in the target book page stored in the title library as the target title.

[0021] In an implementation of the embodiment of the present application, the relevant information includes a duration of the user performing the writing action; and determining the target topic currently being written by the user according to the relevant information and the book page image may include:

[0022] Determining, according to the book page image, a target book page currently being written by the user;

[0023] The target topic is determined from the topics in the target book page according to the duration, the arrangement order of the topics in the target book page and the estimated writing time.

[0024] In an implementation of the embodiment of the present application, after determining the target topic that the user is currently writing, the following steps may also be included:

[0025] When the target interaction action of the user is detected, the target display device is controlled to display the learning content associated with the target topic.

[0026] A second aspect of an embodiment of the present application provides a topic locating device, including:

[0027] An information acquisition module, used to acquire an image of a book page currently written by a user and relevant information of a writing action performed by the user;

[0028] The topic positioning module is used to determine the target topic that the user is currently writing based on the relevant information and the book page image.

[0029] A third aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the topic locating method provided in the first aspect of the embodiment of the present application is implemented.

[0030] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the topic locating method provided in the first aspect of the embodiments of the present application is implemented.

[0031] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the topic locating method provided in the first aspect of the embodiments of the present application.

[0032] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of a topic location method provided by an embodiment of the present application;

[0034] Figure 2 This is a schematic diagram of an operation flow of an electronic device provided in an embodiment of the present application when locating a question;

[0035] Figure 3 It is an operation diagram of a topic location method provided by an embodiment of the present application;

[0036] Figure 4 is an operation diagram of another topic locating method provided by an embodiment of the present application;

[0037] Figure 5 It is a structural framework diagram of a question locating device provided in an embodiment of the present application;

[0038] Figure 6 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures, technologies, etc. are proposed, so as to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, the detailed description of well-known systems, devices, circuits and methods is omitted to prevent unnecessary details from hindering the description of the present application. In addition, in the description of the present application specification and the attached claims, the terms "first", "second", "third" etc. are only used to distinguish the description, and cannot be interpreted as indicating or suggesting relative importance.

[0040] Electronic devices such as tutoring machines and learning tablets can provide high-quality learning resources and have rich functions such as dictation, recitation, translation, question search and point reading. When users encounter difficult questions while doing homework, they can use these electronic devices to search for answers to the questions or other related learning content. However, when locating questions, users are generally required to manually take photos and select question information, which is cumbersome for users to operate.

[0041] To address this problem, the embodiments of the present application provide a method, device, electronic device, and storage medium for locating a topic, which can conveniently locate the topic currently being written by the user and simplify user operations. For more specific technical implementation details of the embodiments of the present application, please refer to the various embodiments described below.

[0042] It should be understood that the execution subjects of the various method embodiments of the present application are various types of electronic devices, such as mobile phones, tablet computers, wearable devices, learning machines, tutoring machines, learning tablets, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), large-screen TVs, etc. The embodiments of the present application do not impose any restrictions on the specific types of the electronic devices.

[0043] See also Figure 1 , showing a topic location method provided by an embodiment of the present application, including:

[0044] 101. Obtaining an image of a book page currently being written by a user and information related to a writing action performed by the user;

[0045] The execution subject of the embodiment of the present application can be an electronic device with teaching functions such as a tutoring machine or a learning tablet. When a user uses a book to do homework in front of an electronic device, the front camera of the electronic device can be used to capture the front image, thereby obtaining the image of the book page currently written by the user in real time. On the other hand, the electronic device will also obtain relevant information about the writing action performed by the user, which may include but is not limited to: pen tip position, finger position, writing duration, handwriting information and user line of sight information, etc. Among them, the pen tip position, finger position and handwriting information can be obtained by identifying the captured book page image; the writing duration can be obtained by combining page turning action detection with a timer, for example, when the user's page turning action is detected, the timer is started to record the duration of the user writing on the current book page, and then, when the next page is turned, the timer is reset and the duration of the user writing on the next book page is recorded, and so on; the user's line of sight information can be obtained by an eye tracker or a camera in a suitable position, mainly used to determine the user's line of sight when writing on the current book page.

[0046] In an implementation of the embodiment of the present application, before obtaining the image of the book page currently written by the user and the related information of the writing action performed by the user, the following may also be included:

[0047] (1) Detect whether there is an obstruction in front of the electronic device;

[0048] (2) If there is an obstruction in front of the electronic device, detect whether there is a face or a book in front of the electronic device;

[0049] (3) If there is a face or a book in front of the electronic device, the electronic device is controlled to enter a specified scene mode.

[0050] The electronic device can detect in real time whether there is an obstruction in front of it. When there is an obstruction, face detection is started to further determine whether the obstruction is a person. When there is no obstruction, it means that there is definitely no person in front of the electronic device, so there is no need to start face detection, which can reduce the consumption of device resources. Specifically, when detecting whether there is an obstruction in front of the electronic device, an infrared pair tube or a range sensor can be used, and a detection method based on direct measurement of flight time (dtof) or a detection method based on indirect measurement of flight time (itof), etc., and the embodiment of the present application does not limit the specific method of detecting the obstruction. If an obstruction is detected in front of the electronic device, the front camera of the electronic device can be turned on to start face detection and book detection. In actual operation, the image can be analyzed and processed to identify whether there are faces and books in the image at the same time, wherein various face detection algorithms can be used to identify faces, and book target detection algorithms based on deep learning can be used to identify books. If the electronic device detects that there are faces and books in front of it at the same time, it can be determined with a high probability that the user is doing homework or reading a book, thereby controlling the electronic device to enter a specified scene mode such as a homework mode or a reading mode. In addition, in order to improve the accuracy of scene mode control, judgment can also be made based on the user's working time pattern. For example, when a face and a book are detected in front of the electronic device at the same time, and the current time is within the user's regular working time range, the electronic device is controlled to enter the corresponding scene mode.

[0051] Furthermore, after controlling the electronic device to enter the specified scene mode, the following steps may also be included:

[0052] (1) Detect the type of book;

[0053] (2) Control the electronic device to enter the functional mode corresponding to the type.

[0054] The electronic device can further detect the type of book the user is currently writing, and thus enter a functional mode corresponding to the type of the book. For example, if the type of book is a textbook, it enters functional mode one: textbook learning mode; if the type of book is an exercise book, it enters functional mode two: exercise book learning mode, and so on. After entering different functional modes, the electronic device can recommend targeted auxiliary learning functions respectively. For example, in the textbook learning mode, it can recommend knowledge point summarization, point reading and other functions. When detecting the type of book, the captured book page image can be used to perform a full-page image search, and match it with a book database stored locally or on a server (such as a textbook library or an exercise book library), and the type of book can be determined based on the matching results.

[0055] In addition, after matching the scene mode and the corresponding function mode, the electronic device can start the silent guard function. For example, when the user is using a textbook to do homework, the electronic device can start the homework mode and enter the textbook learning mode. After that, the electronic device can enter a black screen state and silently guard while the user is doing homework without disturbing the user. At this time, the front camera of the electronic device will continue to monitor the user's written books, handwriting, and sitting posture. When the user asks a question, the electronic device can immediately answer it, and when the user has a bad sitting posture, the electronic device can also immediately remind the user.

[0056] 102. Determine the target topic that the user is currently writing based on the relevant information and the book page image.

[0057] After obtaining the image of the book page currently being written by the user and the relevant information of the writing action performed by the user, the topic currently being written by the user can be determined based on the relevant information and the book page image, and the topic is represented by the target topic. With this setting, the user does not need to manually take pictures and select topics, but only needs to write normally on the book page. The electronic device will automatically detect the relevant information of the user's writing action, and locate the topic currently being written by the user in combination with this part of information and the book page image, thereby simplifying the user operation.

[0058] In one implementation of the embodiment of the present application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; determining the target topic currently being written by the user according to the relevant information and the book page image may include:

[0059] (1) Determine the topic area currently being written by the user from the book page image based on the pen tip position;

[0060] (2) Identify each question in the question area as the target question.

[0061] The relevant information obtained may include the pen tip position of the writing pen used by the user in the book page image. According to the pen tip position, the topic area currently being written by the user can be determined from the book page image, and each topic in the topic area can be determined as the target topic. For example, assuming that the book page image is an A4 page, the A4 page can be divided into three parts: the upper area, the middle area and the lower area, and then the part of the area where the pen tip position is located is detected as the topic area currently being written by the user. The topic area may contain multiple topics (for example, 2-3 topics), and one of these topics is the topic currently being written by the user. It can be seen that this is a fuzzy positioning method, that is, only 2-3 topics including the topic currently being written by the user can be located, and the topic currently being written by the user cannot be clearly located. In addition, after determining the topic area, each topic in the topic area can be displayed on the screen as a target topic, for example, each topic in the topic area can be displayed on the screen of an electronic device. In addition to the pen tip position, the finger position or hand position can also be used to identify the topic area currently being written by the user, and the specific implementation method is the same as the pen tip position.

[0062] In another implementation of the embodiment of the present application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; determining the target topic currently being written by the user based on the relevant information and the book page image may include:

[0063] (1) Obtain the position of each title frame in the book page image;

[0064] (2) respectively detecting the distance between the pen tip position and each question selection position;

[0065] (3) The question within the question selection position that is the shortest distance from the pen tip position is determined as the target question.

[0066] By identifying the image of the book page, the pen tip position of the writing pen used by the user and the position of each question selection frame can be obtained. Each question selection frame position can be a closed frame (such as a rectangular frame) for selecting a single question. The distance between the pen tip position and each question selection frame position is detected respectively, and the question selection frame position with the smallest distance is found, and then the question in the question selection frame position is determined as the target question that the user is currently writing. Similarly, after determining the question selection frame position closest to the pen tip position, the single question in the question selection frame position can be displayed on the screen, for example, the question can be displayed on the screen of an electronic device. In addition to the pen tip position, the finger position or hand position can also be compared with each question selection frame position, and the specific implementation method is the same as the pen tip position.

[0067] In another implementation of the embodiment of the present application, the writing pen used by the user is provided with a pen body camera, and the relevant information includes a question stem information image collected by the pen body camera; the determining the target question currently being written by the user based on the relevant information and the book page image may include:

[0068] (1) Perform OCR recognition on the question stem information image to obtain the question stem information of the question currently being written by the user;

[0069] (2) Search for a question that matches the question stem information from the question database stored in the background as the target question.

[0070] When identifying the position of the pen tip according to the book page image, if the pen tip is blocked by the user's hand, the position of the pen tip cannot be accurately identified, which will cause the accuracy of locating the target question using the pen tip position to decrease. In view of this situation, a miniature pen body camera can be set in the writing pen, and the pen body camera can be installed below the pen holding part of the writing pen to capture part of the stem information of the user's current writing question in real time. After obtaining the stem information image collected by the pen body camera, the electronic device can perform OCR recognition on the stem information image to obtain part of the stem information of the question currently being written by the user. Then, the corresponding question is matched to the system background through the part of the stem information, for example, the question matching the part of the stem information can be found from the question library stored in the background as the target question. The question library can be stored locally or on the server side of the electronic device, and the questions of various textbooks, exercise books or test papers can be pre-collected in the question library. In the specific operation, the part of the stem information recognized by OCR can be matched with the stem information of each question in the question library to find the question with the highest matching degree as the target question.

[0071] Furthermore, searching for a question matching the question stem information from the question library stored in the background as the target question may include:

[0072] (1) Determine the target book page currently being written by the user based on the book page image;

[0073] (2) Search for a question that matches the question stem information from the various questions in the target book page stored in the question library as the target question.

[0074] Since there are many questions stored in the question bank, if the partial stem information identified by OCR is matched with all the questions stored in the question bank, it takes a long time and the accuracy is not high. In view of this problem, the book page currently being written by the user can be identified based on the book page image, and represented by the target book page. Specifically, the book page image can be matched based on the book library or teaching material library scanned page stored locally or on the server of the electronic device, so as to determine which page of which book the book page image belongs to, that is, to determine the target book page. Then, each question of the target book page is found out from the question bank, and the partial stem information identified by OCR is matched with the stem information of each question of the target book page, and the question with the highest matching degree is found as the target question. By setting it in this way, it is only necessary to match the partial stem information identified by OCR with a small amount of questions (each question of the target book page) stored in the question bank, thereby reducing the time consumption of the matching operation and improving the matching accuracy.

[0075] In another implementation of the embodiment of the present application, the relevant information includes the duration of the user's writing action; determining the target topic that the user is currently writing based on the relevant information and the book page image may include:

[0076] (1) Determine the target book page currently being written by the user based on the book page image;

[0077] (2) Determine the target topic from among the topics on the target book page based on the duration, the order of the topics on the target book page, and the estimated writing time.

[0078] As described above, based on the book page image matching of the book library or teaching material library scanned page stored locally or on the server of the electronic device, it is possible to determine which page of which book the book page image belongs to, thereby determining the book page that the user is currently writing, and the book page is represented by the target book page. By using the page turning action detection and the set timer, it is possible to count the duration of the user writing on the target book page, and then according to the known arrangement order and estimated writing time of each topic in the target book page, the target topic that the user is currently writing can be determined from the various topics in the target book page. Specifically, since each topic contained in the target book page is known, the corresponding estimated writing time can be set in advance for each topic, for example, the estimated writing time of a fill-in-the-blank question is 2 minutes, the estimated writing time of a calculation question is 10 minutes, and so on. In addition, since the arrangement order of these topics is also known, after obtaining the duration of the user writing on the target book page, the target topic that the user is currently writing can be estimated. For example, if the questions in the target book page are fill-in-the-blank question 1 (estimated writing time 2 minutes), fill-in-the-blank question 2 (estimated writing time 2 minutes), fill-in-the-blank question 3 (estimated writing time 2 minutes), calculation question 1 (estimated writing time 10 minutes) and calculation question 2 (estimated writing time 10 minutes), and the duration is 9 minutes, then it can be determined that the target question the user is currently writing is calculation question 1.

[0079] In one implementation of the embodiment of the present application, after determining the target topic that the user is currently writing, the following steps may also be included:

[0080] When a target interactive action of the user is detected, the target display device is controlled to display the learning content associated with the target topic.

[0081] After locating the target topic that the user is currently writing, the electronic device can control a target display device (for example, the screen of the electronic device itself, or other display devices connected to the electronic device) to display the learning content associated with the target topic. In actual operation, the user only needs to make a certain target interaction action, and the electronic device controls the target display device to display the learning content associated with the target topic after detecting the user's target interaction action. There are two implementation methods here. The first is that when the user's target interaction action is detected, the electronic device searches for the learning content associated with the target topic and displays it. The second is that the electronic device first searches for the learning content associated with the target topic, and then displays the learning content when the user's target interaction action is detected. The learning content associated with the target topic may include but is not limited to: question video analysis, question text analysis, answer analysis, summary comments, thought inspiration, knowledge point analysis, etc. The target interaction action may include but is not limited to: looking up at the screen, shaking your head, nodding, waving your hand, waving your hand, pointing your finger at the screen, etc. Obviously, by setting it in this way, the user only needs to make a simple interaction action, and the electronic device will automatically display the learning content associated with the current topic being written on the screen. For example, when a user encounters a difficult problem while doing homework, he or she only needs to look up at the screen of the electronic device, and the electronic device will automatically analyze the answer to the difficult problem and display it on the screen, which is very convenient to operate.

[0082] In the embodiment of the present application, electronic devices such as tutoring machines can obtain the image of the book page currently written by the user and the relevant information of the writing action performed by the user, and then determine the target topic currently being written by the user based on the relevant information and the book page image. With this setting, the user does not need to manually take pictures and select topics, but only needs to write normally on the book page. The electronic device will automatically detect the relevant information of the user's writing action, such as the position of the pen tip, the position of the finger, and the duration of writing, and locate the topic currently being written by the user in combination with this information and the book page image, thereby simplifying the user's operation.

[0083] like Figure 2 FIG. 1 is a schematic diagram of an operation flow of an electronic device provided in an embodiment of the present application when locating a topic. Figure 2In the process of learning, the electronic device can actively detect whether there are obstructions, faces and books in front of it, so as to start the learning mode. Scene recognition means that the electronic device can detect the image of the book page on the desk, identify the type of book by searching the whole page, and enter the learning mode corresponding to the book type, such as textbook learning mode, exercise book learning mode, etc. Homework protection means that when the electronic device enters the corresponding learning mode, the screen will be black and silent, but the camera will keep working, monitoring the book page and the user's writing action in real time. Problem positioning means that the electronic device can locate the question or question area that the user is currently writing based on the relevant information of the user's writing action and the collected book page image. Problem analysis viewing means that when the user makes an interactive action such as looking up at the screen, the electronic device can detect the face and light up the screen, and display the explanation video, answer analysis or summary comments related to the question that the user is currently writing on the screen.

[0084] To facilitate understanding of the topic location method provided in the embodiments of the present application, two practical application scenarios are listed below.

[0085] like Figure 3 , which is a schematic diagram of the operation of a topic locating method provided in an embodiment of the present application. Figure 3 What is displayed is the current book page image acquired by the electronic device, and the book page image is divided into three parts: the upper area, the middle area, and the lower area. Since the detected pen tip position is in the lower area of ​​the book page image, the lower area is determined as the topic area currently being written by the user, and the topics 6 and 7 contained in the lower area are determined as the target topics currently being written by the user, thereby achieving fuzzy topic positioning.

[0086] like Figure 4 , which is a schematic diagram of the operation of another topic locating method provided in an embodiment of the present application. Figure 4 What is displayed is the current book page image acquired by the electronic device, and each title frame in the current book page image has been identified, and each title frame contains a single question. Since the detected pen tip position is closest to the title frame 4, the title 4 contained in the title frame 4 is determined as the target title currently being written by the user, thereby achieving accurate title positioning.

[0087] In summary, the embodiment of the present application can complete the title positioning by combining the book page image currently written by the user and the relevant information of the writing action performed by the user, which can effectively simplify the user operation.

[0088] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0089] A topic locating method is mainly described above, and a topic locating device will be described below.

[0090] See also Figure 5 , an embodiment of a title locating device in the embodiment of the present application includes:

[0091] The information acquisition module 501 is used to acquire the image of the book page currently written by the user and the related information of the writing action performed by the user;

[0092] The topic locating module 502 is used to determine the target topic that the user is currently writing based on the relevant information and the book page image.

[0093] In an implementation of the embodiment of the present application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; the title positioning module may include:

[0094] a topic area determination unit, configured to determine the topic area currently being written by the user from the book page image according to the pen tip position;

[0095] The first topic locating unit is used to determine each topic in the topic area as the target topic.

[0096] In an implementation of the embodiment of the present application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; the title positioning module may include:

[0097] A title frame obtaining unit, used to obtain the position of each title frame selection in the book page image;

[0098] A distance detection unit, used to detect the distance between the pen tip position and each of the question selection positions;

[0099] The second question locating unit is used to determine the question within the question selection position that is the shortest distance from the pen tip position as the target question.

[0100] In an implementation of the embodiment of the present application, the writing pen used by the user is provided with a pen body camera, and the relevant information includes a question stem information image collected by the pen body camera; the question locating module may include:

[0101] A question stem information recognition unit, used to perform OCR recognition on the question stem information image to obtain the question stem information of the question currently being written by the user;

[0102] The question matching unit is used to search for a question matching the question stem information from a question library stored in the background as the target question.

[0103] Furthermore, the topic matching unit may include:

[0104] A book page determination subunit, configured to determine the target book page currently being written by the user according to the book page image;

[0105] The topic matching subunit is used to search for a topic matching the topic information from various topics in the target book page stored in the topic library as the target topic.

[0106] In an implementation of the embodiment of the present application, the relevant information includes the duration of the user performing the writing action; the topic locating module may include:

[0107] A book page determination unit, configured to determine a target book page currently being written by the user according to the book page image;

[0108] The third topic locating unit is used to determine the target topic from the topics in the target book page according to the duration, the arrangement order of the topics in the target book page and the estimated writing time.

[0109] In one implementation of the embodiment of the present application, the title locating device may further include:

[0110] The learning content display module is used to control the target display device to display the learning content associated with the target topic when the target interaction action of the user is detected.

[0111] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the topic locating method described in any of the above embodiments is implemented.

[0112] An embodiment of the present application further provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the topic locating method described in any of the above embodiments.

[0113] Figure 6 is a schematic diagram of an electronic device provided by an embodiment of the present application. Figure 6As shown, the electronic device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, the steps in the above-mentioned embodiments of the method for locating a topic are implemented, for example Figure 1 Alternatively, when the processor 60 executes the computer program 62, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 5 Functions of modules 501 to 502 are shown.

[0114] The computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 62 in the electronic device 6.

[0115] The processor 60 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0116] The memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. The memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 61 is used to store the computer program and other programs and data required by the electronic device. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0117] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0120] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0121] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0122] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0125] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A topic location method, characterized in that: include: Obtaining an image of a book page currently being written by a user and information related to a writing action performed by the user; The target topic currently being written by the user is determined according to the relevant information and the book page image.

2. The method according to claim 1, characterized in that The relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; and determining the target topic currently being written by the user according to the relevant information and the book page image includes: Determining the topic area currently being written by the user from the book page image according to the pen tip position; Each topic in the topic area is determined as the target topic.

3. The method according to claim 1, characterized in that The relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; and determining the target topic currently being written by the user according to the relevant information and the book page image includes: Obtaining the position of each title frame in the book page image; Respectively detecting the distance between the pen tip position and each of the question frame selection positions; The question within the question frame selection position that has the smallest distance from the pen tip position is determined as the target question.

4. The method according to claim 1, characterized in that The writing pen used by the user is provided with a pen body camera, and the relevant information includes a question stem information image collected by the pen body camera; and determining the target question currently being written by the user according to the relevant information and the book page image includes: Performing OCR recognition on the question stem information image to obtain the question stem information of the question currently being written by the user; Search the question library stored in the background for a question that matches the question stem information as the target question.

5. The method according to claim 4, characterized in that The step of searching for a question matching the question stem information from a question library stored in the backend as the target question includes: Determining, according to the book page image, a target book page currently being written by the user; A title matching the title information is searched from various titles in the target book page stored in the title library as the target title.

6. The method according to claim 1, characterized in that The relevant information includes the duration of the writing action performed by the user; The step of determining the target topic currently being written by the user according to the relevant information and the book page image includes: Determining, according to the book page image, a target book page currently being written by the user; The target topic is determined from the topics in the target book page according to the duration, the arrangement order of the topics in the target book page and the estimated writing time.

7. The method according to any one of claims 1 to 6, characterized in that: After determining the target topic that the user is currently writing, the method further includes: When the target interaction action of the user is detected, the target display device is controlled to display the learning content associated with the target topic.

8. A question locating device, characterized in that: include: An information acquisition module, used to acquire an image of a book page currently written by a user and relevant information of a writing action performed by the user; The topic positioning module is used to determine the target topic that the user is currently writing based on the relevant information and the book page image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the topic locating method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the topic locating method according to any one of claims 1 to 7 is implemented.

Images