A title positioning method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of teaching equipment, and proposes a question positioning method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a book page image currently written by a user and related information of a writing action performed by the user; and determining a question title currently written by the user according to the related information and the book page image. Through the arrangement, the user does not need to perform manual photographing and question framing operations, and only needs to normally write in the book page. The electronic equipment automatically detects related information of the writing action of the user, such as information of a pen tip position, a finger position and a writing duration, and combines the information and the book page image to position the question currently written by the user, so that the user operation is simplified.

Description

Technical Field

[0001] This application relates to the field of teaching equipment technology, and in particular to a question location method, device, electronic device, and storage medium. Background Technology

[0002] Educational tablets and other electronic devices offer a wealth of learning assistance functions. For example, when users encounter problems they can't solve while doing homework, they can use these devices to search for the corresponding answer and explanation. However, locating the correct question usually requires users to manually take a picture and select the question information, which is cumbersome for users. Summary of the Invention

[0003] In view of this, embodiments of this application provide a question location method, apparatus, electronic device, and storage medium, which can conveniently locate the question that the user is currently writing and simplify user operations.

[0004] The first aspect of this application provides a question location method, including:

[0005] Obtain an image of the book page the user is currently writing on and information related to the user's writing actions;

[0006] Based on relevant information and images on the book pages, determine the target topic the user is currently writing about.

[0007] In this embodiment, the tutoring device or other electronic device can acquire an image of the book page the user is currently writing on, as well as information related to the user's writing actions. Then, based on this information and the book page image, it determines the target question the user is currently writing on. With this setup, the user does not need to manually take photos or select questions; they can simply write normally on the book page. The electronic device automatically detects information related to the user's writing actions, such as pen tip position, finger position, and writing duration. Combining this information with the book page image, it locates the question the user is currently writing on, thus simplifying the user's operation.

[0008] In one implementation of this application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; determining the target topic that the user is currently writing based on the relevant information and the book page image may include:

[0009] Based on the pen tip position, determine the area of ​​the question that the user is currently writing on from the image of the book page;

[0010] Each question within the question area is identified as the target question.

[0011] In one implementation of this application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; determining the target topic that the user is currently writing based on the relevant information and the book page image may include:

[0012] Obtain the position of each title selection box in the image of the book page;

[0013] Detect the distance between the pen tip position and each of the question selection boxes;

[0014] The questions within the question selection frame that have the smallest distance from the pen tip position are identified as the target questions.

[0015] In one implementation of this application, the writing pen used by the user is equipped with a pen-body camera, and the relevant information includes an image of the question stem captured by the pen-body camera; determining the target question that the user is currently writing based on the relevant information and the image of the book page may include:

[0016] The question stem information image is subjected to OCR recognition to obtain the question stem information of the question that the user is currently writing;

[0017] Search the question bank stored in the background for questions that match the question stem information, and use them as the target questions.

[0018] Furthermore, the step of searching for a question matching the question stem information from the question bank stored in the background as the target question may include:

[0019] Based on the image of the book page, determine the target book page that the user is currently writing on;

[0020] The question that matches the question stem information is selected from the questions stored in the question bank within the target book page and used as the target question.

[0021] In one implementation of this application, the relevant information includes the duration of the user's writing action; determining the target topic the user is currently writing based on the relevant information and the book page image may include:

[0022] Based on the image of the book page, determine the target book page that the user is currently writing on;

[0023] The target question is determined from the questions on the target book page based on the duration, the order of the questions on the target book page, and the estimated writing time.

[0024] In one implementation of this application, after determining the target topic that the user is currently writing about, the method may further include:

[0025] When the user's target interaction action is detected, the target display device is controlled to display learning content associated with the target question.

[0026] A second aspect of this application provides a question location device, including:

[0027] The information acquisition module is used to acquire an image of the book page currently being written on by the user and related information about the writing action being performed by the user;

[0028] The question location module is used to determine the target question that the user is currently writing based on the relevant information and the image of the book page.

[0029] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the question location method provided in the first aspect of this application.

[0030] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the question location method provided in the first aspect of this application.

[0031] The fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the question location method provided in the first aspect of this application.

[0032] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0033] Figure 1 This is a flowchart of a question location method provided in an embodiment of this application;

[0034] Figure 2 This is a schematic diagram of an operation process of the electronic device provided in this application when performing question location;

[0035] Figure 3 This is a schematic diagram illustrating the operation of a question location method provided in an embodiment of this application;

[0036] Figure 4 This is an operational schematic diagram of another question location method provided in the embodiments of this application;

[0037] Figure 5 This is a structural framework diagram of a question location device provided in an embodiment of this application;

[0038] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail. Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0040] Tutoring devices and learning tablets offer high-quality learning resources and feature a variety of functions such as dictation, memorization, translation, question searching, and point-and-read functionality. When users encounter difficult questions while doing homework, they can use these devices to search for answers or other related learning content. However, locating questions typically requires users to manually take a picture and select the question information, which is cumbersome.

[0041] To address this issue, embodiments of this application provide a question location method, apparatus, electronic device, and storage medium, which can conveniently locate the question a user is currently writing, simplifying user operations. For more specific technical details regarding the embodiments of this application, please refer to the various embodiments described below.

[0042] It should be understood that the execution subject of the various method embodiments of this application can be various types of electronic devices, such as mobile phones, tablets, wearable devices, learning machines, tutoring machines, learning tablets, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), large-screen TVs, etc. The embodiments of this application do not impose any restrictions on the specific type of electronic device.

[0043] Please see Figure 1 This illustrates a question location method provided by an embodiment of this application, including:

[0044] 101. Obtain the image of the book page the user is currently writing on and related information about the user's writing actions;

[0045] The executing entity of this application embodiment can be an electronic device with teaching functions, such as a tutoring machine or a learning tablet. When a user is doing homework in front of the electronic device using a book, the front-facing camera of the electronic device can capture an image of the page in front, thereby obtaining a real-time image of the page the user is currently writing on. On the other hand, the electronic device will also acquire relevant information about the user's writing actions, which may include, but is not limited to, pen tip position, finger position, writing duration, handwriting information, and user gaze information, etc. Among them, pen tip position, finger position, and handwriting information can be obtained by recognizing the captured image of the page; writing duration can be obtained by combining page-turning action detection with a timer. For example, a timer is started the instant the user's page-turning action is detected to record the duration of the user's writing on the current page. Then, when the next page is turned, the timer is reset, and the duration of the user's writing on the next page is recorded, and so on; user gaze information can be obtained by an eye tracker or a camera in a suitable position, mainly used to determine the user's gaze position when writing on the current page.

[0046] In one implementation of this application, before obtaining the image of the book page currently being written on by the user and related information about the user's writing action, the method may further include:

[0047] (1) Detect whether there are any obstructions in front of the electronic equipment;

[0048] (2) If there is an obstruction in front of the electronic device, detect whether there is a human face and a book in front of the electronic device;

[0049] (3) If there is a human face and a book in front of the electronic device, control the electronic device to enter the specified scene mode.

[0050] Electronic devices can detect the presence of obstructions in front of them in real time. Face detection is only initiated when an obstruction is present to further determine if the obstruction represents a person. If no obstruction is present, it indicates that no person is in front of the electronic device, thus eliminating the need for face detection and reducing resource consumption. Specifically, when detecting obstructions, infrared photodiodes or ranging sensors can be used, along with detection methods based on direct time-of-flight (DTFO) or indirect time-of-flight (ITFO), etc. This application does not limit the specific method for detecting obstructions. If an obstruction is detected, the front-facing camera can be activated to initiate face and book detection. In practice, the image can be analyzed to identify whether both a face and a book are present simultaneously. Face detection algorithms can be used for face recognition, while deep learning-based book detection algorithms can be used for book recognition. If the electronic device detects both a face and a book simultaneously, it can likely determine that the user is doing homework or reading, thereby controlling the electronic device to enter a specified scenario mode, such as a homework mode or a reading mode. In addition, to improve the accuracy of scene mode control, the judgment can be made in combination with the user's work time pattern. For example, when a face and a book are detected in front of the electronic device at the same time, and the current time is within the user's normal work time range, the electronic device is controlled to enter the corresponding scene mode.

[0051] Furthermore, after controlling the electronic device to enter the specified scene mode, it may also include:

[0052] (1) Detect the type of book;

[0053] (2) Control the electronic device to enter the function mode corresponding to this type.

[0054] Electronic devices can further detect the type of book the user is currently writing on, thus entering the corresponding functional mode. For example, if the book is a textbook, it enters functional mode one: textbook learning mode; if the book is a workbook, it enters functional mode two: workbook learning mode, and so on. After entering different functional modes, the electronic device can recommend targeted auxiliary learning functions, such as knowledge point summarization and point-and-read functions in textbook learning mode. When detecting the book type, a full-page image search can be performed using captured images of the book pages, matching them with a local or server-stored book database (such as a textbook database or workbook database). The book type can be determined based on the matching results.

[0055] In addition, after matching the scene mode and the corresponding function mode, the electronic device can activate the silent protection function. For example, when a user is doing homework using a textbook, the electronic device can activate the homework mode and enter the textbook learning mode. Afterward, the electronic device can enter a black screen state, silently protecting the user while they do their homework, without disturbing them. At this time, the front-facing camera of the electronic device will continuously monitor the user's writing, handwriting, and posture. When the user asks a question, the electronic device can immediately answer it, and when the user's posture is poor, the electronic device can also immediately remind them.

[0056] 102. Based on relevant information and images on the book page, determine the target topic that the user is currently writing about.

[0057] After obtaining the image of the book page the user is currently writing on and information related to the user's writing actions, the system can determine the question the user is currently writing on based on this information and the book page image. This question is represented as the target question. With this setup, users no longer need to manually take photos or select questions; they can simply write normally on the book page. The electronic device will automatically detect the user's writing actions and, combined with this information and the book page image, locate the question the user is currently writing on, thus simplifying the user experience.

[0058] In one implementation of this application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; determining the target topic that the user is currently writing based on the relevant information and the book page image may include:

[0059] (1) Determine the area of ​​the question that the user is currently writing from the image of the book page based on the position of the pen tip;

[0060] (2) Identify each question in the question area as the target question.

[0061] The acquired information can include the position of the pen tip in the image of the book page. Based on this pen tip position, the area where the user is currently writing can be determined from the book page image, and each question within that area can be identified as a target question. For example, assuming the book page image is an A4 page, it can be divided into three parts: the upper, middle, and lower areas. The area where the pen tip is located can then be identified as the area where the user is currently writing. This area may contain multiple questions (e.g., 2-3 questions), and one of these questions is the question the user is currently writing. It can be seen that this is a fuzzy localization method, meaning it can only locate 2-3 questions, including the one the user is currently writing, but cannot definitively pinpoint the question being written. Furthermore, after determining the area, each question within that area can be displayed as a target question on the screen, for example, on an electronic device. Besides the pen tip position, the finger or hand position can also be used to identify the area where the user is currently writing, and the specific implementation method is the same as for pen tip position.

[0062] In another implementation of this application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; determining the target topic that the user is currently writing based on the relevant information and the book page image may include:

[0063] (1) Obtain the position of each question box in the image of the book page;

[0064] (2) Detect the distance between the pen tip position and the selected position of each question box respectively;

[0065] (3) Select the questions within the question selection area with the smallest distance from the pen tip as the target questions.

[0066] By recognizing images of book pages, the position of the pen tip and the selection areas for each question can be determined. Each selection area can be a closed box (e.g., a rectangle) used to select a single question. The distance between the pen tip position and each selection area is detected, and the selection area with the smallest distance is identified. The question within this selection area is then designated as the target question the user is currently writing. Similarly, after determining the selection area closest to the pen tip position, the single question within that selection area can be displayed on a screen, such as an electronic device. In addition to the pen tip position, the position of the finger or hand can also be compared with the selection areas, using the same method as for the pen tip position.

[0067] In another implementation of this application, the writing pen used by the user is equipped with a pen-body camera, and the relevant information includes an image of the question stem captured by the pen-body camera; determining the target question that the user is currently writing based on the relevant information and the image of the book page may include:

[0068] (1) Perform OCR recognition on the question stem image to obtain the question stem information of the question that the user is currently writing;

[0069] (2) Search for questions that match the question stem information from the question bank stored in the background, and use them as target questions.

[0070] When identifying the pen tip position based on a book page image, if the pen tip is obscured by the user's hand, the pen tip position cannot be accurately identified, leading to a decrease in the accuracy of locating the target question using the pen tip position. To address this, a miniature pen-body camera can be incorporated into the writing pen. This camera can be installed below the pen's grip area to capture a portion of the question stem information currently being written by the user. After obtaining the question stem information image captured by the pen-body camera, the electronic device can perform OCR recognition on the image to obtain a portion of the question stem information currently being written by the user. Then, this portion of the question stem information is used to match corresponding questions in the system backend. For example, questions matching the portion of the question stem information can be searched from a question database stored in the backend, and selected as the target question. This question database can be stored locally on the electronic device or on a server, and can include questions from various textbooks, exercise books, or test papers. In practice, the portion of the question stem information recognized by OCR can be matched with the question stem information of each question in the question database to find the question with the highest matching degree, which is then selected as the target question.

[0071] Furthermore, the step of searching for questions matching the question stem information from the question bank stored in the background as target questions may include:

[0072] (1) Determine the target page of the book that the user is currently writing on based on the image of the book page;

[0073] (2) Search for questions that match the question stem information from the questions stored in the question bank on the target book page, and use them as target questions.

[0074] Because the question bank contains a large number of questions, matching the partial question stem information recognized by OCR with all the questions in the bank would be time-consuming and inaccurate. To address this issue, we can first identify the page the user is currently writing on based on the image of the book page, representing it as the target book page. Specifically, we can match the book page image with scanned pages from a book or textbook library stored locally on the electronic device or server to determine which page of which book the image belongs to, i.e., the target book page. Then, we search the question bank for each question on the target book page, matching the partial question stem information recognized by OCR with the question stem information of each question on the target book page, and selecting the question with the highest match as the target question. By setting it up this way, we only need to match the partial question stem information recognized by OCR with a small number of questions stored in the question bank (each question on the target book page), thereby reducing the time spent on the matching operation and improving the matching accuracy.

[0075] In another implementation of this application, the relevant information includes the duration of the user's writing action; determining the target topic the user is currently writing based on the relevant information and the image of the book page may include:

[0076] (1) Determine the target page of the book that the user is currently writing on based on the image of the book page;

[0077] (2) Based on the duration, the order of the questions on the target book page, and the estimated writing time, determine the target questions from the questions on the target book page.

[0078] As mentioned earlier, by matching scanned pages from a book library or textbook library stored locally on the electronic device or server with a book page image, it is possible to determine which page of which book the image belongs to, thus identifying the page the user is currently writing on. This page is represented by the target book page. Using page-turning action detection and a set timer, the duration of the user's writing on the target book page can be calculated. Then, based on the known order of the questions within the target book page and the estimated writing time, the target question the user is currently writing on can be determined from among the questions on the target book page. Specifically, since all the questions within the target book page are known, an estimated writing time can be pre-set for each question; for example, the estimated writing time for a fill-in-the-blank question is 2 minutes, and the estimated writing time for a calculation question is 10 minutes, and so on. Furthermore, since the order of these questions is also known, after obtaining the duration of the user's writing on the target book page, the target question the user is currently writing on can be estimated. For example, if the questions on the target book page are, in order, fill-in-the-blank question 1 (estimated writing time 2 minutes), fill-in-the-blank question 2 (estimated writing time 2 minutes), fill-in-the-blank question 3 (estimated writing time 2 minutes), calculation question 1 (estimated writing time 10 minutes), and calculation question 2 (estimated writing time 10 minutes), and the duration is 9 minutes, then it can be determined that the target question the user is currently writing is calculation question 1.

[0079] In one implementation of this application, after determining the target topic that the user is currently writing about, the method may further include:

[0080] When a user's target interaction action is detected, the target display device is controlled to display learning content associated with the target question.

[0081] After locating the target question the user is currently writing about, the electronic device can control a target display device (such as the screen of the electronic device itself, or other display devices connected to the electronic device) to display learning content associated with the target question. In practice, the user only needs to perform a target interaction action, and the electronic device, upon detecting the user's target interaction action, controls the target display device to display the learning content associated with the target question. There are two implementation methods: the first is that the electronic device searches for and displays learning content associated with the target question only after detecting the user's target interaction action; the second is that the electronic device searches for learning content associated with the target question first, and then displays the learning content only when the user's target interaction action is detected. The learning content associated with the target question can include, but is not limited to: question video analysis, question text analysis, answer analysis, summary comments, thought process inspiration, knowledge point analysis, etc. The target interaction action can include, but is not limited to: looking up at the screen, shaking the head, nodding, waving, pointing at the screen, etc. Obviously, with this setting, the user only needs to perform a simple interaction action, and the electronic device will automatically display the learning content associated with the currently writing question on the screen. For example, when a user encounters a difficult problem while doing homework, they only need to look up at the screen of the electronic device, and the electronic device will automatically display the solution to the problem on the screen, making the operation very convenient.

[0082] In this embodiment, the tutoring device or other electronic device can acquire an image of the book page the user is currently writing on, as well as information related to the user's writing actions. Then, based on this information and the book page image, it determines the target question the user is currently writing on. With this setup, the user does not need to manually take photos or select questions; they can simply write normally on the book page. The electronic device automatically detects information related to the user's writing actions, such as pen tip position, finger position, and writing duration. Combining this information with the book page image, it locates the question the user is currently writing on, thus simplifying the user's operation.

[0083] like Figure 2 The diagram shown illustrates an operation flow of the electronic device provided in this application when locating questions. Figure 2The process involves several steps, including active triggering, scene recognition, homework protection, problem location, and problem analysis. Active triggering means the electronic device can proactively detect obstructions, faces, and books in front of it, thus initiating the homework mode. Scene recognition means the electronic device can detect images of book pages on the desk, identify the book type through a whole-page image search, and enter the corresponding learning mode, such as textbook learning mode, workbook learning mode, etc. Homework protection means that once the electronic device enters the corresponding learning mode, it will remain silent with the screen off, but the camera will continue to monitor the book page and the user's writing actions in real time. Problem location means that the electronic device can locate the problem or area the user is currently writing on based on information related to the user's writing actions and the captured book page images. Problem analysis means that when the user makes interactive actions such as looking up at the screen, the electronic device can detect the face, light up the screen, and display learning content such as explanatory videos, answer explanations, or summaries related to the problem the user is currently writing on.

[0084] To facilitate understanding of the question location method provided in the embodiments of this application, two practical application scenarios are listed below.

[0085] like Figure 3 The diagram shown is an operational schematic of a question location method provided in an embodiment of this application. Figure 3 The image displayed is of the current book page as captured by the electronic device. This page is divided into three parts: the upper region, the middle region, and the lower region. Since the detected pen tip position is in the lower region of the book page image, the lower region is identified as the area where the user is currently writing the question. Furthermore, questions 6 and 7 within the lower region are identified as the target questions the user is currently writing, thus achieving fuzzy question localization.

[0086] like Figure 4 The diagram shown is an operational schematic of another question location method provided in an embodiment of this application. Figure 4 The display shows the image of the current book page acquired by the electronic device, and has identified each question box in the current book page image, with each question box containing a single question. Since the detected pen tip position is closest to question box 4, question 4 contained in question box 4 is identified as the target question that the user is currently writing, thus achieving accurate question positioning.

[0087] In summary, the embodiments of this application can locate the question by combining the image of the book page the user is currently writing on and the relevant information of the user's writing actions, which can effectively simplify the user's operation.

[0088] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0089] The above mainly describes a question location method; the following will describe a question location device.

[0090] Please see Figure 5 One embodiment of a question location device in this application includes:

[0091] The information acquisition module 501 is used to acquire an image of the book page currently being written on by the user and related information about the writing action being performed by the user;

[0092] The question location module 502 is used to determine the target question that the user is currently writing based on the relevant information and the image of the book page.

[0093] In one implementation of this application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; the question location module may include:

[0094] The question area determination unit is used to determine the question area that the user is currently writing in from the book page image based on the pen tip position;

[0095] The first question location unit is used to identify each question in the question area as the target question.

[0096] In one implementation of this application, the relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; the question location module may include:

[0097] The question box acquisition unit is used to acquire the position of each question box in the book page image;

[0098] A distance detection unit is used to detect the distance between the pen tip position and each of the question selection boxes;

[0099] The second question positioning unit is used to identify the questions within the question selection frame that have the smallest distance from the pen tip position as the target questions.

[0100] In one implementation of this application, the writing pen used by the user is equipped with a pen-body camera, and the relevant information includes an image of the question stem captured by the pen-body camera; the question location module may include:

[0101] The question stem information recognition unit is used to perform OCR recognition on the question stem information image to obtain the question stem information of the question that the user is currently writing.

[0102] The question matching unit is used to search for questions that match the question stem information from the question bank stored in the background, and use them as the target questions.

[0103] Furthermore, the question matching unit may include:

[0104] The book page determination subunit is used to determine the target book page that the user is currently writing on based on the book page image;

[0105] The question matching subunit is used to search for questions that match the question stem information from the questions stored in the question bank within the target book page, and use these questions as the target questions.

[0106] In one implementation of this application, the relevant information includes the duration of the user's writing action; the question location module may include:

[0107] The book page determination unit is used to determine the target book page that the user is currently writing on based on the book page image;

[0108] The third question location unit is used to determine the target question from the questions on the target book page based on the duration, the order of the questions on the target book page, and the estimated writing time.

[0109] In one implementation of this application, the question location device may further include:

[0110] The learning content display module is used to control the target display device to display learning content associated with the target question when the user's target interaction action is detected.

[0111] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the question location method as described in any of the above embodiments.

[0112] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to execute the question location method as described in any of the above embodiments.

[0113] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of this application. Figure 6As shown, the electronic device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the embodiments of the various question location methods described above, for example... Figure 1 Steps 101 to 102 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The functions of modules 501 to 502 are shown.

[0114] The computer program 62 can be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 62 in the electronic device 6.

[0115] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0116] The memory 61 can be an internal storage unit of the electronic device 6, such as a hard disk or memory. The memory 61 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 61 can include both internal and external storage units of the electronic device 6. The memory 61 is used to store the computer program and other programs and data required by the electronic device. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0117] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0118] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0119] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0120] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0125] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for locating questions, characterized in that, include: Obtain an image of the book page the user is currently writing on and information related to the user's writing action; Based on the relevant information and the image of the book page, determine the target topic that the user is currently writing on; The relevant information includes the duration of the user's writing on the target book page they are currently writing on; determining the target topic the user is currently writing on based on the relevant information and the book page image includes: The target book page is determined based on the book page image; The target question is determined from the questions on the target book page based on the duration, the order of the questions on the target book page, and the estimated writing time.

2. The method as described in claim 1, characterized in that, The relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; determining the target topic that the user is currently writing based on the relevant information and the book page image includes: Based on the pen tip position, determine the area of ​​the question that the user is currently writing on from the image of the book page; Each question within the question area is identified as the target question.

3. The method as described in claim 1, characterized in that, The relevant information includes the position of the pen tip of the writing pen used by the user in the book page image; determining the target topic that the user is currently writing based on the relevant information and the book page image includes: Obtain the selected positions of each question box in the image of the book page; Detect the distance between the pen tip position and each of the question selection boxes; The questions within the question selection frame that have the smallest distance from the pen tip position are identified as the target questions.

4. The method as described in claim 1, characterized in that, The writing pen used by the user is equipped with a camera, and the relevant information includes an image of the question stem captured by the pen's camera; determining the target question that the user is currently writing based on the relevant information and the image of the book page includes: The question stem information image is subjected to OCR recognition to obtain the question stem information of the question that the user is currently writing; Search the question bank stored in the background for questions that match the question stem information, and use them as the target questions.

5. The method as described in claim 4, characterized in that, The step of searching for a question that matches the question stem information from the question bank stored in the background, and using it as the target question, includes: Based on the image of the book page, determine the target book page that the user is currently writing on; The question that matches the question stem information is selected from the questions stored in the question bank within the target book page and used as the target question.

6. The method according to any one of claims 1 to 5, characterized in that, After determining the target topic that the user is currently writing about, the process also includes: When the user's target interaction action is detected, the target display device is controlled to display learning content associated with the target question.

7. A question positioning device, characterized in that, include: The information acquisition module is used to acquire an image of the book page currently being written on by the user and related information about the writing action being performed by the user; The question location module is used to determine the target question that the user is currently writing based on the relevant information and the image of the book page; The relevant information includes the duration of the user's writing on the target book page they are currently writing on; the question location module includes: A book page determination unit is used to determine the target book page based on the book page image; The third question location unit is used to determine the target question from the questions on the target book page based on the duration, the order of the questions on the target book page, and the estimated writing time.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the question location method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the question location method as described in any one of claims 1 to 6.

Images