Linkage analysis method and device, equipment and storage medium

Abstract

The invention provides a linkage analysis method, device and equipment and a storage medium, through cooperative work of double cameras, a first camera is used for giving out a question answering result in real time, a second camera is linked to capture user behavior data in the display process of the question answering result, and the user behavior data is analyzed and judged, so that the user experience is improved. According to the method, the questions related to the knowledge points or similar to the text content of the target question are recommended, so that personalized learning content recommendation adaptive to the understanding level of the current user can be dynamically provided for the user, the learning efficiency can be improved, the user is helped to better consolidate the knowledge points, and the learning experience is optimized.

Description

Technical Field

[0001] The present application relates to the field of intelligent teaching technology, and in particular to a linkage analysis method, device, equipment and storage medium. Background Art

[0002] With the rapid development of technology, the use of intelligent learning devices, such as learning machines and smart tablets, is becoming increasingly popular in education. Using intelligent learning devices to complete homework and conduct independent learning has become a new trend in modern education, providing a new way to improve learning efficiency and optimize learning outcomes.

[0003] However, current smart learning devices on the market still have certain functional limitations. Primarily, most are limited to providing basic functions such as e-textbooks, online courses, and question banks. They lack in-depth analysis of students' behavioral characteristics during homework and their grasp of key knowledge points, making it difficult to provide personalized tutoring tailored to students' current understanding levels. Summary of the Invention

[0004] In view of this, the present application provides a linkage analysis method, apparatus, device and storage medium to address the deficiencies in the related art.

[0005] In a first aspect of the present application, a linkage analysis method is provided, which is applied to an electronic device equipped with a first camera and a second camera, wherein the first camera is used to capture a desktop image and the second camera is used to capture a user image. The method includes:

[0006] Recognize the desktop image captured by the first camera to obtain a target question to be answered, and display a target answer result corresponding to the target question;

[0007] During the display of the target question answering result, controlling the second camera to capture a user image and recognize the user image to obtain user behavior data, wherein the user behavior data at least includes user expression data;

[0008] If it is determined based on the user behavior data that a first type of topic with knowledge points associated with the target topic is recommended, the first type of topic is obtained and displayed; if it is determined based on the user behavior data that a second type of topic with text content that meets similarity requirements with the target topic is recommended, the second type of topic is obtained and displayed;

[0009] In response to a selection operation performed by the user on any displayed question, the question-answering result corresponding to the selected question is displayed.

[0010] According to an embodiment of the present application, if it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes:

[0011] Acquire first expression data, where the first expression data is expression data in the user expression data that meets a first determination condition and is used to represent an uncertain emotion;

[0012] If the first expression data meets the first setting requirement, determining to recommend the first category of topics;

[0013] Based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving knowledge points that are a subset of the target question's knowledge points are obtained from a preset question bank, and the obtained questions are displayed.

[0014] According to an embodiment of the present application, if it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes:

[0015] Acquire second expression data, where the second expression data is expression data in the user expression data that meets a second determination condition and is used to represent a positive emotion;

[0016] If the second expression data meets the second setting requirement, determining to recommend the first category of topics;

[0017] Based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving more knowledge points or having more knowledge points than the target question are obtained from the preset question bank, and the obtained questions are displayed.

[0018] According to one embodiment of the present application, if it is determined based on the user behavior data that a second category of topics whose text content meets the similarity requirement with the target topic is recommended, obtaining the second category of topics and displaying them includes:

[0019] Acquiring third expression data, where the third expression data is expression data in the user expression data that meets a third determination condition and is used to represent a negative emotion;

[0020] If the third expression data meets the third setting requirement, determining to recommend the second category of questions;

[0021] Then, a plurality of questions whose text contents meet the similarity requirement with the target question are obtained from a preset question bank, and the obtained questions are displayed.

[0022] According to one embodiment of the present application, the user behavior data further includes user sitting posture data. If it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes:

[0023] Determining whether there is non-standard sitting posture data that meets a fourth determination condition in the user sitting posture data, and determining whether there is first expression data that meets a first determination condition and is used to represent an uncertain emotion in the user expression data;

[0024] If the non-standard sitting posture data exists and the first expression data does not exist, it is determined that the first type of question is recommended; then, based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question, at least one question involving knowledge points that are a subset of the knowledge points of the target question, and at least one question involving more knowledge points or having a higher difficulty level than the target question is obtained from a preset question bank, and the obtained questions are displayed;

[0025] If the non-standard sitting posture data exists and the first expression data exists, it is determined to recommend the first type of questions; then, based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving knowledge points that are a subset of the target question's knowledge points are obtained from a preset question bank, and the obtained questions are displayed.

[0026] According to one embodiment of the present application, the step of recognizing the desktop image captured by the first camera to obtain a target question to be answered includes:

[0027] Recognizing the desktop image captured by the first camera to obtain a user fingertip block and a book page block in the desktop image;

[0028] Performing question frame detection on the book page block to obtain a question frame block for each question on the book page;

[0029] According to the degree of overlap between the user's fingertip block and the question frame block of each question, a target question frame block is determined, and text recognition is performed on the target question frame block to obtain the target question to be answered.

[0030] According to one embodiment of the present application, displaying the target question answering result corresponding to the target question includes:

[0031] According to the text content of the target question, searching for the question with the highest similarity in the preset question bank;

[0032] The question-and-answer results bound to the question with the highest similarity are displayed.

[0033] According to one embodiment of the present application, in response to a user selecting any displayed question, displaying the question answering result corresponding to the selected question includes:

[0034] In response to a user selecting any of the displayed questions, if the selection operation is used to instruct display of the question-answering result, the question-answering result corresponding to the selected question is directly displayed;

[0035] If the selection operation is used to instruct answering a question, the text content of the selected question is displayed, and after it is determined that the user has completed answering the question, the answer result corresponding to the selected question is displayed.

[0036] In a second aspect of the present application, a linkage analysis device is provided, which is applied to an electronic device equipped with a first camera and a second camera, wherein the first camera is used to capture a desktop image and the second camera is used to capture a user image, and the device includes:

[0037] A first question-answering unit, configured to recognize the desktop image captured by the first camera, obtain a target question to be answered, and display a target answer result corresponding to the target question;

[0038] a behavior recognition unit, configured to control the second camera to capture a user image during the display of the target question answering result, and recognize the user image to obtain user behavior data, wherein the user behavior data at least includes user expression data;

[0039] a recommendation unit configured to obtain and display a first category of questions having knowledge points associated with the target question if the user behavior data determines that the first category of questions are recommended; and obtain and display a second category of questions if the user behavior data determines that the second category of questions are recommended if the text content of the second category of questions satisfies a similarity requirement with the target question;

[0040] The second question-answering unit is configured to, in response to a selection operation performed by the user on any displayed question, display the question-answering result corresponding to the selected question.

[0041] In a third aspect of the present application, an electronic device is provided, comprising a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement the steps of the method proposed in the above embodiment.

[0042] In a fourth aspect of the present application, a machine-readable storage medium is provided, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by a processor, the steps of the method proposed in the above embodiment are implemented.

[0043] As can be seen from the above technical solution, the target question to be answered is obtained by recognizing the desktop image captured by the first camera, and the target answer result corresponding to the target question is displayed. During the display of the target answer result, the second camera is controlled to capture the user image and recognize the user image to obtain user behavior data. If the user behavior data determines that a first type of question with knowledge points related to the target question is recommended, the first type of question is obtained and displayed; if the user behavior data determines that a second type of question with text content similar to the target question is recommended, the second type of question is obtained and displayed; in response to the user's selection operation on any displayed question, the answer result corresponding to the selected question is displayed. Through the collaborative work of the two cameras, the first camera is used to provide the answer result in real time, and the second camera is linked to capture user behavior data during the display of the answer result. Through the analysis and judgment of the user behavior data, questions related to the knowledge points of the target question or with similar text content are recommended, thereby dynamically providing users with personalized learning content recommendations that are adapted to the current user's understanding level, thereby improving learning efficiency, helping users better consolidate knowledge points, and optimizing the learning experience.

[0044] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a linkage analysis method provided in an embodiment of the present application;

[0046] Figure 2 This is a flowchart of recommending a first-category topic provided by an embodiment of the present application;

[0047] Figure 3 This is a flowchart of recommending a first-category topic provided by another embodiment of the present application;

[0048] Figure 4 This is a flowchart of recommending the second category of questions provided by an embodiment of the present application;

[0049] Figure 5 This is a flowchart of recommending a first-category topic provided by another embodiment of the present application;

[0050] Figure 6 This is a schematic structural diagram of a linkage analysis device provided in an embodiment of the present application;

[0051] Figure 7 It is a schematic diagram of the hardware structure of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0053] The terms used in this application are for the purpose of describing particular embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0054] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0055] With the rapid development of technology, the use of intelligent learning devices, such as learning machines and smart tablets, is becoming increasingly popular in education. Using intelligent learning devices to complete homework and conduct independent learning has become a new trend in modern education, providing a new way to improve learning efficiency and optimize learning outcomes.

[0056] However, current smart learning devices on the market still have certain functional limitations. Primarily, most are limited to providing basic functions such as e-textbooks, online courses, and question banks. They lack in-depth analysis of students' behavioral characteristics during homework and their grasp of key knowledge points, making it difficult to provide personalized tutoring tailored to students' current understanding levels.

[0057] In view of this, an embodiment of the present application discloses a linkage analysis method to address the deficiencies in the related art.

[0058] like Figure 1 As shown, Figure 1 This is a flow chart of a linkage analysis method provided in an embodiment of the present application.

[0059] The linkage analysis method is applied to an electronic device equipped with a first camera and a second camera.

[0060] The embodiments of this application do not specifically limit the type of electronic device. For example, the electronic device may include a tablet electronic device, such as a tablet computer, a tablet learning device, a tablet game console, or a portable electronic device, such as a smartphone, an e-reader, or the like.

[0061] Furthermore, the embodiments of this application do not specifically limit the manner in which the first camera and the second camera are mounted on the electronic device. For example, the first camera and the second camera may be integrated as inherent components of the electronic device, or may be detachable components that are detachably connected to the electronic device via a standardized interface.

[0062] In addition, the embodiments of the present application do not specifically limit the locations where the first camera and the second camera are installed on the electronic device. For example, taking a tablet-type electronic device as an example, the tablet-type electronic device may include a tablet body and an external camera module, wherein the external camera module may include a first camera and a second camera, and the location where the external camera module is installed on the tablet body includes but is not limited to the front, back, or frame. For example, the external camera module is installed on the frame of the tablet body, specifically, it can be installed on one of the top frames when the tablet body is in use.

[0063] It should be noted that for electronic devices equipped with a first camera and a second camera, the second camera can be oriented toward the user to capture images of the user, including but not limited to image information such as the user's sitting posture, demeanor, and facial expressions. The first camera can be oriented toward the desktop in front of the user to capture images of the desktop in front of the user, including but not limited to image information such as book pages, written content, questions, toys, stationery, and the user's fingers.

[0064] The linkage analysis method may include the following steps:

[0065] S101: Recognize the desktop image captured by the first camera to obtain a target question to be answered, and display a target answer result corresponding to the target question.

[0066] The shooting direction of the first camera can face the desktop in front of the user, and is used to capture the desktop image on the desktop in front of the user in real time.

[0067] In some embodiments, the electronic device can automatically trigger the first camera to capture a desktop image in real time upon power-up. Of course, the embodiments of this application do not specifically limit the method for "triggering the first camera to capture a desktop image in real time." In practical applications, other methods may include application startup triggering, manual triggering, behavior recognition triggering, voice control triggering, and biometric triggering.

[0068] In some embodiments, real-time single-frame recognition can be used, that is, image recognition is directly performed on the desktop image captured by the first camera in real time. Alternatively, frame extraction recognition can be used, that is, frame extraction is performed on the desktop image captured by the first camera in real time, and image recognition is performed on the extracted desktop image. For example, for the N frames of desktop images captured by the first camera within a preset time period (such as 1 second), a comprehensive score is calculated for each frame of the desktop image, such as a clarity score, structural integrity assessment, etc., and the frame with the highest comprehensive score is extracted for image recognition. It should be noted that the embodiments of the present application do not specifically limit the method of "recognizing the desktop image captured by the first camera". In actual applications, multi-frame fusion recognition, hybrid recognition, etc. can also be used.

[0069] In some embodiments, the desktop image captured by the first camera may be preprocessed, such as scaling to a fixed size, color space conversion, noise suppression, and image data standardization, and the preprocessed image may be recognized.

[0070] The target question to be answered may be a question specified by the user through a specific interaction method, and the electronic device is required to provide an answer result.

[0071] For example, specific interaction methods may include physical pointing interactions, such as a user's fingertip pointing at a target question, or a stylus tip pointing at a target question; gesture interactions, such as framing a target question with a gesture, or circling a target question with a stylus, etc. The embodiments of this application do not specifically limit specific interaction methods.

[0072] For example, the Q&A results include but are not limited to basic Q&A information, such as standard answers to questions, detailed problem-solving steps, analysis of relevant knowledge points, etc.; multimedia explanation resources, such as graphic and text explanation materials, video analysis tutorials, interactive demonstration animations, etc.

[0073] In some embodiments, the target question to be answered refers to the question pointed to by the user's fingertip. The desktop image captured by the first camera is recognized to obtain the target question to be answered, including:

[0074] The desktop image captured by the first camera is recognized to obtain the user fingertip block and the book page block in the desktop image respectively. Specifically, the desktop image captured by the first camera can be input into a pre-trained fingertip detection model, which can recognize and output the specific coordinate position of the user's fingertip in the image. According to the coordinate position of the user's fingertip, the area where the user's fingertip is located can be determined, that is, the user's fingertip block. At the same time, the desktop image captured by the first camera is input into a pre-trained book page detection model, which can recognize and output the coordinate position of the book page in the image. According to the coordinate position of the book page, the area where the book page is located can be determined, that is, the book page block. Exemplarily, the fingertip detection model can adopt models such as YOLO v8 and EfficientDet. The book page detection model can adopt models such as DINO (DETR with Improved Denoising Anchor Boxes) and Swin Transformer. In practical applications, both can adopt other models, and the embodiments of the present application do not make specific limitations on this.

[0075] The title frame detection is performed on the book page block to obtain the title frame block of each question in the book page. Specifically, after obtaining the book page block in the desktop image, the book page block can be cropped out from the desktop image. The cropped book page block image is preprocessed, such as scaling to a fixed size, image data standardization, etc., to adapt to the model input requirements. The preprocessed image is input into a pre-trained title frame detection model, which can identify and output the coordinate position of the title frame of each question in the image. According to the coordinate position of the title frame, the area enclosed by the title frame of each question in the book page, that is, the title frame block, can be determined. Exemplarily, the title frame detection model can adopt models such as YOLO v8 and EfficientDet. In practical applications, the embodiments of the present application do not make specific limitations on this.

[0076] According to the degree of overlap between the user's fingertip block and the question frame block of each question, the target question frame block is determined, and text recognition is performed on the target question frame block to obtain the target question to be answered. Specifically, the coordinates of the user's fingertip block are first converted to the coordinate system of the book page block image to realize coordinate system one. The degree of overlap (IoU, Intersection over Union) between the user's fingertip block and the question frame block of each question is calculated, and the question frame block with the largest overlap with the user's fingertip block is selected as the target question frame block. Text recognition is performed on the target question frame block, such as extracting the text content in the target question frame block through optical character recognition OCR (Optical Character Recognition) to obtain the target question to be answered.

[0077] In this embodiment, by introducing a selection mechanism based on the degree of overlap, the accuracy and reliability of locating the target topic are effectively improved.

[0078] In some embodiments, displaying the target question answering result corresponding to the target question includes:

[0079] According to the text content of the target question, the question with the highest similarity is retrieved in the preset question bank. Specifically, the text content of the target question can be analyzed to extract key features, including but not limited to question type, knowledge points, question structure, and semantic features, etc. Utilize similarity algorithms, such as cosine similarity, Jaccard coefficient, etc., to compare the extracted key features with the features of each question in the preset question bank, and calculate the similarity score. Sort the questions in the question bank according to the similarity score, and finally select the question with the highest similarity score as the retrieval result. Exemplarily, the question bank can be an ES search library built with ES (Elasticsearch) as the core retrieval engine. Of course, the embodiments of the present application do not specifically limit the construction method of the question bank, and other high-performance retrieval solutions, such as Faiss, Milvus, etc., can also be used in actual applications.

[0080] The question answering result bound to the question with the highest similarity is displayed. Specifically, the search results contain the question ID of the question with the highest similarity, and the question answering result bound to the question is obtained based on the question ID, and the question answering result is displayed on the display screen of the electronic device to show the question answering result to the user.

[0081] S102: During the display of the target question answering result, the second camera is controlled to capture a user image, and the user image is recognized to obtain user behavior data, where the user behavior data at least includes user expression data.

[0082] The second camera can be oriented toward the user to capture an image of the user. When the target Q&A results begin to display, the second camera is activated to capture the user image. This image capture process continues until the target Q&A results are fully displayed. After the target Q&A results are fully displayed, image recognition is performed on the captured user image to obtain user behavior data, which includes at least user expression data.

[0083] In some embodiments, full-frame recognition can be used, i.e., all user images collected during the display of the target question-answering results are recognized frame by frame. Alternatively, frame-by-frame recognition can be used, i.e., frames of user images collected during the display of the target question-answering results are sampled and recognized. For example, for N frames of user images collected within a preset time period (e.g., 1 second) during the display process, a comprehensive score, such as a clarity score, structural integrity assessment, etc., is calculated for each frame of the user image, and the frame with the highest comprehensive score is extracted for recognition.

[0084] User behavior data is recorded in time series, with each time point associated with a set of user behavior data. Each set of user behavior data can include information from one or more dimensions, including at least user expression data, such as happiness, joy, confusion, hesitation, anger, and rage.

[0085] In some embodiments, user images are recognized to obtain user behavior data, including:

[0086] The user image is fed into a pre-trained face detection model, such as a multi-task cascaded convolutional neural network (MTCNN) or a retina network (RetinaNet). This model can identify the facial region in the user image and output the coordinates of the facial bounding box. Based on the face detection results, the face image is cropped from the user image and pre-processed, such as image alignment, lighting correction, and size normalization. The pre-processed face image is fed into a pre-trained expression recognition model, such as VGG-Face or FaceNet. This model can recognize and output user expressions, including but not limited to happiness, joy, confusion, hesitation, anger, and rage. The resulting user expression data is associated with the time point when the corresponding user image was collected to form a set of user behavior data.

[0087] In addition, user behavior data can also be extended to other dimensions, including but not limited to: user sitting posture data, user handheld item data and / or user leaving the table data.

[0088] In some embodiments, the user behavior data also includes user posture data. The user image is recognized to obtain user behavior data, including:

[0089] User images are fed into a pre-trained sitting posture detection model, such as ResNet18 or EfficientNet. This model can identify human body regions in the user image and recognize and classify human postures. The model outputs sitting postures including, but not limited to, standard sitting (upright body, arms placed naturally), forward leaning (body leaning forward close to the table), backward leaning (body leaning back against the chair back), sideways (body leaning to either side), lying on the table (upper body lying on the table), and lying on the back (body fully leaning back). The resulting user sitting posture data is associated with the time point of the corresponding user image collection to form a set of user behavior data.

[0090] In some embodiments, user behavior data also includes data on items held by the user. User behavior data is obtained by recognizing the user image, including:

[0091] The user image is fed into a pre-trained handheld object detection model, such as YOLO v8 or EfficientDet. This model can identify the hand region in the user image and perform object detection and classification on the hand region. The output of the model includes, but is not limited to, stationery (such as pens, compasses, correction tapes), daily necessities (such as water cups, paper towels, glasses), and electronic products (such as smartphones, headphones, and electronic watches). Unclear or uncommon items are categorized as "other." The obtained user handheld object data is then associated with the time point when the corresponding user image was collected to form a set of user behavior data.

[0092] In some embodiments, user behavior data also includes user leaving table data. User image recognition is performed to obtain user behavior data, including:

[0093] The user image is fed into a pre-trained desk-leaving detection model, such as YOLO v8 or EfficientDet. This model can identify the human body and desktop area in the user image and determine the user's status by analyzing the spatial relationship between the two. The model outputs two status categories: desk-leaving state (the human body is completely separated from the desktop area) and desk-standing state (the human body overlaps the desktop area). The resulting desk-leaving data is then associated with the time the corresponding user image was collected to form a set of user behavior data.

[0094] In some embodiments, a fourth judgment condition is pre-set for identifying non-standard sitting posture data from the user's sitting posture data, for example. The fourth judgment condition can be defined based on the detection results of key points of the human body, such as spinal curvature, head tilt angle, shoulder position, etc., including but not limited to the judgment conditions of angle threshold and time threshold. For example, the judgment condition of the angle threshold can be set as: the forward tilt angle of the key point of the human body exceeds a set threshold (such as 30, 35 degrees), the backward tilt angle exceeds a set threshold (such as 20, 25 degrees) and / or the side tilt angle exceeds a set threshold (such as 10, 15 degrees), and the judgment condition of the time threshold can be set as: when the user's sitting posture exceeds the above-mentioned angle threshold, the duration of this state exceeds a set threshold (such as 3, 5 seconds).

[0095] If the user's sitting posture data contains non-standard sitting posture data that meets the fourth judgment condition, a prompt message will be displayed to the user, including but not limited to displaying a prompt message on the device interface, playing a voice prompt and / or vibration feedback, to remind the user to adjust their sitting posture and develop good study habits. Exemplary non-standard sitting posture data that meets the fourth judgment condition include but are not limited to lying on the table, lying on your back, and other sitting postures.

[0096] In some embodiments, it can be determined whether the target handheld item category, such as electronic products, snacks, etc., exists in the user's handheld item data. If the target handheld item category exists and any of the following conditions are met, a prompt message is displayed to the user, including: the cumulative number of occurrences of the target handheld item category exceeds a set threshold (e.g., 5 times) or the duration of a single occurrence of the target handheld item category exceeds a set threshold (e.g., 20 seconds).

[0097] In some embodiments, it is possible to determine whether there is an away status in the user away status data. If there is an away status and any of the following conditions is met, a prompt message is displayed to the user. The conditions include: the cumulative number of occurrences of the away status exceeds a set threshold (such as 5 times) and the duration of a single away status exceeds a set threshold (such as 20 seconds).

[0098] S103: If it is determined based on the user behavior data that a first type of topic having knowledge points associated with the target topic is recommended, the first type of topic is obtained and displayed; if it is determined based on the user behavior data that a second type of topic having text content that meets similarity requirements with the target topic is recommended, the second type of topic is obtained and displayed.

[0099] Based on the analysis and judgment of user behavior data, it can be determined to recommend to the user the first type of questions with relevant knowledge points to the target question, or to recommend the second type of questions whose text content meets the similarity requirements with the target question, and obtain the corresponding questions for display.

[0100] For example, a list of the acquired first / second category questions may pop up on a side of the display screen of the electronic device to be presented to the user for selection.

[0101] In some embodiments, if it is determined based on the user behavior data that the first / second category of topics is recommended, obtaining the first / second category of topics and displaying them includes:

[0102] According to the preset judgment conditions, the specified data that meets the judgment conditions is obtained from the user behavior data; if the specified data meets the corresponding set requirements, it is determined to recommend the first / second category questions to the user, and the first / second category questions are obtained and displayed.

[0103] Exemplarily, the preset judgment conditions may include: a first judgment condition, used to obtain specified data for representing uncertain emotions from the user's expression data, recorded as the first expression data, including confusion, bewilderment, hesitation, etc.; a second judgment condition, used to obtain specified data for representing positive emotions from the user's expression data, recorded as the second expression data, including happiness, excitement, pleasure, etc.; a third judgment condition, used to obtain specified data for representing negative emotions from the user's expression data, recorded as the third expression data, including anger, rage, etc.

[0104] For example, the corresponding set requirement may be that the cumulative number of occurrences of the specified data exceeds a set threshold and / or the duration of a single specified data exceeds a set threshold.

[0105] Exemplarily, the first category of questions having knowledge points associated with the target question may include questions having the same knowledge points as the target question, questions involving knowledge points that are a subset of the target question's knowledge points, or questions involving more knowledge points or having a higher difficulty level than the target question.

[0106] For example, the similarity requirement can be a top N similarity ranking. That is, the second category of recommended questions whose text content meets the similarity requirement with the target question refers to the questions in the recommended question bank whose text content similarity with the target question ranks in the top N. Of course, the embodiments of this application do not specifically limit the similarity requirement. For example, the similarity requirement can also be a similarity threshold, a similarity ratio, etc.

[0107] For a detailed description of this embodiment, please refer to the description below, which will not be described in detail here.

[0108] S104: In response to the user selecting any of the displayed questions, the question-answering result corresponding to the selected question is displayed.

[0109] After obtaining and displaying the first / second category questions as described in S103, the user can select any of the displayed questions. When the user's selection operation is detected, the question-answering result corresponding to the selected question is obtained in response to the selection operation and displayed to the user.

[0110] For example, the user may select any displayed question by clicking, touching, or other interactive methods, which is not specifically limited in the embodiments of the present application.

[0111] In some embodiments, in response to a user selecting any displayed question, the answer result corresponding to the selected question is displayed, which may include the following two scenarios:

[0112] Scenario 1: Directly display the Q&A results.

[0113] In response to a selection operation performed by the user on any displayed question, if the selection operation is used to instruct to directly display the question-answering result, the question-answering result corresponding to the selected question is directly displayed.

[0114] For example, for any displayed question, the user can select the question by triggering a preset control (such as a "view answer" button) for indicating that the answer to the question corresponding to the question should be displayed directly. In response to this selection operation, the answer to the question corresponding to the question is directly displayed.

[0115] Scenario 2: Answer the questions first, then display the results.

[0116] In response to the user's selection operation on any displayed question, if the selection operation is used to indicate answering the question, the text content of the selected question is displayed, and after determining that the user has completed answering the question, the answer result corresponding to the selected question is displayed.

[0117] For example, for any displayed question, the user can select the question by triggering a preset control (such as a "Start Answering" button) for instructing to answer the question. In response to this selection, the text content of the question is displayed, and after detecting that the user has completed the answer, the answer result corresponding to the question is displayed.

[0118] In an embodiment of the present application, a target question to be answered is obtained by recognizing a desktop image captured by a first camera, and a target answer result corresponding to the target question is displayed. During the display of the target answer result, a second camera is controlled to capture a user image and recognize the user image to obtain user behavior data. If a first category of questions having knowledge points associated with the target question is determined to be recommended based on the user behavior data, the first category of questions is obtained and displayed; if a second category of questions having text content that meets similarity requirements with the target question is determined to be recommended based on the user behavior data, the second category of questions is obtained and displayed; and in response to a user selecting any of the displayed questions, the answer result corresponding to the selected question is displayed. Through the collaborative work of the two cameras, the first camera is used to provide the answer result in real time, and the second camera is linked to capture user behavior data during the display of the answer result. By analyzing and judging the user behavior data, questions related to the knowledge points of the target question or with similar text content are recommended, thereby dynamically providing users with personalized learning content recommendations that are adapted to the current user's understanding level, thereby improving learning efficiency, helping users better consolidate knowledge points, and optimizing the learning experience.

[0119] like Figure 2 As shown, in some embodiments, if it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes:

[0120] S201: Acquire first expression data, where the first expression data is expression data in the user expression data that meets a first determination condition and is used to represent an uncertain emotion.

[0121] For example, the first facial expression data may include expressions that can represent uncertainty, such as confusion, frowning, and hesitation. The user may have questions about or be unable to understand the displayed target question answering results. In such cases, the user's face will typically show confusion, bewilderment, hesitation, and other expressions, all of which express uncertainty.

[0122] A first determination condition can be pre-set to obtain first expression data from user expression data that satisfies the first determination condition and is used to represent an uncertain emotion. In some embodiments, the first determination condition can be defined based on the Facial Action Coding System (FACS). For a piece of user expression data, if the user expression data contains a specific combination of action units (AUs) used to represent an uncertain emotion, the user expression data is determined to meet the first determination condition.

[0123] Exemplarily, specific AU combinations for representing uncertain emotions include but are not limited to AU1 (inner eyebrow raised)+AU4 (eyebrow frowned)+AU7 (eyelid tightened), AU18 (pursed lips)-AU23 (tightened lips), and the like.

[0124] It should be noted that the embodiments of the present application do not specifically limit the definition method of the first judgment condition. In actual applications, the first judgment condition can also be defined based on emotion dimensions, intensity scores, key facial features, etc.

[0125] S202: If the first expression data meets the first setting requirement, determine to recommend the first category of topics.

[0126] For example, the first set requirement may be that the cumulative number of occurrences of the first expression data exceeds a set threshold and / or the duration of a single first expression data exceeds a set threshold.

[0127] S203: Based on the knowledge points involved in the target question, obtain from a preset question bank at least one question involving the same knowledge points as the target question and at least one question involving knowledge points that are a subset of the target question's knowledge points, and display the obtained questions.

[0128] The target question's text content can be analyzed to extract the knowledge points involved. For example, if the target question involves knowledge points {A, B, C}, a search is performed in the pre-set question bank based on the knowledge points involved in the target question to obtain at least one question that involves the same knowledge points as the target question, as well as at least one question that involves knowledge points that are a subset of the target question's knowledge points.

[0129] For example, a question involving the same knowledge points as the target question and multiple questions involving knowledge points that are subsets of the target question's knowledge points can be obtained.

[0130] It should be noted that the term "identical knowledge points" refers to questions with the same number and content of knowledge points, for example, questions that also cover knowledge points {A, B, C}. A subset of knowledge points refers to a question whose set of knowledge points is a proper subset of the target question's knowledge point set, meaning that all knowledge points in the former belong to the latter and the number of elements is strictly smaller than that in the latter. For example, questions that cover knowledge points {A}, {B}, {C}, {A, B}, {A, C}, or {B, C}.

[0131] In this embodiment, by obtaining first facial expression data (such as confusion, bewilderment, etc.) that satisfies the first judgment condition and is used to represent an uncertain emotion from the user's facial expression data, and when the first facial expression data meets the first set requirement, the following questions are recommended to the user: at least one question involving the same knowledge points as the target question, which is used to help the user reorganize and consolidate relevant knowledge points and deepen their understanding of the core content; at least one question involving knowledge points that are a subset of the target question's knowledge points, which is used to reduce the learning difficulty and help the user gradually establish an understanding framework for relevant knowledge points. In this way, when the user has questions about or difficulty understanding the displayed target question-answering results, based on the user's behavioral feedback, personalized learning content recommendations that are adapted to the user's current level of understanding can be provided to the user, thereby improving learning efficiency, helping the user better consolidate knowledge points, and optimizing the learning experience.

[0132] like Figure 3 As shown, in some embodiments, if it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes:

[0133] S301: Acquire second expression data, where the second expression data is expression data in the user expression data that meets a second determination condition and is used to represent a positive emotion.

[0134] For example, the second facial expression data may include expressions that represent positive emotions, such as happiness, excitement, and pleasure. The user may find that the displayed target question answer is easy to understand, meets expectations, or is satisfied. In this case, the user's face will typically show expressions such as happiness, excitement, and pleasure, which all express positive emotions.

[0135] A second determination condition can be pre-set to obtain second expression data from the user expression data that satisfies the second determination condition and is used to represent positive emotions. In some embodiments, the second determination condition can be defined based on FACS. For a piece of user expression data, if the user expression data contains a specific AU combination that represents positive emotions, the user expression data is determined to meet the second determination condition.

[0136] For example, specific AU combinations for representing positive emotions include, but are not limited to, AU6 (cheeks raised)+AU12 (mouth corners raised), AU12+AU25 (lips parted), and the like.

[0137] It should be noted that the embodiments of the present application do not specifically limit the definition method of the second determination condition. For example, the second determination condition can also be defined based on emotion dimensions, intensity scores, key facial features, etc.

[0138] S302: If the second expression data meets the second setting requirement, determine to recommend the first category of topics.

[0139] For example, the second set requirement may be that the cumulative number of occurrences of the second expression data exceeds a set threshold and / or the duration of a single second expression data exceeds a set threshold.

[0140] S303: Based on the knowledge points involved in the target question, obtain from a preset question bank at least one question involving the same knowledge points as the target question and at least one question involving more knowledge points or having more knowledge points than the target question, and display the obtained questions.

[0141] The target question's text content can be analyzed to extract the knowledge points involved. Based on the knowledge points involved in the target question, a search is performed in the pre-set question bank to obtain at least one question that covers the same knowledge points as the target question, as well as at least one question that covers more knowledge points or has a higher difficulty level than the target question.

[0142] For example, multiple questions involving the same knowledge points as the target question can be obtained, as well as multiple questions involving one more knowledge point or one more difficulty level than the target question, that is, questions that are slightly more difficult than the target question.

[0143] In this embodiment, by obtaining second expression data (such as happiness, joy, etc.) that satisfies the second judgment condition and is used to represent positive emotions from the user's expression data, and when the second expression data meets the second set requirement, the following questions are recommended to the user: at least one question involving the same knowledge points as the target question, which is used to help the user reorganize and consolidate relevant knowledge points and deepen their understanding of the core content; at least one question involving more knowledge points or more knowledge points of difficulty than the target question, which is used to provide more challenging content, stimulate the user's learning interest, and expand their knowledge application ability. In this way, when the user finds the displayed target question answering result easy to understand, meets expectations, or is satisfied, the user can be provided with personalized learning content recommendations that are adapted to the current user's understanding level based on the user's behavioral feedback, thereby improving learning efficiency, helping the user better consolidate knowledge points, and optimizing the learning experience.

[0144] like Figure 4 As shown, in some embodiments, if it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes:

[0145] S401: Acquire third expression data, where the third expression data is expression data in the user expression data that meets a third determination condition and is used to represent negative emotions.

[0146] For example, the third expression data may include expressions that represent negative emotions, such as anger and rage. Since the target Q&A result is the Q&A result associated with the question in the question bank with the highest text content similarity score to the target question, there may be cases where the displayed target Q&A result is incorrect. For example, the knowledge points, problem-solving methods, or answer logic involved in the target Q&A result are completely different from the actual requirements or context of the target question, or the target Q&A result contains obvious errors. In this case, the user's face will usually show expressions such as anger and rage, which all express negative emotions.

[0147] A third determination condition can be pre-set to obtain third expression data representing negative emotions from the user expression data that satisfies the third determination condition. In some embodiments, the third determination condition can be defined based on FACS. For a piece of user expression data, if the user expression data contains a specific AU combination representing negative emotions, the user expression data is determined to meet the third determination condition.

[0148] For example, specific AU combinations for representing negative emotions include but are not limited to AU4+AU7+AU23, AU9 (nose wrinkled)+AU10 (upper lip raised)+AU17 (lower lip raised), and the like.

[0149] It should be noted that the embodiments of the present application do not specifically limit the definition method of the third judgment condition. In actual applications, the third judgment condition can also be defined based on emotion dimensions, intensity scores, key facial features, etc.

[0150] S402: If the third expression data meets the third setting requirement, determine to recommend the second category of topics.

[0151] For example, the third set requirement may be that the cumulative number of occurrences of the third expression data exceeds a set threshold and / or the duration of a single third expression data exceeds a set threshold.

[0152] S403: A plurality of questions whose text contents meet similarity requirements with the target question are obtained from a preset question bank, and the obtained questions are displayed.

[0153] Specifically, the text content of the target question can be analyzed to extract key features, including but not limited to question type, knowledge points, question structure, and semantic features. Using similarity algorithms such as cosine similarity and the Jaccard coefficient, the extracted key features are compared with the features of each question in the preset question bank to calculate a similarity score. The questions in the question bank are then sorted based on the similarity scores, ultimately displaying the top N questions by similarity.

[0154] In this embodiment, by obtaining third expression data that meets the third judgment condition and is used to represent negative emotions from the user's expression data, and when the third expression data meets the third setting requirement, multiple questions whose text content meets the similarity requirement with the target question are obtained from the question bank for display. In this way, when the displayed target question-answering result is wrong, more accurate and relevant learning content recommendations can be provided to the user based on the user's behavioral feedback, thereby improving learning efficiency, helping users to better consolidate knowledge points, and optimizing the learning experience.

[0155] like Figure 5 As shown, in some embodiments, the user behavior data further includes user sitting posture data, and if it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes:

[0156] S501: Determine whether there is non-standard sitting posture data that meets a fourth determination condition in the user sitting posture data, and determine whether there is first expression data that meets a first determination condition and is used to represent an uncertain emotion in the user expression data;

[0157] For detailed descriptions of the non-standard sitting posture data satisfying the fourth determination condition and the first expression data satisfying the first determination condition, please refer to the above and will not be repeated here.

[0158] S502: If the non-standard sitting posture data exists and the first expression data does not exist, it is determined to recommend the first type of question; then, based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question, at least one question involving knowledge points that are a subset of the target question's knowledge points, and at least one question involving more knowledge points or more knowledge points than the target question is obtained from a preset question bank, and the obtained questions are displayed.

[0159] If there is non-standard sitting posture data (such as lying on the table, lying on the back, etc.), and there is no first expression data used to represent uncertainty emotions (such as confusion, bewilderment, etc.), it means that the user feels that the displayed target question-answering result is boring or does not understand it at all. In this case, the following questions can be recommended to the user: at least one question involving the same knowledge points as the target question, at least one question involving knowledge points that are a subset of the target question's knowledge points, and at least one question involving more knowledge points or more knowledge points of difficulty than the target question. In this way, learning content can be recommended in a targeted manner based on user behavioral feedback: if the user feels that the displayed target question-answering result is boring, the user's attention can be re-attracted through multi-level question types to avoid a decrease in learning efficiency due to boredom; if the user does not understand the displayed target question-answering result at all, the learning threshold can be lowered by simplifying the difficulty of the questions, and at the same time, with the help of multi-level question types, the user can gradually establish an understanding framework of relevant knowledge points.

[0160] S503: If the non-standard sitting posture data exists and the first expression data exists, it is determined to recommend the first type of question; then, based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving knowledge points that are a subset of the target question's knowledge points are obtained from a preset question bank, and the obtained questions are displayed.

[0161] If there is non-standard sitting posture data (such as lying on the table, lying on your back, etc.), and there is first expression data for representing uncertainty emotions (such as confusion, bewilderment, etc.), it means that the user has doubts about the displayed target question-answering results or has difficulty understanding them, and may feel tired or inattentive due to long-term learning. In this case, the following questions can be recommended to the user: at least one question that involves the same knowledge points as the target question, and at least one question that involves knowledge points that are a subset of the target question's knowledge points. Therefore, when the user has doubts about the target question-answering results or has difficulty understanding them, and feels tired or inattentive, learning content of moderate difficulty and relevant content can be recommended in a targeted manner based on the user's behavioral feedback, so as to help the user gradually establish an understanding framework of relevant knowledge points, while reducing the user's fatigue and re-stimulating their interest in learning and concentration.

[0162] The above content describes the method provided by this application. The following describes the device provided by this application:

[0163] See Figure 6 , is a schematic diagram of the structure of a linkage analysis device provided in an embodiment of the present application. The linkage analysis device is applied to an electronic device equipped with a first camera and a second camera, wherein the first camera is used to capture desktop images and the second camera is used to capture user images.

[0164] like Figure 6 As shown, the device may include:

[0165] The first question-answering unit 610 is configured to recognize the desktop image captured by the first camera, obtain a target question to be answered, and display a target answer result corresponding to the target question;

[0166] The behavior recognition unit 620 is configured to control the second camera to capture a user image during the display of the target question answering result, and to recognize the user image to obtain user behavior data, wherein the user behavior data includes at least user expression data;

[0167] The recommendation unit 630 is configured to obtain and display the first category of topics if it is determined based on the user behavior data that a first category of topics having knowledge points associated with the target topic is recommended; and obtain and display the second category of topics if it is determined based on the user behavior data that a second category of topics having text content that meets a similarity requirement with the target topic is recommended;

[0168] The second question-answering unit 640 is configured to, in response to a user selecting any displayed question, display a question-answering result corresponding to the selected question.

[0169] Optionally, the recommendation unit 630 is specifically configured to:

[0170] Acquire first expression data, where the first expression data is expression data in the user expression data that meets a first determination condition and is used to represent an uncertain emotion;

[0171] If the first expression data meets the first setting requirement, determining to recommend the first category of topics;

[0172] Based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving knowledge points that are a subset of the target question's knowledge points are obtained from a preset question bank, and the obtained questions are displayed.

[0173] Optionally, the recommendation unit 630 is specifically configured to:

[0174] Acquire second expression data, where the second expression data is expression data in the user expression data that meets a second determination condition and is used to represent a positive emotion;

[0175] If the second expression data meets the second setting requirement, determining to recommend the first category of topics;

[0176] Based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving more knowledge points or having more knowledge points than the target question are obtained from the preset question bank, and the obtained questions are displayed.

[0177] Optionally, the recommendation unit 630 is specifically configured to:

[0178] Acquiring third expression data, where the third expression data is expression data in the user expression data that meets a third determination condition and is used to represent a negative emotion;

[0179] If the third expression data meets the third setting requirement, determining to recommend the second category of questions;

[0180] Then, a plurality of questions whose text contents meet the similarity requirement with the target question are obtained from a preset question bank, and the obtained questions are displayed.

[0181] Optionally, the user behavior data further includes user sitting posture data. The recommendation unit 630 is specifically configured to:

[0182] Determining whether there is non-standard sitting posture data that meets a fourth determination condition in the user sitting posture data, and determining whether there is first expression data that meets a first determination condition and is used to represent an uncertain emotion in the user expression data;

[0183] If the non-standard sitting posture data exists and the first expression data does not exist, it is determined that the first type of question is recommended; then, based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question, at least one question involving knowledge points that are a subset of the knowledge points of the target question, and at least one question involving more knowledge points or having a higher difficulty level than the target question is obtained from a preset question bank, and the obtained questions are displayed;

[0184] If the non-standard sitting posture data exists and the first expression data exists, it is determined to recommend the first type of questions; then, based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving knowledge points that are a subset of the target question's knowledge points are obtained from a preset question bank, and the obtained questions are displayed.

[0185] Optionally, the first question-answering unit 610 is specifically configured to:

[0186] Recognizing the desktop image captured by the first camera to obtain a user fingertip block and a book page block in the desktop image;

[0187] Performing question frame detection on the book page block to obtain a question frame block for each question on the book page;

[0188] According to the degree of overlap between the user's fingertip block and the question frame block of each question, a target question frame block is determined, and text recognition is performed on the target question frame block to obtain the target question to be answered.

[0189] Optionally, the first question-answering unit 610 is specifically configured to:

[0190] According to the text content of the target question, searching for the question with the highest similarity in the preset question bank;

[0191] The question-and-answer results bound to the question with the highest similarity are displayed.

[0192] Optionally, the second question-answering unit 640 is specifically configured to:

[0193] In response to a user selecting any of the displayed questions, if the selection operation is used to instruct display of the question-answering result, the question-answering result corresponding to the selected question is directly displayed;

[0194] If the selection operation is used to instruct answering a question, the text content of the selected question is displayed, and after it is determined that the user has completed answering the question, the answer result corresponding to the selected question is displayed.

[0195] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0196] The embodiment of the present application also provides a hardware structure. Figure 7 , Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0197] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0198] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0199] It should be noted that, in this document, relational terms such as target and objective are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0200] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A linkage analysis method, characterized in that: Applied to an electronic device equipped with a first camera and a second camera, wherein the first camera is used to capture a desktop image and the second camera is used to capture a user image, the method includes: Recognize the desktop image captured by the first camera to obtain a target question to be answered, and display a target answer result corresponding to the target question; During the display of the target question answering result, controlling the second camera to capture a user image and recognize the user image to obtain user behavior data, wherein the user behavior data at least includes user expression data; If it is determined based on the user behavior data that a first type of topic with knowledge points associated with the target topic is recommended, the first type of topic is obtained and displayed; if it is determined based on the user behavior data that a second type of topic with text content that meets similarity requirements with the target topic is recommended, the second type of topic is obtained and displayed; In response to a selection operation performed by the user on any displayed question, the question-answering result corresponding to the selected question is displayed.

2. The method according to claim 1, characterized in that If it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes: Acquire first expression data, where the first expression data is expression data in the user expression data that meets a first determination condition and is used to represent an uncertain emotion; If the first expression data meets the first setting requirement, determining to recommend the first category of topics; Based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving knowledge points that are a subset of the target question's knowledge points are obtained from a preset question bank, and the obtained questions are displayed.

3. The method according to claim 1, characterized in that If it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes: Acquire second expression data, where the second expression data is expression data in the user expression data that meets a second determination condition and is used to represent a positive emotion; If the second expression data meets the second setting requirement, determining to recommend the first category of topics; Based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving more knowledge points or having more knowledge points than the target question are obtained from the preset question bank, and the obtained questions are displayed.

4. The method according to claim 1, wherein If it is determined based on the user behavior data that a second type of topic is recommended whose text content meets the similarity requirement with the target topic, obtaining the second type of topic and displaying it includes: Acquiring third expression data, where the third expression data is expression data in the user expression data that meets a third determination condition and is used to represent a negative emotion; If the third expression data meets the third setting requirement, determining to recommend the second category of questions; Then, a plurality of questions whose text contents meet the similarity requirement with the target question are obtained from a preset question bank, and the obtained questions are displayed.

5. The method according to claim 1, wherein The user behavior data also includes user sitting posture data. If it is determined based on the user behavior data that a first category of questions having knowledge points associated with the target question is recommended, obtaining the first category of questions and displaying them includes: Determining whether there is non-standard sitting posture data that meets a fourth determination condition in the user sitting posture data, and determining whether there is first expression data that meets a first determination condition and is used to represent an uncertain emotion in the user expression data; If the non-standard sitting posture data exists and the first expression data does not exist, it is determined that the first type of question is recommended; then, based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question, at least one question involving knowledge points that are a subset of the knowledge points of the target question, and at least one question involving more knowledge points or having a higher difficulty level than the target question is obtained from a preset question bank, and the obtained questions are displayed; If the non-standard sitting posture data exists and the first expression data exists, it is determined to recommend the first type of questions; then, based on the knowledge points involved in the target question, at least one question involving the same knowledge points as the target question and at least one question involving knowledge points that are a subset of the target question's knowledge points are obtained from a preset question bank, and the obtained questions are displayed.

6. The method according to claim 1, characterized in that The step of recognizing the desktop image captured by the first camera to obtain a target question to be answered includes: Recognizing the desktop image captured by the first camera to obtain a user fingertip block and a book page block in the desktop image; Performing question frame detection on the book page block to obtain a question frame block for each question on the book page; According to the degree of overlap between the user's fingertip block and the question frame block of each question, a target question frame block is determined, and text recognition is performed on the target question frame block to obtain the target question to be answered.

7. The method according to claim 1, characterized in that The display of the target question answering result corresponding to the target question includes: According to the text content of the target question, searching for the question with the highest similarity in the preset question bank; The question-and-answer results bound to the question with the highest similarity are displayed.

8. The method according to claim 1, characterized in that In response to the user selecting any displayed question, displaying the answer result corresponding to the selected question includes: In response to a user selecting any of the displayed questions, if the selection operation is used to instruct display of the question-answering result, the question-answering result corresponding to the selected question is directly displayed; If the selection operation is used to instruct answering a question, the text content of the selected question is displayed, and after it is determined that the user has completed answering the question, the answer result corresponding to the selected question is displayed.

9. A linkage analysis device, characterized in that: Applicable to an electronic device equipped with a first camera and a second camera, wherein the first camera is used to capture a desktop image and the second camera is used to capture a user image, the device comprises: A first question-answering unit, configured to recognize the desktop image captured by the first camera, obtain a target question to be answered, and display a target answer result corresponding to the target question; a behavior recognition unit, configured to control the second camera to capture a user image during the display of the target question answering result, and recognize the user image to obtain user behavior data, wherein the user behavior data at least includes user expression data; a recommendation unit configured to obtain and display a first category of questions having knowledge points associated with the target question if the user behavior data determines that the first category of questions are recommended; and obtain and display a second category of questions if the user behavior data determines that the second category of questions are recommended if the text content of the second category of questions satisfies a similarity requirement with the target question; The second question-answering unit is configured to, in response to a selection operation performed by the user on any displayed question, display the question-answering result corresponding to the selected question.

10. An electronic device, characterized in that: The system comprises a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor is configured to execute the machine-executable instructions to implement the method according to any one of claims 1 to 8.

11. A machine-readable storage medium, characterized in that The machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images