Learning state detection method and device, storage medium and electronic equipment

Abstract

The invention provides a learning state detection method and device, a storage medium and electronic equipment, and the method is applied to the technical field of computers, and comprises the steps: employing a camera to collect a behavior image of a target object, and based on the target state information of the target object in the behavior image, judging whether the target object is in a learning state, and if the target object is in the learning state, acquiring the duration of the learning state of the target object, and if the duration reaches a preset learning threshold, outputting prompt voice. According to the method, whether the target object is in the learning state or not can be judged according to the state of the target object collected by the camera, prompting is carried out when the duration time of the learning state is too long, automatic monitoring of the learning state is achieved, and the situations of fatigue and learning efficiency reduction caused by long-time learning of the target object are avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and more particularly, to a learning state detection method and device, a storage medium and an electronic device. BACKGROUND

[0002] Using a camera to monitor an object in a scene can be used to protect the personal safety of the object in the scene. For example, parents or schools can install a camera in a home scene or a classroom scene to achieve safety protection for students. In the prior art, in order to meet the needs of parents and teachers for remote monitoring and guidance of students, the learning state of a student can be determined by audio and video collected by a camera. However, parents and teachers cannot view the camera image for a long time, and a long learning time can cause mental fatigue and reduce learning efficiency. Therefore, a method for automatically detecting learning time is needed. SUMMARY

[0003] Embodiments of the present application provide a learning state detection method and device, a storage medium and an electronic device. The method can determine whether a target object is in a learning state based on the state of the target object collected by a camera, and provide a prompt when the learning state lasts for too long. The automatic monitoring of the learning state can avoid the situation of fatigue and reduced learning efficiency caused by long learning time of the target object.

[0004] In a first aspect, embodiments of the present application provide a learning state detection method, which includes:

[0005] capturing a behavior image of a target object by using a camera;

[0006] determining whether the target object is in a learning state based on target state information of the target object in the behavior image;

[0007] if the target object is in a learning state, obtaining a continuous time length of the learning state of the target object;

[0008] if the continuous time length reaches a preset learning threshold, outputting a prompt voice.

[0009] In a second aspect, embodiments of the present application provide a learning state detection device, which includes:

[0010] a behavior image capturing unit configured to capture a behavior image of a target object by using a camera;

[0011] a learning state determining unit configured to determine whether the target object is in a learning state based on target state information of the target object in the behavior image;

[0012] if the target object is in a learning state, obtaining a duration of the learning state of the target object;

[0013] a voice prompt unit, configured to output a prompt voice if the duration reaches a preset learning threshold.

[0014] In a third aspect, an embodiment of the present application provides a computer storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and performing the method steps described above.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, which can include a processor and a memory, wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and performing the method steps described above.

[0016] In one or more embodiments of the present application, a camera is used to collect a behavior image of a target object, based on target state information of the target object in the behavior image, it is determined whether the target object is in a learning state, if the target object is in the learning state, a duration of the learning state of the target object is obtained, and if the duration reaches a preset learning threshold, a prompt voice is output. Through the camera, it is determined whether the target object is in the learning state, and a prompt is given when the duration of the learning state is too long, so as to realize automatic monitoring of the learning state, and avoid the situation that the target object is tired and the learning efficiency is reduced due to long-time learning. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a scene schematic diagram provided by an embodiment of the present application for monitoring a target object;

[0019] Figure 2 is a flow schematic diagram of a learning state detection method provided by an embodiment of the present application;

[0020] Figure 3 is a flow schematic diagram of a learning state judgment provided by an embodiment of the present application;

[0021] Figure 4 is a flow schematic diagram of a state judgment based on a model provided by an embodiment of the present application;

[0022] Figure 5is a flowchart of a model training process provided by an embodiment of the present application.

[0023] Figure 6 is a flowchart of a prompt voice output process provided by an embodiment of the present application.

[0024] Figure 7 is a structural diagram of a learning state detection device provided by an embodiment of the present application.

[0025] Figure 8 is a structural diagram of a learning state detection device provided by an embodiment of the present application.

[0026] Figure 9 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0028] In today's daily life, work scene, camera monitoring has become an indispensable security tool, which can be applied to family, school or work scene, and the events occurring in the scene are monitored by the camera to ensure the safety and order in the scene. However, due to the diverse needs of people, the monitoring scene and role of the camera are also expanding. For example, in the learning scene of family, school, etc., parents and school staff can use the camera to remotely monitor the learning situation of students. They can remotely check the learning situation of students, understand whether students are focused on learning, whether they encounter difficulties and need help, or whether they arrange their activities reasonably during the rest time. Remote monitoring can enable parents or school staff to master the learning dynamics of students without disturbing them. However, if the learning state is maintained for a long time, it will cause students to be physically and mentally exhausted, and their concentration will be reduced, which will not only reduce the learning efficiency, but also affect the physical and mental health of students. Therefore, students need to be reminded of the length of time spent on learning. The embodiments of the present application provide a learning state detection device, which can collect images of a target object through a camera and identify whether the target object is in a learning state, so as to prompt the target object when it maintains a learning state for a long time. The learning state detection device can be applied to a family scene or a school teaching scene, and the target object can be a child or a student. The learning state detection method provided by the embodiments of the present application can be realized by relying on a computer program and can run on a learning state detection device based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool application. The learning state detection device can call and control the camera. In addition to collecting the terminal interface of the target object, the camera can also monitor the target object to ensure the safety of the target object in the family scene or the school teaching scene.

[0029] Please see Figure 1 A scene diagram for monitoring a target object is provided for the embodiments of the present application. The camera can be installed by guardians, school staff, etc. The camera visual angle range can cover the desk and seat range of the target object, so as to facilitate the collection of the current behavior of the target object. For example, the visual angle range of the camera can include various parts of the body of the target object, such as shoulders, back, waist, legs and hands, etc. It can also include the articles used by the target object, such as books, pens and terminal devices, etc. The learning state detection device can determine the current behavior of the target object through the collected images, such as whether the target object is sitting in front of the desk, whether it is reading a book or writing homework, so as to determine whether the target object is in a learning state.

[0030] The learning state detection method provided by the present application will be described in detail in combination with specific embodiments.

[0031] Please see Figure 2A flowchart of a learning state detection method is provided for the embodiments of the present application. As shown in Figure 2 The method of the embodiments of the present application can include the following steps S101-S104.

[0032] S101, acquiring a behavior image of the target object by using a camera.

[0033] Specifically, when the target object enters the visual range of the camera, the learning state detection device can control the camera to acquire the behavior image of the target object. The behavior image is used to show the current behavior of the target object, for example, it can include various parts of the body of the target object, such as shoulders, back, waist, legs, and hands, etc., and it can also include the articles used by the target object. For the purpose of detecting the current behavior of the target object, the learning state detection device can control the camera to continuously acquire the behavior image of the target object, for example, the behavior image of the target object can be acquired according to a preset acquisition frequency. The preset acquisition frequency can be related to the hardware condition of the camera, or it can be related to the computing power of the learning state detection device. It can be the initial setting of the learning state detection device, or it can be set by the guardian of the target object or relevant staff.

[0034] S102, judging whether the target object is in a learning state based on target state information of the target object in the behavior image.

[0035] Specifically, the learning state detection device can obtain the target state information of the target object from the behavior image. The target state information is used to indicate the state of the target object in the behavior image, for example, it can reflect the back posture, head posture, static action, and interactive articles of the target object, etc. The learning state detection device can determine whether the target object is in a learning state according to the target state information. For example, if it is determined according to the target state information that the target object is sitting in front of a desk and holding a pen to write, it can be determined that the target object is in a learning state. If it is determined according to the target state information that the target object is sitting in front of a desk but holding a mobile phone to play an electronic game, it can be determined that the target object is not in a learning state.

[0036] S103, if the target object is in a learning state, obtaining the duration of the learning state of the target object.

[0037] Specifically, if it is determined that the target object is in a learning state, the duration of the learning state of the target object can be obtained. Since the learning state detection device can continuously acquire the behavior image of the target object and determine whether it is in a learning state, the duration can be the duration of the learning state of the target object continuously detected by the learning state detection device, which is used to reflect the continuous learning time of the target object.

[0038] S104, output a prompt voice if the duration reaches a preset learning threshold.

[0039] Specifically, long-time learning state will cause the student's body, mental fatigue, lack of concentration, but will reduce the learning efficiency, but also affect the student's physical and mental health, so you can detect whether the duration reaches the preset learning threshold, the preset learning threshold is used to determine whether the learning duration of the target object is too long, which can be the initial setting of the learning state detection device, or can be set by the guardian or relevant staff, if the detection duration reaches the preset learning threshold, the learning state detection device can output a prompt voice, for prompting the target object to learn for too long, should take a proper rest, for example, the prompt voice can be "learned for so long, take a break, move your body, relax your eyes."

[0040] In the embodiments of the present application, the camera is used to collect the behavior image of the target object, and based on the target state information of the target object in the behavior image, it is determined whether the target object is in a learning state, if the target object is in a learning state, the duration of the target object learning state is obtained, if the duration reaches a preset learning threshold, a prompt voice is output. By the camera to collect the target object state to determine whether it is in a learning state, and prompt when the learning state duration is too long, to realize the automatic monitoring of the learning state, to avoid the situation of fatigue and learning efficiency caused by long-time learning of the target object.

[0041] Please refer to Figure 3 , the present application provides a flowchart of learning state judgment. As Figure 3 shown, in one or more embodiments of the present application, the step S102 can include the following steps S201-S202.

[0042] S201, obtain the target state information of the target object in the behavior image.

[0043] Specifically, the learning state detection device can obtain the target state information of the target object from the behavior image, and the target state information is used to represent the state of the target object in the behavior image.

[0044] S202, if the target state information meets the learning state condition, it is determined that the target object is in a learning state.

[0045] Specifically, the learning state detection apparatus can pre-set a learning state condition, the learning state condition being used to determine whether the target object is in the learning state, and the learning state condition can limit the state information of the object in the learning state. If the target state information meets the learning state condition, it can be determined that the target object is in the learning state. For example, the learning state condition can set the body posture and static action of the target object. If the body posture reflected by the target state information indicates that the target object is sitting in front of a desk, and the static action reflected by the target state information indicates that the target object is writing, it can be determined that the target state information meets the learning state condition. The learning state condition can be set by the learning state detection apparatus based on the learning state related information on the Internet, or can be set by the guardian or relevant staff according to the target object and the monitored scene.

[0046] In the embodiments of the present application, the target state information of the target object in the behavior image is acquired, and if the target state information meets the learning state condition, it is determined that the target object is in the learning state. The target state information is judged by the learning state condition to determine that the target object is in the learning state, thereby improving the accuracy of learning state detection.

[0047] In one or more embodiments of the present application, the target state information includes a target object posture and a target object behavior, the target object posture being used to represent the body posture of the target object, and the target object behavior representing the behavior action of the target object in the behavior image.

[0048] Optionally, the target object posture can include a head posture, a shoulder posture, a back posture, a leg posture, and / or a whole body posture, etc. The head posture can include head position, head-body relative angle, head-shoulder-neck relative angle, etc. The shoulder posture can include shoulder position, shoulder level, shoulder-back-waist relative angle, etc. The back posture can include back contour shape, spine physiological curvature, back-seat back contact area, etc. The leg posture can include waist position, whether the waist contacts the seat back, etc. The leg posture can include leg position, leg posture, whether the leg is placed on the ground, etc. The whole body posture can include body center of gravity position, body symmetry, body-desk relative position, etc. Acquiring the posture of each part of the body of the target object can help the learning state detection apparatus to comprehensively and accurately determine the body posture of the target object and judge whether the body posture of the target object meets the learning state.

[0049] Optionally, due to the limited camera view range, the blocking of tables and chairs, etc., the collected behavior image may not contain the head, shoulders, back and legs of the target object at the same time, so the learning state detection device can only obtain the posture information corresponding to the existing parts. The learning state detection device can perform body part detection processing on the behavior image to determine which parts are present in the behavior image and the positions of each part, and then extract the corresponding posture information based on the positions of each part.

[0050] Optionally, the target object behavior can include the target object's line of sight focus, hand movement and / or interactive object, etc., the line of sight focus can be used to represent what the target object is looking at, for example, the target object is looking at a book or a terminal device, the hand movement and the interactive object can be used to represent the action the target object is performing, for example, if the hand movement is writing and the interactive object is a book, the target object may be taking notes or doing homework, if the hand movement is operating the screen and the interactive object is a terminal device, the target object may be playing an electronic game.

[0051] Optionally, the same interactive object and similar hand movement can also represent different actions of the target object, for example, when the interactive object is a terminal device, the target object may be playing an electronic game or learning online courses or viewing learning materials, so the learning state detection device can further obtain the interactive object state, the interactive object state can represent the state displayed by the interactive object in the behavior image, for example, when the interactive object is a book, the interactive object state can be the content currently displayed by the book, when the interactive object is a terminal device, the interactive object state can be the content displayed by the terminal interface.

[0052] The step S202 can include the following steps:

[0053] If the target object posture matches the learning posture feature and the target object behavior matches the learning behavior feature, it is determined that the target object is in a learning state.

[0054] Specifically, since the target state information can include the target object posture and the target object behavior, the preset learning state condition can be used to determine whether the target object meets the learning state from the aspects of the posture and the behavior, that is, the learning state condition can include learning posture features and learning behavior features. The learning posture features are used to determine whether the target object posture meets the posture in the learning state, for example, the learning posture features can include various postures of the object in the learning state, such as a correct sitting posture in front of a desk, thereby helping the learning state detection apparatus to determine whether the target object is in the learning state from the aspect of the posture of the target object. The learning behavior features are used to determine whether the target object behavior meets the behavior in the learning state, for example, the learning behavior features can include various behavior actions of the object in the learning state, such as writing a book with educational content or viewing a terminal device with online course content.

[0055] Further, if only the target object posture matches the learning posture features in the target state information, the target object can not be in the learning state, for example, although the target object is sitting in front of the desk in a correct posture, the target object does not perform the behavior in the learning state but plays an electronic game. Similarly, if only the target object behavior matches the learning behavior features in the target state information, the target object can not be in the learning state, for example, although the target object is viewing the book, the target object is not sitting in front of the desk but walking in the room, and the target object can only be arranging the book. Therefore, to determine that the target object is in the learning state, the target object posture and the target object behavior need to meet the learning state condition, that is, if the target object posture matches the learning posture features and the target object behavior matches the learning behavior features, the learning state detection apparatus can determine that the target object is in the learning state.

[0056] In the embodiments of the present application, the target state information includes the target object posture and the target object behavior, and if the target object posture matches the learning posture features and the target object behavior matches the learning behavior features, it is determined that the target object is in the learning state. By combining the posture and the behavior to determine the learning state of the target object, the situation of misjudgment caused by the one-dimension data is reduced, and the accuracy of the learning state detection of the target object is improved.

[0057] To further improve the accuracy of the sitting posture detection and improve the calculation efficiency, the learning state detection apparatus can train a convolutional neural network (CNN) to participate in the extraction of the target state information and the determination of the learning state condition, and since the CNN is a deep learning model and can be applied to the fields of image recognition and natural language processing, the CNN can help to learn the features of the target state information and perform analysis.

[0058] In one or more embodiments of the present application, the step S201 can include the following steps:

[0059] The behavior image is input into a state detection model, and a target object posture and a target object behavior of the target object are obtained based on the state detection model.

[0060] Specifically, the learning state detection apparatus can input the collected behavior image into the state detection model. The state detection model can have the ability to extract features of the behavior image to obtain target state information, and compare the target state information with the learning state condition to obtain the current state of the target object. The learning state detection apparatus can use the state detection model to obtain the target object posture and the target object behavior of the target object.

[0061] Please refer to Figure 4 A flowchart for state judgment based on a model is provided for the embodiments of the present application. As shown in Figure 4 The step S202 in one or more embodiments of the present application can include the following steps S301-S303.

[0062] S301, the target object posture and the learning posture feature are matched and processed by using the state detection model, and a posture detection result output by the state detection model is obtained.

[0063] Specifically, the state detection model can match and process the target object posture and the learning posture feature, and output a posture detection result. For example, if the state detection model determines that the target object posture and the learning posture feature are matched, the behavior image can be labeled with a learning posture label. If the state detection model determines that the target object posture and the learning posture feature are not matched, the behavior image can be labeled with a non-learning posture label. The posture detection result is generated based on the labeled learning posture label or non-learning posture label. The posture detection result containing the learning posture label indicates that the target object is in a learning posture, and the posture detection result containing the non-learning posture label indicates that the target object is in a non-learning posture.

[0064] S302, the target object behavior and the learning behavior feature are matched and processed by using the state detection model, and a behavior detection result output by the state detection model is obtained.

[0065] Specifically, the state detection model can perform matching processing on the target object behavior and the learning behavior feature, and output a behavior detection result. For example, if the state detection model determines that the target object behavior and the learning behavior feature match, the behavior image can be labeled with a learning behavior label. If the state detection model determines that the target object behavior and the learning behavior feature do not match, the behavior image can be labeled with a non-learning behavior label. The learning behavior label or the non-learning behavior label is generated to generate a behavior detection result. The behavior detection result containing the learning behavior label indicates that the target object is in a learning behavior, and the behavior detection result containing the non-learning behavior label indicates that the target object is in a non-learning behavior.

[0066] S303, determining that the target object is in a learning state based on the posture detection result and the behavior detection result.

[0067] Specifically, after the learning state detection apparatus obtains the posture detection result and the behavior detection result, if the posture detection result indicates that the target object is in a learning posture and the behavior detection result indicates that the target object is in a learning behavior, it is determined that the target object is in a learning state. If the posture detection result indicates that the target object is in a learning posture and the behavior detection result indicates that the target object is in a non-learning behavior, the posture detection result indicates that the target object is in a non-learning posture and the behavior detection result indicates that the target object is in a learning behavior, or the posture detection result indicates that the target object is in a non-learning posture and the behavior detection result indicates that the target object is in a non-learning behavior, it is determined that the target object is in a non-learning state.

[0068] In the embodiments of the present application, the behavior image is input into the state detection model, the target object posture and the target object behavior of the target object are obtained based on the state detection model, the state detection model is used to perform matching processing on the target object posture and the learning posture feature, the posture detection result output by the state detection model is obtained, the state detection model is used to perform matching processing on the target object behavior and the learning behavior feature, the behavior detection result output by the state detection model is obtained, and the target object is determined to be in a learning state based on the posture detection result and the behavior detection result. Through the model participating in the process of target state information extraction and learning state condition judgment, the efficiency of learning state detection is improved, and the accuracy of learning state detection is also improved.

[0069] See Figure 5 A flowchart of model training is provided for the embodiments of the present application. As shown in Figure 5 The learning state detection method can further include steps S401-S403.

[0070] S401, creating an initial state recognition model, obtaining a sample behavior image, a sample behavior label corresponding to the sample behavior image, and a sample posture label.

[0071] Specifically, an initial state recognition model can be created, which has a preliminary ability to generate a posture detection result and a behavior detection result according to a behavior image. A sample behavior image, a sample behavior label corresponding to the sample behavior image, and a sample posture label can also be obtained. The sample behavior image can be a real behavior image of a sample object photographed by a relevant staff member. The sample behavior label and the sample posture label can be labels manually labeled by the relevant staff member for the sample behavior image.

[0072] Optionally, the sample behavior image can include a behavior image containing the head, shoulders, back, legs, and whole body of the sample object, or a behavior image containing only part of the body of the sample object, such as a behavior image containing only the head and back of the sample object, or a behavior image containing only the head and legs of the sample object, and the like. Thus, after the model is trained, the state recognition model obtained can accurately determine whether the target object is in a learning posture according to the unobstructed body part when an obstructed behavior image of the body part of the target object is obtained, thereby improving the robustness of the state recognition model.

[0073] S402, inputting the sample behavior image into the initial state recognition model to obtain training behavior labels and training posture labels output by the initial state recognition model.

[0074] Specifically, the sample behavior image and the sample pressure sensing data can be input into the initial state recognition model. The initial state recognition model can obtain sample state information in the sample behavior image, which can include a sample object posture and a sample object behavior of the sample object, thereby determining training behavior labels and training posture labels corresponding to the sample behavior image. The training behavior labels and the training posture labels are respectively behavior labels and posture labels predicted by the initial state recognition model based on the current learning state detection capability, wherein the behavior labels include learning behavior labels and non-learning behavior labels, and the posture labels include learning posture labels and non-learning posture labels.

[0075] Optionally, the process of the initial state recognition model for target state information extraction and learning state condition judgment on the sample behavior image can refer to the above embodiments. For example, the initial state recognition model can obtain a sample object posture and a sample object behavior from the sample behavior image, and then perform matching processing on the sample object posture and a learning posture feature to obtain training posture labels, and perform matching processing on the sample object behavior and a learning behavior feature to obtain training behavior labels.

[0076] S403, performing parameter adjustment processing on the initial state recognition model based on the training behavior labels and the sample behavior labels, and the training posture labels and the sample posture labels until the model training is completed, to obtain a state recognition model.

[0077] Specifically, the loss function of the initial state recognition model can be calculated based on the training behavior label and the sample behavior label, the training posture label and the sample posture label, the initial state recognition model is adjusted in parameter based on the loss function in the back propagation training process, until the initial state recognition model completes the model training, and the state recognition model is obtained.

[0078] Optionally, the initial posture recognition type can be adjusted in parameter based on the training behavior label and the sample behavior label, the training posture label and the sample posture label, until the training termination condition is reached, and the state recognition model is obtained. The training termination condition can include that the loss function decreases to a preset loss value and remains for a preset time length, the preset loss value is used to determine whether the prediction accuracy of the initial state recognition model has reached the standard, and the preset time length is used to determine whether the stability of the initial state recognition model has reached the standard. The preset loss value and the preset time length can be the initial settings of the learning state detection device, or can be set by relevant staff. For example, the preset loss value can be 0.05, and the preset time length can be 100 model training rounds. The training termination condition can also include that the model training rounds of the initial state recognition model reach a preset round, for example, the preset round can be 100 times. Using the training round as the training termination condition is very intuitive and easy to understand, and does not require complex calculation or judgment conditions. Only the round needs to be set at the beginning of training, so that the management of the training process is more simple and concise.

[0079] In the embodiments of the present application, an initial state recognition model is created, a sample behavior image, a sample behavior label corresponding to the sample behavior image and a sample posture label are obtained, the sample behavior image is input into the initial state recognition model, training behavior labels and training posture labels output by the initial state recognition model are obtained, and the initial state recognition model is adjusted in parameter based on the training behavior labels and the sample behavior labels, and the training posture labels and the sample posture labels, until the model training is completed, and the state recognition model is obtained. Through model training based on the sample behavior image and the corresponding artificial annotation label, the accuracy and reliability of the state recognition model are improved, so that the state recognition model can adapt to different application environments and improve the generalization ability of the model.

[0080] The target object needs to balance the time allocation of the learning state and the game state, the rest state and other non-game states, and continuously maintaining the learning state will also lead to the reduction of the learning efficiency of the target object. The length of time for which the target object maintains the learning state needs to be obtained to avoid physical and mental fatigue caused by long-time learning.

[0081] In one or more embodiments of the present application, the step S103 can include the following steps:

[0082] If the target object is in the learning state, a single duration of the learning state of the target object is acquired, and a cumulative duration of the learning state of the target object is acquired.

[0083] Specifically, if it is detected that the target object is in the learning state, a single learning duration of the target object in the learning state can be acquired, and the single duration is the duration of the target object entering the learning state. A cumulative duration of the target object in the learning state can also be acquired, and the cumulative duration can be the cumulative time of the target object entering the learning state within a preset time. The preset time is an initial setting of the learning state detection device, and can also be set by a guardian or a relevant staff. For example, the preset time can be 24 hours.

[0084] Since the duration can include the single duration and the cumulative duration, the learning state detection device can output different prompt voices for the two durations.

[0085] See Figure 6 A flowchart of a prompt voice output is provided for the embodiments of the present application. As shown in Figure 6 The step S104 described in one or more embodiments of the present application can include steps S501-S502.

[0086] S501, if the single duration is greater than a first preset time, a first rest prompt voice is output.

[0087] Specifically, if the single duration is greater than the first preset time, the first rest prompt voice can be output. The first preset time can be used to limit the duration of continuous learning of the target object, so as to avoid eye fatigue caused by long-time use of eyes by the target object. The first preset time can be an initial setting of the learning state detection device, and can also be set by a user or a relevant staff. For example, the first preset time can be 45 minutes. The first rest prompt voice can prompt the target object that the continuous learning time is too long, and the target object needs to pay attention to the rest. For example, the first rest prompt voice can be "has been continuously learning for 45 minutes, please pay attention to rest your eyes".

[0088] S502, if the cumulative duration is greater than a second preset time, a second rest prompt voice is output.

[0089] Specifically, if the cumulative duration is greater than the second preset duration, a second rest reminder voice can be output. The second preset duration can be used to display the total learning time of the target object within the preset time, to avoid physical and mental fatigue caused by too long learning time in a short period of time, and to avoid low knowledge digestion rate and low learning efficiency caused by knowledge cramming. It can be the initial setting of the terminal interface detection device, and can also be set by the guardian or relevant staff. The second rest reminder voice can indicate that the target object does not need to continue learning within the preset time and needs to rest or do outdoor activities. For example, the second rest reminder voice can be "Today's study time has reached 3 hours, please take a rest."

[0090] Optionally, the learning status detection device may include at least one preset tone. The preset tone may be set by the learning status detection device or relevant staff based on an electronic tone on the Internet, or may be input and set by the guardian or teacher of the target object. The target object or the guardian or teacher of the target object may select and set the target preset tone from the preset tones. The learning status detection device may output a prompt voice based on the target preset tone. The prompt voice may include a first rest voice and a second rest voice. The learning status detection device may generate a prompt voice based on the target preset tone based on text-to-speech (TTS) technology. Since the target object may be a child, the target preset tone may be used to simulate the voice of their guardian or teacher. While reminding the target object to pay attention to the length of study, it may also soothe the target object's emotions and meet the personalized needs of different objects.

[0091] In an embodiment of the present application, if the target subject is in a learning state, the single duration of the target subject's learning state and the cumulative duration of the target subject's learning state are obtained. If the single duration is greater than a first preset duration, a first rest prompt voice is output; if the cumulative duration is greater than a second preset duration, a second rest prompt voice is output. By using single duration reminders and cumulative duration reminders, while preventing the target subject from studying continuously for a long time, the total study time of the target subject within a period of time can also be controlled. While balancing study and rest time, it also avoids damage to the target subject's eyes and low learning efficiency.

[0092] The following will be combined with the Figure 7 -Attached Figure 8 , the learning state detection device provided in the embodiment of the present application is introduced in detail. Figure 7 -Attached Figure 8 The learning state detection device in the present application is used to execute Figures 1-6 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figures 1-6The illustrated embodiment.

[0093] Referring to Figure 7 , which shows a structure diagram of a learning state detection device provided by an example embodiment of the present application. The learning state detection device can be realized by software, hardware, or a combination of the two to become all or part of the device. The device 1 includes a behavior image acquisition unit 11, a learning state judgment unit 12, a duration acquisition unit 13, and a voice prompt unit 14.

[0094] The behavior image acquisition unit 11 is configured to acquire a behavior image of a target object using a camera;

[0095] The learning state judgment unit 12 is configured to judge whether the target object is in a learning state based on target state information of the target object in the behavior image;

[0096] The duration acquisition unit 13 is configured to acquire a duration of the learning state of the target object if the target object is in the learning state;

[0097] The voice prompt unit 14 is configured to output a prompt voice if the duration reaches a preset learning threshold.

[0098] In the embodiment, the behavior image of the target object is acquired using the camera, the target object is judged to be in the learning state based on the target state information of the target object in the behavior image, the duration of the learning state of the target object is acquired if the target object is in the learning state, and the prompt voice is output if the duration reaches the preset learning threshold. The state of the target object is judged by the camera to be in the learning state, and a prompt is given when the duration of the learning state is too long, so as to realize automatic monitoring of the learning state and avoid the situation of fatigue and low learning efficiency caused by long-time learning of the target object.

[0099] Referring to Figure 8 , which shows a structure diagram of a learning state detection device provided by an example embodiment of the present application. The learning state detection device can be realized by software, hardware, or a combination of the two to become all or part of the device. The device 1 includes a behavior image acquisition unit 11, a model training unit 15, a learning state judgment unit 12, a duration acquisition unit 13, and a voice prompt unit 14.

[0100] The behavior image acquisition unit 11 is configured to acquire a behavior image of a target object using a camera;

[0101] The model training unit 15 is configured to create an initial state recognition model, acquire a sample behavior image, a sample behavior label corresponding to the sample behavior image, and a sample posture label;

[0102] input the sample behavior image into the initial state recognition model, and obtain training behavior labels and training posture labels output by the initial state recognition model;

[0103] Based on the training behavior labels and the sample behavior labels, and the training posture labels and the sample posture labels, the initial state recognition model is subjected to parameter adjustment processing until model training is completed, and a state recognition model is obtained.

[0104] The learning state judgment unit 12 is configured to judge whether the target object is in a learning state based on target state information of the target object in the behavior image.

[0105] Optionally, the learning state judgment unit 12 is specifically configured to obtain target state information of the target object in the behavior image.

[0106] If the target state information meets the learning state condition, it is determined that the target object is in a learning state.

[0107] Optionally, the target state information includes target object posture and target object behavior.

[0108] The learning state judgment unit 12 is specifically configured to determine that the target object is in a learning state if the target object posture matches the learning posture feature and the target object behavior matches the learning behavior feature.

[0109] Optionally, the learning state judgment unit 12 is specifically configured to input the behavior image into a state detection model.

[0110] Based on the state detection model, the target object posture and the target object behavior of the target object are obtained.

[0111] The state detection model is used to match the target object posture and the learning posture feature, and obtain a posture detection result output by the state detection model.

[0112] The state detection model is used to match the target object behavior and the learning behavior feature, and obtain a behavior detection result output by the state detection model.

[0113] Based on the posture detection result and the behavior detection result, it is determined that the target object is in a learning state.

[0114] The duration acquisition unit 13 is configured to, if the target object is in a learning state, acquire a duration of the learning state of the target object.

[0115] Optionally, the duration obtaining unit 13 is specifically configured to: if the target object is in a learning state, obtain a single duration of the learning state of the target object, and obtain a cumulative duration of the learning state of the target object.

[0116] The voice prompting unit 14 is configured to: if the duration reaches a preset learning threshold, output a prompt voice.

[0117] Optionally, the voice prompting unit 14 is specifically configured to: if the single duration is greater than a first preset duration, output a first rest prompt voice.

[0118] If the cumulative duration is greater than a second preset duration, a second rest prompt voice is outputted.

[0119] In the embodiment, an initial state recognition model is created, sample behavior images, sample behavior image corresponding sample behavior labels and sample posture labels are obtained, the sample behavior images are input into the initial state recognition model, training behavior labels and training posture labels output by the initial state recognition model are obtained, and the initial state recognition model is subjected to parameter adjustment processing based on the training behavior labels and the sample behavior labels and the training posture labels and the sample posture labels until model training is completed to obtain a state recognition model. The model is trained through the sample behavior images and corresponding artificial labeling labels, the accuracy and reliability of the state recognition model are improved, the state recognition model can be adapted to different application environments, and the generalization ability of the model is improved. The behavior images of the target object are collected by the camera, the target state information includes the target object posture and the target object behavior, if the target object posture matches the learning posture feature and the target object behavior matches the learning behavior feature, it is determined that the target object is in the learning state. The learning state of the target object is determined by combining the posture and the behavior, the one-dimensional data bias is reduced to avoid misjudgment, and the accuracy of the learning state detection of the target object is improved. The behavior images are input into the state detection model, the target object posture and the target object behavior of the target object are obtained based on the state detection model, the state detection model is used to match the target object posture and the learning posture feature, the posture detection result output by the state detection model is obtained, the state detection model is used to match the target object behavior and the learning behavior feature, the behavior detection result output by the state detection model is obtained, and it is determined that the target object is in the learning state based on the posture detection result and the behavior detection result. The model is used to participate in the process of target state information extraction and learning state condition judgment, the efficiency of the learning state detection is improved, and the accuracy of the learning state detection is also improved. If the target object is in the learning state, the single continuous duration of the learning state of the target object is obtained, and the cumulative continuous duration of the learning state of the target object is obtained, if the single continuous duration is greater than a first preset duration, a first rest prompt voice is output, and if the cumulative continuous duration is greater than a second preset duration, a second rest prompt voice is output. The single continuous duration and the cumulative continuous duration are reminded to prevent the target object from learning continuously for a long time, and the total learning time of the target object within a period of time is controlled, the learning and rest time are balanced, and the eyes of the target object are also prevented from being damaged and the learning efficiency is improved.

[0120] It should be noted that the learning state detection apparatus provided in the above embodiments is only used for illustrating the division of the functional modules, and in actual applications, the above functions can be completed by different functional modules according to the needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the learning state detection apparatus and the learning state detection method provided in the above embodiments belong to the same concept, and the implementation process is embodied in the method embodiments, which will not be described here.

[0121] The serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0122] The embodiments of the present application also provide a computer storage medium, which can store a plurality of instructions, the instructions being suitable for being loaded and executed by a processor to implement the learning state detection method of the embodiments as shown in the above Figures 1-6 The specific implementation process can refer to the specific description of the embodiments as shown in the above Figures 1-6 The specific implementation process can refer to the specific description of the embodiments as shown in the above

[0123] The present application also provides a computer program product, which stores at least one instruction, the at least one instruction being loaded and executed by the processor to implement the learning state detection method of the embodiments as shown in the above Figures 1-6 The specific implementation process can refer to the specific description of the embodiments as shown in the above Figures 1-6 The specific implementation process can refer to the specific description of the embodiments as shown in the above

[0124] Please refer to Figure 9 , which shows the structural block diagram of the electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application can include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140 and a bus 150. The processor 110, the memory 120, the input device 130 and the output device 140 can be connected through the bus 150.

[0125] The processor 110 can include one or more processing cores. The processor 110 connects various parts within the entire electronic device with various interfaces and lines, performs various functions of the terminal 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Alternatively, the processor 110 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor 110 can integrate a combination of one or several of a central processing unit (CPU), a graphics processor (GPU), and a modem, etc. Among them, the CPU mainly processes an operating system, a user page, and an application program, etc.; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 110, but can be implemented by a separate communication chip.

[0126] The memory 120 can include a random access memory (RAM) and can also include a read-only memory (ROM). Alternatively, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc., and the operating system can be an Android system, an IOS system developed by Apple Inc., a system developed based on the Android system or the IOS system, or other systems.

[0127] The memory 120 can be divided into an operating system space and a user space, the operating system runs in the operating system space, and native and third-party application programs run in the user space. In order to ensure that different third-party application programs can achieve good running effect, the operating system allocates corresponding system resources for different third-party application programs. However, there are also differences in the demand for system resources in different application scenarios in the same third-party application program. For example, in the local resource loading scenario, the third-party application program has a higher requirement for the disk reading speed; in the animation rendering scenario, the third-party application program has a higher requirement for the GPU performance. However, the operating system and the third-party application program are independent of each other, and the operating system often cannot timely perceive the current application scenario of the third-party application program, so that the operating system cannot perform targeted system resource adaptation according to the specific application scenario of the third-party application program.

[0128] In order to enable the operating system to distinguish the specific application scenario of the third-party application program, it is necessary to open up the data communication between the third-party application program and the operating system, so that the operating system can obtain the current scenario information of the third-party application program at any time, and then perform targeted system resource adaptation based on the current scenario.

[0129] The input device 130 is configured to receive input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is configured to output instructions or data, and the output device 140 includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are a touch display screen.

[0130] The touch display screen can be designed as a full screen, a curved screen, or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, and the present application does not limit this.

[0131] In addition, those skilled in the art can understand that the structure of the electronic device shown in the above-mentioned drawings does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the drawings, or combine certain components, or different component arrangements. For example, the electronic device also includes radio frequency circuitry, an input unit, a sensor, audio circuitry, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module, and the like, which are not described here.

[0132] In Figure 9 The processor 110 can be configured to invoke the learning state detection application program stored in the memory 120, and specifically perform the following operations:

[0133] The camera is used to collect the behavior image of the target object;

[0134] Based on the target state information of the target object in the behavior image, it is judged whether the target object is in a learning state;

[0135] If the target object is in a learning state, the duration of the learning state of the target object is obtained;

[0136] If the duration reaches a preset learning threshold, a prompt voice is output.

[0137] In one embodiment, the processor 110, when executing the operation of judging whether the target object is in a learning state based on the target state information of the target object in the behavior image, specifically performs the following operations:

[0138] Obtain the target state information of the target object in the behavior image;

[0139] If the target state information meets the learning state condition, it is determined that the target object is in a learning state.

[0140] In one embodiment, the target state information includes target object posture and target object behavior;

[0141] The processor 110, when executing the operation of determining that the target object is in a learning state if the target state information meets the learning state condition, specifically performs the following operations:

[0142] If the target object posture matches the learning posture feature and the target object behavior matches the learning behavior feature, it is determined that the target object is in a learning state.

[0143] In one embodiment, the processor 110, when executing the operation of obtaining the target state information of the target object in the behavior image, specifically performs the following operations:

[0144] Input the behavior image into a state detection model;

[0145] Obtain the target object posture and target object behavior of the target object based on the state detection model;

[0146] The processor 110, when executing the operation of determining that the target object is in a learning state if the target object posture matches the learning posture feature and the target object behavior matches the learning behavior feature, specifically performs the following operations:

[0147] Match the target object posture and the learning posture feature using the state detection model to obtain the posture detection result output by the state detection model;

[0148] The state detection model is used to match the behavior and learning behavior characteristics of the target object, and obtain a behavior detection result output by the state detection model;

[0149] Based on the posture detection result and the behavior detection result, it is determined that the target object is in a learning state.

[0150] In one embodiment, the processor 110, before executing the input of the behavior image into the state detection model, further executes the following operations:

[0151] An initial state recognition model is created, and a sample behavior image, a sample behavior label and a sample posture label corresponding to the sample behavior image are obtained;

[0152] The sample behavior image is input into the initial state recognition model, and a training behavior label and a training posture label output by the initial state recognition model are obtained;

[0153] Based on the training behavior label and the sample behavior label, and the training posture label and the sample posture label, the initial state recognition model is subjected to parameter adjustment processing until model training is completed, and a state recognition model is obtained.

[0154] In one embodiment, the processor 110, when executing the operation of obtaining the duration of the learning state of the target object if the target object is in a learning state, specifically executes the following operation:

[0155] If the target object is in a learning state, a single duration of the learning state of the target object is obtained, and an accumulated duration of the learning state of the target object is obtained.

[0156] In one embodiment, the processor 110, when executing the operation of outputting the prompt voice if the duration reaches a preset learning threshold, specifically executes the following operation:

[0157] If the single duration is greater than a first preset duration, a first rest prompt voice is output;

[0158] If the accumulated duration is greater than a second preset duration, a second rest prompt voice is output.

[0159] In the embodiment, an initial state recognition model is created, sample behavior images, sample behavior labels corresponding to the sample behavior images, and sample posture labels are obtained, the sample behavior images are input into the initial state recognition model, training behavior labels and training posture labels output by the initial state recognition model are obtained, and the initial state recognition model is subjected to parameter adjustment processing based on the training behavior labels and the sample behavior labels and the training posture labels and the sample posture labels until model training is completed, to obtain a state recognition model. The model is trained through the sample behavior images and corresponding artificial labeling labels, the accuracy and reliability of the state recognition model are improved, the state recognition model can be adapted to different application environments, and the generalization ability of the model is improved. The behavior images of the target object are collected by using a camera, the target state information includes the posture of the target object and the behavior of the target object, if the posture of the target object matches the learning posture feature and the behavior of the target object matches the learning behavior feature, it is determined that the target object is in a learning state. The learning state of the target object is determined by combining the posture and the behavior, the one-dimensional data is reduced, the accuracy of the learning state detection of the target object is improved, the behavior images are input into the state detection model, the posture of the target object and the behavior of the target object are obtained based on the state detection model, the posture of the target object and the learning posture feature are matched by using the state detection model, the posture detection result output by the state detection model is obtained, the behavior of the target object and the learning behavior feature are matched by using the state detection model, the behavior detection result output by the state detection model is obtained, and it is determined that the target object is in a learning state based on the posture detection result and the behavior detection result. The model is used to participate in the process of target state information extraction and learning state condition judgment, the efficiency of the learning state detection is improved, and the accuracy of the learning state detection is also improved. If the target object is in a learning state, a single continuous duration of the learning state of the target object is obtained, and a cumulative continuous duration of the learning state of the target object is obtained, if the single continuous duration is greater than a first preset duration, a first rest prompt voice is output, and if the cumulative continuous duration is greater than a second preset duration, a second rest prompt voice is output. The single continuous duration and the cumulative continuous duration are reminded, the total learning time of the target object in a period of time is controlled while preventing the target object from learning continuously for a long time, the learning and rest time are balanced, and the eyes of the target object are also prevented from being damaged and the learning efficiency is improved.

[0160] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The program can be stored in a computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiments can be included. The storage medium can be a magnetic disc, an optical disc, a read-only memory, or a random access memory.

[0161] The above merely provides the preferred embodiments of the present application, and cannot be used to limit the scope of the present application. Any equivalent changes made according to the claims of the present application shall still fall within the scope of the present application.

[0162] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the behavior images and target state information involved in the present application are obtained under sufficient authorization.

Claims

1. A learning state detection method, characterized in that: The method comprises: A camera is used to collect behavioral images of the target object; determining whether the target object is in a learning state based on target state information of the target object in the behavior image; If the target object is in a learning state, obtaining the duration of the target object's learning state; If the duration reaches a preset learning threshold, a prompt voice is output.

2. The method according to claim 1, characterized in that The determining whether the target object is in a learning state based on the target state information of the target object in the behavior image includes: Acquiring target state information of the target object in the behavior image; If the target state information satisfies a learning state condition, it is determined that the target object is in a learning state.

3. The method according to claim 2, characterized in that The target state information includes the target object posture and target object behavior; If the target state information satisfies a learning state condition, determining that the target object is in a learning state includes: If the target object's posture matches the learning posture feature, and the target object's behavior matches the learning behavior feature, it is determined that the target object is in a learning state.

4. The method according to claim 3, characterized in that The acquiring of target state information of the target object in the behavior image includes: Inputting the behavior image into a state detection model; Acquire a target object posture and a target object behavior of the target object based on the state detection model; If the target object posture matches the learning posture feature, and the target object behavior matches the learning behavior feature, determining that the target object is in a learning state includes: Using the state detection model to match the target object posture and the learning posture feature to obtain a posture detection result output by the state detection model; Using the state detection model to match the target object behavior and learning behavior characteristics to obtain a behavior detection result output by the state detection model; It is determined that the target object is in a learning state based on the posture detection result and the behavior detection result.

5. The method according to claim 4, characterized in that Before inputting the behavior image into the state detection model, the method further includes: Creating an initial state recognition model, obtaining a sample behavior image, a sample behavior label corresponding to the sample behavior image, and a sample posture label; Inputting the sample behavior image into the initial state recognition model, and obtaining the training behavior label and the training posture label output by the initial state recognition model; Based on the training behavior labels and the sample behavior labels, as well as the training posture labels and the sample posture labels, parameter adjustment processing is performed on the initial state recognition model until model training is completed to obtain a state recognition model.

6. The method according to claim 1, characterized in that If the target object is in a learning state, obtaining a duration of the target object's learning state includes: If the target object is in a learning state, a single duration of the target object's learning state is obtained, and a cumulative duration of the target object's learning state is obtained.

7. The method according to claim 6, characterized in that If the duration reaches a preset learning threshold, a prompt voice is output, including: If the single duration is longer than the first preset duration, outputting a first rest prompt voice; If the accumulated duration is greater than a second preset duration, a second rest prompt voice is output.

8. A learning status detection device, characterized in that: The device comprises: A behavior image acquisition unit, used for acquiring behavior images of a target object using a camera; a learning state judgment unit, configured to judge whether the target object is in a learning state based on the target state information of the target object in the behavior image; a duration acquisition unit, configured to acquire the duration of the target object's learning state if the target object is in the learning state; The voice prompt unit is used to output a prompt voice if the duration reaches a preset learning threshold.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps according to any one of claims 1 to 7.

Images