Terminal interface detection method and device, storage medium and electronic equipment

Abstract

The invention provides a terminal interface detection method and device, a storage medium and electronic equipment, the method is applied to the technical field of computers, and the method comprises the steps that a camera is adopted to collect a target terminal interface operated by a target object, the target terminal interface is matched with a game interface sample library, and the game interface sample library is obtained; whether the target terminal interface is a game interface or not is judged, if the target terminal interface is the game interface, it is determined that the target object is in a game state, and prompt voice is output based on the game state. According to the method, whether the current terminal interface of the target object is the game interface or not is detected, whether the target object is in the game state or not is judged, and voice prompt is performed, so that the target object is reminded to continue learning, and game addiction is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and more particularly, to a terminal interface detection method and device, a storage medium and an electronic device. BACKGROUND

[0002] Using a camera to monitor an object in a scene can be used to protect the personal safety of the object in the scene. For example, parents or schools can install a camera in a home scene or a classroom scene to achieve safety protection for students. In the prior art, parents or teachers can determine the learning state of students by audio and video collected by the camera, but parents and teachers cannot view the camera image for a long time, and students are young and lack concentration, so a method for automatically detecting that a student is not in a learning state and providing a prompt is needed. SUMMARY

[0003] The embodiments of the present application provide a terminal interface detection method, device, storage medium and electronic device. The method can determine whether a target object is in a game state and output a prompt voice by detecting whether a current terminal interface of the target object is a game interface, so as to remind the target object to continue learning and avoid being addicted to games.

[0004] In a first aspect, the embodiments of the present application provide a terminal interface detection method, which comprises:

[0005] collecting a target terminal interface operated by a target object by using a camera;

[0006] matching the target terminal interface with a game interface sample library to determine whether the target terminal interface is a game interface;

[0007] if the target terminal interface is a game interface, determining that the target object is in a game state, and outputting a prompt voice based on the game state.

[0008] In a second aspect, the embodiments of the present application provide a terminal interface detection device, which comprises:

[0009] an interface collection unit configured to collect a target terminal interface operated by a target object by using a camera;

[0010] an interface determination unit configured to match the target terminal interface with a game interface sample library to determine whether the target terminal interface is a game interface;

[0011] a voice prompt unit configured to determine that the target object is in a game state if the target terminal interface is a game interface, and output a prompt voice based on the game state.

[0012] In a third aspect, an embodiment of the present application provides a computer storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and performing the method steps described above.

[0013] In a fourth aspect, an embodiment of the present application provides an electronic device, which can include a processor and a memory, wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and performing the method steps described above.

[0014] In one or more embodiments of the present application, a camera is used to collect a target terminal interface operated by a target object, the target terminal interface is matched with a game interface sample library, it is judged whether the target terminal interface is a game interface, if the target terminal interface is a game interface, it is determined that the target object is in a game state, and a prompt voice is output based on the game state. By detecting whether the current terminal interface of the target object is a game interface, it is judged whether the target object is in a game state and a voice prompt is given, so as to remind the target object to continue learning and avoid being addicted to games. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0016] Figure 1 is a scene schematic diagram of a target object monitoring provided by an embodiment of the present application;

[0017] Figure 2 is a flow schematic diagram of a terminal interface detection method provided by an embodiment of the present application;

[0018] Figure 3 is a flow schematic diagram of a terminal interface judgment based on a model provided by an embodiment of the present application;

[0019] Figure 4 is a flow schematic diagram of a matching process provided by an embodiment of the present application;

[0020] Figure 5 is a flow schematic diagram of an interface recognition result generation provided by an embodiment of the present application;

[0021] Figure 6 is a flow schematic diagram of an interface recognition model training provided by an embodiment of the present application;

[0022] Figure 7 is a flowchart of a process for prompting voice output provided by an embodiment of the present application;

[0023] Figure 8 is a structural diagram of a terminal interface detection device provided by an embodiment of the present application;

[0024] Figure 9 is a structural diagram of a terminal interface detection device provided by an embodiment of the present application;

[0025] Figure 10 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0027] In today's daily life, work scene, camera monitoring has become an indispensable security protection tool, which can be applied to family, school or work scene, and the events occurring in the scene are monitored by the camera to ensure the safety and order in the scene. However, due to the diverse needs of people, the monitoring scene and role of the camera are also expanding. For example, in the learning scene of family, school, etc., parents and school staff can use the camera to remotely monitor the learning situation of students. They can remotely check the learning situation of students, understand whether students are focused on learning, whether they need help due to difficulties, or whether they arrange their activities reasonably during rest time. Remote monitoring can enable parents or school staff to master the learning dynamics of students without disturbing them. With the development of smart electronic devices, students often need to use terminal devices such as mobile phones, tablets or computers when learning, for example, they can use terminal devices to learn online courses and view electronic learning materials. However, it is difficult for parents or school staff to continuously view camera monitoring videos for a long time, and students, due to their young age, are easily distracted and difficult to maintain concentration for a long time. They may use terminal devices for activities unrelated to learning, such as playing electronic games. The terminal interface detection device provided in the embodiments of the present application can obtain the terminal interface of the terminal device held by the target object through the monitoring screen of the camera, and then determine whether the target object is in a game state through the content on the terminal interface. The terminal interface detection device can be applied in a family scene or a school teaching scene, and the target object can be a child or a student. The terminal interface detection method provided in the embodiments of the present application can be realized by relying on a computer program and can run on a terminal interface detection device based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool application. The terminal interface detection device can call and control the camera. In addition to capturing the terminal interface of the target object, the camera can also monitor the target object to ensure the safety of the target object in the family scene or the school teaching scene.

[0028] See also Figure 1 A target object monitoring scene is provided for the embodiments of the present application. The camera can be installed by guardians, school staff, etc. The camera viewing angle range needs to cover the target object and the terminal device held by the target object, for example, it can cover the range of the desk and chair of the target object. When it is detected that the target object is using the terminal device, the terminal interface detection device can use the camera to obtain the target terminal interface, which is the terminal interface content of the terminal device being used by the target object. The terminal interface detection device can determine whether the target object is in a game state through the target terminal interface, thereby prompting the target object to stop playing and continue learning.

[0029] The terminal interface detection method provided by the present application will be described in detail below in combination with specific embodiments.

[0030] Please refer to Figure 2 A flowchart of a terminal interface detection method is provided for the embodiments of the present application. As shown in Figure 2 The method of the embodiments of the present application can include the following steps S101-S103.

[0031] S101, a camera is used to collect a target terminal interface operated by a target object.

[0032] Specifically, when the target object enters the visual range of the camera, the terminal interface detection device can collect the monitoring image of the target object. If the terminal device operated by the target object appears in the monitoring image, the target terminal interface of the terminal device can be collected. In order to prompt the target object to maintain a learning state and avoid indulging in electronic games, the terminal interface detection device can control the camera to continuously collect the monitoring image of the target object, for example, the monitoring image of the target object can be collected at a preset collection frequency. The preset collection frequency can be related to the hardware condition of the camera, or related to the computing power of the terminal interface detection device. It can be the initial setting of the terminal interface detection device, or it can be set by the guardian of the target object or relevant staff.

[0033] S102, the target terminal interface is matched with a game interface sample library for processing, and it is determined whether the target terminal interface is a game interface.

[0034] Specifically, the terminal interface detection device can establish a game interface sample library. The game interface sample library can include multiple game interfaces. The game interfaces can be obtained by the terminal interface detection device from the Internet, for example, they can include various game login interfaces, playing interfaces, operation interfaces, etc. The terminal interface detection device can match the collected target terminal interface with the game interface sample library. If there is a game interface in the game interface sample library that matches the target terminal interface, it is determined that the target terminal interface is a game interface. Otherwise, if there is no game interface in the game interface sample library that matches the target terminal interface, it is determined that the target terminal interface is not a game interface.

[0035] S103, if the target terminal interface is a game interface, it is determined that the target object is in a game state, and a prompt voice is output based on the game state.

[0036] Specifically, if the target terminal interface is a game interface, it can be determined that the target object is in a game state, which indicates that the target object is not in a learning state but is operating an electronic game. In order to prompt the target object to continue learning and avoid being addicted to the electronic game, the terminal interface detection device can output a prompt voice according to the game state. The prompt voice can be a prompt for the target user to stop operating the electronic game and continue learning or rest. For example, the prompt voice can be "The game time should not be too long, and you should pay attention to rest your eyes."

[0037] In the embodiments of the present application, a camera is used to collect a target terminal interface operated by a target object, and the target terminal interface is matched with a game interface sample library to determine whether the target terminal interface is a game interface. If the target terminal interface is a game interface, it is determined that the target object is in a game state, and a prompt voice is output based on the game state. By detecting whether the current terminal interface of the target object is a game interface, it is determined whether the target object is in a game state and a voice prompt is given, so as to remind the target object to continue learning and avoid being addicted to the game.

[0038] In order to further improve the accuracy of terminal interface detection and improve the calculation efficiency, and to perceive more deep feature information in the matching process, the terminal interface detection device can train a convolutional neural network (CNN) to participate in the matching process. Since CNN is a deep learning model, it can be applied to image recognition, natural language processing and other fields, which can help to learn the feature information in the target terminal interface and match it with the game interface sample library.

[0039] Please refer to Figure 3 A flowchart of a terminal interface judgment based on a model is provided for the embodiments of the present application. As shown in Figure 3 In one or more embodiments of the present application, the step S102 can include the following steps S201-S203.

[0040] S201, input the target terminal interface into the interface recognition model.

[0041] Specifically, the terminal interface detection device can input the collected target terminal interface into the interface recognition model. The interface recognition model can be trained by the terminal interface detection device, and can determine whether the input terminal interface is a game interface.

[0042] S202, use the interface recognition model to match the target terminal interface with the game interface sample library.

[0043] Specifically, after the interface recognition model obtains the target terminal interface, the target terminal interface can be matched with all game interface samples in the game interface sample library. If the interface recognition model detects that there is a game interface in the game interface sample library that matches the target terminal interface, it can be determined that the target terminal interface is a game interface. Otherwise, if no game interface in the game interface sample library that matches the target terminal interface is detected, it is determined that the target terminal interface is not a game interface.

[0044] Optionally, the game interfaces in the game interface sample library can be associated and saved according to game categories, that is, the game interface sample library can include at least one game category, and the game interfaces corresponding to each game category in the at least one game category. The game category can be used to distinguish the type of electronic game, and can be divided based on the gameplay, theme, platform, and other factors of the electronic game. For example, it can include action games, role-playing games, strategy games, and simulation games. However, the content and gameplay of electronic games are becoming increasingly rich, and the game category can also be used to distinguish different electronic games, that is, each electronic game can be a type of electronic game. Such a division method is more detailed, but it also tests the storage management capabilities of the game interface sample library. The game interfaces corresponding to each game category can include, for example, game main interfaces, battle interfaces, menu interfaces, task interfaces, and other game interfaces of different interface types. The content on the game interfaces in the game interface sample can reflect the corresponding game category and interface type.

[0045] S203, obtaining the interface recognition result output by the interface recognition model.

[0046] Specifically, the interface recognition model can output the interface recognition result according to the matching result. When the interface recognition model detects a game interface in the game interface sample library that matches the target terminal interface, the target terminal interface can be labeled as a game interface, and the interface recognition result can be generated based on the game interface label. Otherwise, when the interface recognition model does not detect a game interface in the game interface sample library that matches the target terminal interface, the target terminal interface can be labeled as a non-game interface, and the interface recognition result can be generated based on the non-game interface label. The game interface label and the non-game interface label are used to inform the terminal interface detection device whether the target terminal interface is a game interface. If the terminal interface detection device obtains the game interface label in the interface recognition result, it indicates that the target terminal interface is a game interface. If the terminal interface detection device obtains the non-game interface label in the interface recognition result, it indicates that the target terminal interface is not a game interface.

[0047] In the embodiment of the present application, the target terminal interface is input into the interface recognition model, the interface recognition model is used to perform matching processing on the target terminal interface and the game interface sample library, and an interface recognition result output by the interface recognition model is obtained. Through the model participating in the matching processing of the terminal interface and the game interface, the efficiency of the matching processing is improved, and the accuracy of the game interface recognition is also improved.

[0048] Please refer to Figure 4 A flowchart of the matching processing is provided for the embodiment of the present application. As shown in Figure 4 The step S202 in one or more embodiments of the present application can include the following steps S301-S302.

[0049] S301, the interface recognition model is used to obtain target feature elements of the target terminal interface.

[0050] Specifically, in order to perform the matching processing more meticulously and accurately, the terminal interface detection device can use the interface recognition model to extract target feature elements of the target terminal interface. The feature elements can be elements for distinguishing game interfaces from non-game interfaces, or elements for distinguishing different game categories, which can include game icons, specific game screen styles, etc. The target feature elements are feature elements extracted from the target terminal interface by the interface recognition model. For example, in a game interface of the action game category, the feature elements can include blood bars, energy bars, and operation keys such as attack and jump. In a game interface of the role-playing game category, the feature elements can include task prompt boxes and map interfaces. In a game interface of the strategy game category, the feature elements can include resource bars and unit information columns. In addition, since the feature elements also include game screen styles, the feature elements can also include cartoon styles and realistic styles.

[0051] S302, the interface recognition model is used to perform matching processing on the target feature elements and the game interface sample library.

[0052] Specifically, the game interface sample library can include different game categories, game interfaces corresponding to each game category, and feature elements in each game interface. Since the game interface sample library can store game interfaces of different section types, the feature elements stored in the game interface sample library can include feature elements of game interfaces at different stages and under different operations. The terminal interface detection apparatus can use the interface recognition model to match the target feature elements with the feature elements of the game interfaces in the game interface sample library. For example, if the matching rate of the target feature elements with the feature elements in the game interface sample library reaches a preset matching rate, it can be determined that the target terminal interface matches the terminal interface sample library. The preset matching rate can be an initial setting of the terminal interface detection apparatus, or can be set by a guardian or relevant staff, for example, the preset matching rate can be 90%.

[0053] Since the feature elements are key feature elements in the terminal interface and the game interface, matching between the feature elements can avoid comparing the entire target terminal interface with all game interfaces at the pixel level, thereby reducing the amount of calculation required for matching processing and improving the efficiency of matching processing. Moreover, the feature elements are unique and distinctive elements in the game interface. Even if the game interface changes with the game progress and player operation, the game interface can still be identified through the feature elements, thereby improving the accuracy and robustness of the matching processing.

[0054] In the embodiments of the present application, the interface recognition model is used to obtain the target feature elements of the target terminal interface, and the interface recognition model is used to match the target feature elements with the game interface sample library. Matching the terminal interface and the game interface sample library through the feature elements reduces the amount of calculation for matching processing, improves the efficiency of matching processing, and also improves the accuracy and stability of matching processing.

[0055] Since the game interface sample library can also include game categories, and the feature elements included in the game interfaces corresponding to different game categories are not the same, and the difference in feature elements is not only reflected in the game icons but also in the game picture style, the terminal interface detection apparatus can determine the game category corresponding to the target terminal interface based on the determination of whether the target terminal interface is a game interface.

[0056] Please refer to Figure 5 A flowchart of interface recognition result generation is provided for the embodiments of the present application. As shown in Figure 5 The steps S401-S402 can be included before the step S203 in one or more embodiments of the present application.

[0057] S401, if the target terminal interface and the game interface sample library do not match, generating an interface recognition result based on a non-game interface label.

[0058] Specifically, if there is no game interface in the game interface sample library that matches the target terminal interface, for example, the matching rate of the target feature element with the feature elements in the game interface sample library is lower than the preset matching rate, it can be determined that the target terminal interface and the game interface sample library do not match, the interface recognition model can label the target terminal interface with a non-game interface label, and generate an interface recognition result based on the non-game interface label.

[0059] S502, if the target terminal interface matches the game interface sample library, obtaining a target game category corresponding to the target terminal interface, and generating an interface recognition result based on the target game category and the game interface label.

[0060] Specifically, if there is a game interface in the game interface sample library that matches the target terminal interface, for example, the matching rate of the target feature element with the feature elements in the game interface sample library is higher than the preset matching rate, it can be determined that the target terminal interface and the game interface sample library match, and the interface recognition model can label the target terminal interface with a game interface label. Further, the interface recognition model can also obtain a target game category corresponding to the target terminal interface, and generate an interface recognition result based on the target game category and the game interface label.

[0061] Optionally, since the feature elements in the game interface sample library are associated with the game interface and the game category corresponding to the game interface, and different games may use elements from the same commercial material library, and games from the same manufacturer may also share elements, the game category with the largest proportion among the feature elements matching the target feature element can be determined as the target game category, the feature elements matching the target feature element in the game interface sample library are confirmed as matching feature elements, and then the proportion of the feature elements corresponding to each game category in the matching feature elements is determined, and the game category with the largest proportion is determined as the target game category. For example, the matching feature elements include feature elements corresponding to game category A, game category B and game category C, the proportion of the feature elements corresponding to game category A in the matching feature elements is 85%, the proportion of the feature elements corresponding to game category B in the matching feature elements is 30%, and the proportion of the feature elements corresponding to game category C in the matching feature elements is 15%. Therefore, the feature elements corresponding to game category A have the largest proportion in the matching feature elements, so game category A can be determined as the target game category corresponding to the target terminal interface.

[0062] Optionally, the target feature elements can also be matched with the feature elements corresponding to each game category in the game interface sample library, and the matching rates with the feature elements of each game category are obtained. If there is a matching rate greater than the preset matching rate, it can be determined that the target terminal interface matches the game interface sample library, and the game category corresponding to the maximum matching rate is determined as the target game category. For example, the matching rate of the target feature elements with the feature elements of game category A is 95%, the matching rate of the target feature elements with the feature elements of game category B is 92%, the matching rate of the target feature elements with the feature elements of game category C is 30%, and the preset matching rate is 90%. There is a matching rate greater than the preset matching rate, so the target terminal interface matches the game interface sample library, and the matching rate corresponding to game category A is the largest, so game category A can be determined as the target game category corresponding to the target terminal interface.

[0063] In the embodiment of the present application, if the target terminal interface does not match the game interface sample library, the interface recognition result is generated based on the non-game interface label, and if the target terminal interface matches the game interface sample library, the target game category corresponding to the target terminal interface is obtained, and the interface recognition result is generated based on the target game category and the game interface label. By determining the target game category corresponding to the target terminal interface, the content of the terminal interface can be further identified by specifying the game category, reducing the possibility of misjudgment, and facilitating individualized prompting and guidance of the target object according to the game category.

[0064] Please refer to Figure 6 A flowchart of interface recognition model training is provided for the embodiment of the present application. As shown in Figure 6 Before the step S201 in one or more embodiments of the present application, the following steps S501-S503 can also be included.

[0065] S501, create an initial interface recognition model, obtain a sample terminal interface, a sample interface recognition result corresponding to the sample terminal interface, and a game interface sample library.

[0066] Specifically, an initial interface recognition model can be created, which has a preliminary ability to match the terminal interface with the game interface in the game interface sample library. The sample terminal interface, the sample interface recognition result corresponding to the sample terminal interface, and the game interface sample library can also be obtained. The game interface sample library can also participate in the training process of the initial interface recognition model, which can improve the accuracy of the matching process of the initial interface recognition model. The sample terminal interface can be obtained from the Internet by the terminal interface detection device, or various terminal device interfaces intercepted or photographed by relevant staff, which can include game interfaces and non-game interfaces. The sample interface recognition result can be manually annotated by relevant staff, for example, relevant staff can manually annotate the sample terminal interface with game interface labels and non-game interface labels.

[0067] Optionally, since the game interface sample library can also contain game categories, the initial interface recognition model can also learn the ability to determine the corresponding game category of the terminal interface during the model training process. Therefore, the game category corresponding to the sample terminal interface can also be included in the sample interface recognition result containing the game interface label.

[0068] S502, input the sample terminal interface into the initial interface recognition model, and obtain the training interface recognition result output by the initial interface recognition model.

[0069] Specifically, the sample terminal interface can be input into the initial interface recognition model, which can match the sample terminal interface with the game interface in the game interface sample library. If there is a game interface in the game interface sample library that matches the sample terminal interface, the initial interface recognition model can label the sample terminal interface with a game interface label and generate a training interface recognition result based on the game interface label. If there is no game interface in the game interface sample library that matches the sample terminal interface, the initial interface recognition model can label the sample terminal interface with a non-game interface label and generate a training interface recognition result based on the non-game interface label.

[0070] Optionally, if there is a game interface in the game interface sample library that matches the sample terminal interface, the initial interface recognition model can also determine the training game category corresponding to the matched game interface, which is the game category corresponding to the sample terminal interface calculated by the initial interface recognition model. Then, the training interface recognition result is generated based on the game interface label and the training game category.

[0071] Optionally, the specific steps of the matching process of the initial interface recognition model can refer to the above-mentioned embodiment steps S301-S302 and S401-S402.

[0072] S503, based on the training interface recognition result and the sample interface recognition result, performing parameter adjustment processing on the initial interface recognition model until the model training is completed, and obtaining the interface recognition model.

[0073] Specifically, the loss function of the initial interface recognition model can be calculated based on the training interface recognition result and the sample interface recognition result, and the initial interface recognition model can be adjusted in the backward propagation training process based on the loss function until the initial interface recognition model is trained to obtain the interface recognition model.

[0074] Optionally, the training interface recognition result and the sample interface recognition result can be used to adjust the parameters of the initial interface recognition model until the training termination condition is met, and the interface recognition model is obtained. The training termination condition can include that the loss function decreases to a preset loss value and remains for a preset time length, the preset loss value is used to determine whether the prediction accuracy of the initial interface recognition model has reached the standard, and the preset time length is used to determine whether the stability of the initial interface recognition model has reached the standard. The preset loss value and the preset time length can be the initial settings of the terminal interface detection device, or can be set by relevant personnel. For example, the preset loss value can be 0.05, and the preset time length can be 100 model training rounds. The training termination condition can also include that the model training rounds of the initial interface recognition model reach a preset round, for example, the preset round can be 100 times. Using the training rounds as the training termination condition is very intuitive and easy to understand, and does not require complex calculation or judgment conditions. Only the rounds need to be set at the beginning of the training, so that the management of the training process is more simple and concise.

[0075] In the embodiments of the present application, an initial interface recognition model is created, a sample terminal interface, a sample interface recognition result corresponding to the sample terminal interface, and a game interface sample library are obtained, the sample terminal interface is input into the initial interface recognition model, a training interface recognition result output by the initial interface recognition model is obtained, and based on the training interface recognition result and the sample interface recognition result, the initial interface recognition model is adjusted until the model training is completed, and the interface recognition model is obtained. Through model training based on the sample terminal interface and the game interface sample library, the accuracy and reliability of the interface recognition model are improved, so that the interface recognition model can adapt to different application environments and improve the generalization ability of the model.

[0076] The target object needs to balance the time allocation of the learning state, the game state and the rest state, and continuously maintaining the learning state will also reduce the learning efficiency of the target object, and long-time maintaining the game state may also lead to the target object being addicted to electronic games. Therefore, the terminal interface detection device does not directly perform voice prompting as long as it detects that the target object is in the game state, but can further analyze the time length of the target object in the game state.

[0077] Please refer to Figure 7 A flowchart for outputting a prompt voice is provided for the embodiments of the present application. As shown in Figure 7 S103, the step S103 in one or more embodiments of the present application can further include steps S601-S602.

[0078] S601, if the single duration of the game state is greater than a first preset duration, output a first rest prompt voice.

[0079] Specifically, the terminal interface detection device can obtain the single duration of the game state, which is the duration of the target object entering the game state. If the single duration of the game state is greater than a first preset duration, a first rest prompt voice can be output. The first preset duration can be used to limit the duration of continuous playing of the electronic game, and can be the initial setting of the terminal interface detection device or set by the guardian or relevant staff. For example, it can be 30 minutes. The first rest prompt voice is used to prompt the target object that the time of playing the electronic game is too long and needs to rest or learn. For example, the first rest prompt voice can be "Please rest your eyes for 30 minutes of continuous game playing".

[0080] S602, if the cumulative duration of the game state is greater than a second preset duration, output a second rest prompt voice.

[0081] Specifically, the terminal interface detection device can obtain the cumulative duration of the game state, which is the cumulative time of the target object entering the game state within a preset time. If the cumulative duration of the game state is greater than a second preset duration, a second rest prompt voice is output. The preset time and the second preset duration can be the initial setting of the terminal interface detection device or set by the guardian or relevant staff. For example, the preset time can be 24 hours and the second preset duration can be 2 hours. The second preset duration can be used to determine whether the target object has the possibility of being addicted to the electronic game. The game time of the target object within the preset time is displayed to maintain enough remaining time for rest and learning. The second rest prompt voice is used to prompt the target object that the game state cannot be entered within the preset time and needs to rest or learn. For example, the second rest prompt voice can be "Please learn or rest for 2 hours of game time today".

[0082] In the embodiments of the present application, if the single duration of the game state is greater than the first preset duration, the first rest prompt voice is output, and if the cumulative duration of the game state is greater than the second preset duration, the second rest prompt voice is output. Through the single duration reminder and the cumulative duration reminder, the total game time of the target object within a period of time can be controlled while preventing the target object from playing games for a long time, and the eyes of the target object are prevented from being damaged, the learning of the target object is prevented from being delayed, and the target object is prevented from resting at the same time.

[0083] Since the application scene of the terminal interface detection device can be a family, a school, and the like, the target objects monitored are students with a small age, and the students have the characteristics of a relatively scattered attention, a poor self-control, and a low cognitive level. Therefore, not all electronic games are suitable for the target objects to play, and the terminal interface detection device can not only detect whether the target terminal interface is a game interface, but also further detect the target game type. Therefore, the guardian or the relevant staff can limit the game type played by the target object according to the needs, so as to avoid the target object from playing electronic games that are not suitable for the age of the target object without the guardian or the relevant staff supervising.

[0084] In one or more embodiments of the present application, the terminal interface detection method can further include the following steps:

[0085] If the target game type exists in the game type blacklist, an alarm prompt voice is output.

[0086] Specifically, the guardian or the relevant staff can set a game type blacklist in the terminal interface detection device. For example, game types involving bloody violence and the like can be added to the game type blacklist. If the target game type corresponding to the target terminal interface exists in the game type blacklist, an alarm prompt voice can be output. The alarm prompt voice can be used to prompt the target object to stop playing the current electronic game.

[0087] Optionally, if the target object is still in the game state after the prompt voice is output, a game alarm information can be sent to a preset contact terminal. The prompt voice can include the first rest prompt voice, the second rest prompt voice, and the alarm prompt voice. The game alarm information prompts the holder of the preset contact terminal that the target object is in the game state for a long time, which can cause the risk of damaging the eyes and delaying the learning and rest. The holder of the preset contact terminal needs to prompt the target object. The preset contact terminal can be set by the guardian or the relevant staff. For example, the holder of the preset contact terminal can be the guardian of the target object.

[0088] In the embodiment of the present application, if the target game category exists in the game category blacklist, an alarm prompt voice is output. By limiting the game category played by the target object, the target object is prevented from contacting content affecting mental health or generating negative values.

[0089] The application will be described in more detail below with reference to the accompanying drawings. Figure 8 - The terminal interface detection device Figure 9 The terminal interface detection device provided by the embodiment of the present application will be described in detail. It should be noted that the terminal interface detection device Figure 8 - The terminal interface detection device Figure 9 The terminal interface detection device in the embodiment of the present application is used to execute the method of the embodiment of the present application. Figures 1-7 The specific technical details are not disclosed, please refer to the embodiment shown in the present application. Figures 1-7 The specific technical details are not disclosed, please refer to the embodiment shown in the present application.

[0090] Please refer to Figure 8 , which shows the structure diagram of the terminal interface detection device provided by an exemplary embodiment of the present application. The terminal interface detection device can be realized by software, hardware or a combination of the two to become all or part of the device. The device 1 includes an interface collection unit 11, an interface judgment unit 12 and a voice prompt unit 13.

[0091] The interface collection unit 11 is used to collect the target terminal interface operated by the target object by using a camera;

[0092] The interface judgment unit 12 is used to match the target terminal interface with a game interface sample library, and judge whether the target terminal interface is a game interface;

[0093] The voice prompt unit 13 is used to determine that the target object is in a game state if the target terminal interface is a game interface, and output a prompt voice based on the game state.

[0094] In the embodiment, the target terminal interface operated by the target object is collected by using a camera, the target terminal interface is matched with a game interface sample library, it is judged whether the target terminal interface is a game interface, if the target terminal interface is a game interface, it is determined that the target object is in a game state, and a prompt voice is output based on the game state. By detecting whether the current terminal interface of the target object is a game interface, it is judged whether the target object is in a game state and a voice prompt is made, so as to achieve the purpose of reminding the target object to continue learning and avoiding indulging in games.

[0095] Please refer to Figure 9Fig. 1 is a structural schematic diagram of a terminal interface detection device according to an example embodiment of the present application. The terminal interface detection device can be implemented as all or part of the device by software, hardware or a combination of both. The device 1 comprises an interface collection unit 11, a model training unit 14, an interface judgment unit 12, a voice prompt unit 13 and an alarm output unit 15.

[0096] The interface collection unit 11 is configured to collect a target terminal interface operated by a target object by using a camera.

[0097] The model training unit 14 is configured to create an initial interface recognition model, acquire a sample terminal interface, a sample interface recognition result corresponding to the sample terminal interface and a game interface sample library.

[0098] The sample terminal interface is input into the initial interface recognition model to acquire a training interface recognition result output by the initial interface recognition model.

[0099] The initial interface recognition model is subjected to parameter adjustment processing based on the training interface recognition result and the sample interface recognition result until model training is completed to obtain an interface recognition model.

[0100] The interface judgment unit 12 is configured to match the target terminal interface with the game interface sample library to determine whether the target terminal interface is a game interface.

[0101] Optionally, the interface judgment unit 12 is specifically configured to input the target terminal interface into the interface recognition model.

[0102] The target terminal interface is matched with the game interface sample library by using the interface recognition model. The game interface sample library comprises at least one game category and a game interface corresponding to each game category in the at least one game category.

[0103] An interface recognition result output by the interface recognition model is acquired.

[0104] Optionally, the interface judgment unit 12 is specifically configured to acquire a target feature element of the target terminal interface by using the interface recognition model.

[0105] The target feature element is matched with the game interface sample library by using the interface recognition model.

[0106] Optionally, the interface judgment unit 12 is further configured to generate an interface recognition result based on a non-game interface label if the target terminal interface and the game interface sample library do not match.

[0107] If the target terminal interface matches the game interface sample library, a target game category corresponding to the target terminal interface is acquired, and an interface recognition result is generated based on the target game category and a game interface label.

[0108] The voice prompt unit 13 is configured to determine that the target object is in a game state if the target terminal interface is a game interface, and output a prompt voice based on the game state.

[0109] Optionally, the voice prompt unit 13 is specifically configured to output a first rest prompt voice if a single duration of the game state is greater than a first preset duration.

[0110] If a cumulative duration of the game state is greater than a second preset duration, a second rest prompt voice is output.

[0111] The alarm output unit 15 is configured to output an alarm prompt voice if the target game category exists in a game category blacklist.

[0112] In the embodiment, an initial interface recognition model is created, sample terminal interfaces, sample interface recognition results corresponding to the sample terminal interfaces, and a game interface sample library are obtained, the sample terminal interfaces are input into the initial interface recognition model, training interface recognition results output by the initial interface recognition model are obtained, and the initial interface recognition model is subjected to parameter adjustment processing based on the training interface recognition results and the sample interface recognition results until model training is completed, so as to obtain an interface recognition model. The model is trained through the sample terminal interfaces and the game interface sample library, the accuracy and reliability of the interface recognition model are improved, the interface recognition model can be adapted to different application environments, and the generalization ability of the model is improved. A camera is used to collect a target terminal interface operated by a target object, the target terminal interface is input into the interface recognition model, the target terminal interface is subjected to target feature element extraction through the interface recognition model, and the target feature element is subjected to matching processing with the game interface sample library through the interface recognition model. The terminal interface and the game interface sample library are subjected to matching processing through the feature element, the calculation amount of the matching processing is reduced, the efficiency of the matching processing is improved, and the accuracy and stability of the matching processing are also improved. If the target terminal interface and the game interface sample library do not match, an interface recognition result is generated based on a non-game interface label, if the target terminal interface matches the game interface sample library, a target game type corresponding to the target terminal interface is obtained, and an interface recognition result is generated based on the target game type and a game interface label. The target game type corresponding to the target terminal interface is determined, the content of the terminal interface can be further recognized through the explicit game type, the possibility of misjudgment is reduced, the target object can be prompted and guided individually according to the game type, and an interface recognition result output by the interface recognition model is obtained. The efficiency of the matching processing is improved through the model participating in the matching processing process of the terminal interface and the game interface, and the accuracy of the game interface recognition is also improved. If the single continuous duration of the game state is greater than a first preset duration, a first rest prompt voice is output, and if the cumulative continuous duration of the game state is greater than a second preset duration, a second rest prompt voice is output. The single continuous duration and the cumulative continuous duration are reminded, the target object is prevented from playing games for a long time, the total game time of the target object within a period of time is controlled, the game and the learning are balanced, and the eyes of the target object are prevented from being damaged, the learning of the target object is prevented from being delayed, and the target object is prevented from resting. If the target game type exists in a game type blacklist, an alarm prompt voice is output. The game type played by the target object is limited, and the target object is prevented from contacting content that affects mental health or produces a negative value.

[0113] The target terminal interface is matched with a game interface sample library, it is judged whether the target terminal interface is a game interface, if the target terminal interface is a game interface, it is determined that the target object is in a game state, and a prompt voice is output based on the game state. By detecting whether the current terminal interface of the target object is a game interface, it is judged whether the target object is in a game state and a voice prompt is given, so as to remind the target object to continue learning and avoid being addicted to games.

[0114] It should be noted that the terminal interface detection device provided in the above embodiment is used to execute the terminal interface detection method, and only the division of the above functional modules is used as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the terminal interface detection device and the terminal interface detection method provided in the above embodiment belong to the same concept, and the implementation process is described in detail in the method embodiment, which will not be repeated here.

[0115] The serial numbers of the above embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0116] The embodiment of the present application further provides a computer storage medium, which can store a plurality of instructions, the instructions being suitable for being loaded and executed by a processor to execute the terminal interface detection method of the above-described Figures 1-7 The specific execution process can refer to the specific description of the terminal interface detection method of the above-described Figures 1-7 The specific execution process can refer to the specific description of the terminal interface detection method of the above-described

[0117] The present application further provides a computer program product, which stores at least one instruction, the at least one instruction being loaded and executed by the processor to execute the terminal interface detection method of the above-described Figures 1-7 The specific execution process can refer to the specific description of the terminal interface detection method of the above-described Figures 1-7 The specific execution process can refer to the specific description of the terminal interface detection method of the above-described

[0118] Please refer to Figure 10 , which shows a structural block diagram of an electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application can include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140 and a bus 150. The processor 110, the memory 120, the input device 130 and the output device 140 can be connected through the bus 150.

[0119] The processor 110 can include one or more processing cores. The processor 110 connects various parts within the entire electronic device with various interfaces and lines, performs various functions of the terminal 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Alternatively, the processor 110 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor 110 can integrate a combination of one or several of a central processing unit (CPU), a graphics processor (GPU), and a modem, etc. Among them, the CPU mainly processes an operating system, a user page, and an application program, etc.; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 110, but can be implemented by a separate communication chip.

[0120] The memory 120 can include a random access memory (RAM) and can also include a read-only memory (ROM). Alternatively, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc., and the operating system can be an Android system, an IOS system developed by Apple Inc., a system developed based on the Android system or the IOS system, or other systems.

[0121] The memory 120 can be divided into an operating system space and a user space, the operating system runs in the operating system space, and native and third-party application programs run in the user space. In order to ensure that different third-party application programs can achieve good running effect, the operating system allocates corresponding system resources for different third-party application programs. However, there are also differences in the demand for system resources in different application scenarios in the same third-party application program. For example, in the local resource loading scenario, the third-party application program has a higher requirement for the disk reading speed; in the animation rendering scenario, the third-party application program has a higher requirement for the GPU performance. However, the operating system and the third-party application program are independent of each other, and the operating system often cannot timely perceive the current application scenario of the third-party application program, so that the operating system cannot perform targeted system resource adaptation according to the specific application scenario of the third-party application program.

[0122] In order to enable the operating system to distinguish the specific application scenario of the third-party application program, it is necessary to open up the data communication between the third-party application program and the operating system, so that the operating system can obtain the current scenario information of the third-party application program at any time, and then perform targeted system resource adaptation based on the current scenario.

[0123] The input device 130 is configured to receive input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is configured to output instructions or data, and the output device 140 includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are a touch display screen.

[0124] The touch display screen can be designed as a full screen, a curved screen, or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, and the present application does not limit this.

[0125] In addition, those skilled in the art can understand that the structure of the electronic device shown in the above-mentioned drawings does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the drawings, or combine certain components, or different component arrangements. For example, the electronic device also includes radio frequency circuitry, an input unit, a sensor, audio circuitry, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module, and the like, which are not described here.

[0126] In Figure 10 In the electronic device shown in the figure, the processor 110 can be configured to invoke the terminal interface detection application program stored in the memory 120, and specifically perform the following operations:

[0127] The camera is used to collect a target terminal interface operated by a target object;

[0128] The target terminal interface is matched with a game interface sample library to determine whether the target terminal interface is a game interface;

[0129] If the target terminal interface is a game interface, it is determined that the target object is in a game state, and a prompt voice is output based on the game state.

[0130] In one embodiment, when the processor 110 performs matching the target terminal interface with the game interface sample library to determine whether the target terminal interface is a game interface, the following operations are specifically performed:

[0131] The target terminal interface is input into an interface recognition model;

[0132] The target terminal interface is matched with the game interface sample library using the interface recognition model, the game interface sample library includes at least one game category, and each game category in the at least one game category corresponds to a game interface;

[0133] An interface recognition result output by the interface recognition model is obtained.

[0134] In one embodiment, when the processor 110 performs matching the target terminal interface with the game interface sample library using the interface recognition model, the following operations are specifically performed:

[0135] The target terminal interface is matched with the game interface sample library using the interface recognition model, the game interface sample library includes at least one game category, and each game category in the at least one game category corresponds to a game interface;

[0136] The target terminal interface is matched with the game interface sample library using the interface recognition model.

[0137] In one embodiment, before the processor 110 performs obtaining the interface recognition result output by the interface recognition model, the following operations are further performed:

[0138] If the target terminal interface and the game interface sample library do not match, an interface recognition result is generated based on a non-game interface label;

[0139] If the target terminal interface matches the game interface sample library, a target game category corresponding to the target terminal interface is obtained, and an interface recognition result is generated based on the target game category and a game interface label.

[0140] In one embodiment, before the processor 110 performs inputting the target terminal interface into the interface recognition model, the following operations are further performed:

[0141] create an initial interface recognition model, obtain a sample terminal interface, a sample interface recognition result corresponding to the sample terminal interface, and a game interface sample library;

[0142] input the sample terminal interface into the initial interface recognition model, and obtain a training interface recognition result output by the initial interface recognition model;

[0143] based on the training interface recognition result and the sample interface recognition result, perform parameter adjustment processing on the initial interface recognition model until model training is completed, and obtain an interface recognition model.

[0144] In one embodiment, the processor 110, when outputting the prompt voice based on the game state, specifically performs the following operations:

[0145] if the single duration of the game state is greater than a first preset duration, a first rest prompt voice is output;

[0146] if the cumulative duration of the game state is greater than a second preset duration, a second rest prompt voice is output.

[0147] In one embodiment, the processor 110, when executing the terminal interface detection method, further performs the following operations:

[0148] if the target game category exists in a game category blacklist, an alarm prompt voice is output.

[0149] In the embodiment, an initial interface recognition model is created, sample terminal interfaces, sample interface recognition results corresponding to the sample terminal interfaces, and a game interface sample library are obtained, the sample terminal interfaces are input into the initial interface recognition model, training interface recognition results output by the initial interface recognition model are obtained, and the initial interface recognition model is subjected to parameter adjustment processing based on the training interface recognition results and the sample interface recognition results until model training is completed, so as to obtain an interface recognition model. The model is trained through the sample terminal interfaces and the game interface sample library, the accuracy and reliability of the interface recognition model are improved, the interface recognition model can be adapted to different application environments, and the generalization ability of the model is improved. A camera is used to collect a target terminal interface operated by a target object, the target terminal interface is input into the interface recognition model, the target terminal interface is subjected to target feature element extraction through the interface recognition model, and the target feature element is subjected to matching processing with the game interface sample library through the interface recognition model. The terminal interface and the game interface sample library are subjected to matching processing through the feature element, the calculation amount of the matching processing is reduced, the efficiency of the matching processing is improved, and the accuracy and stability of the matching processing are also improved. If the target terminal interface and the game interface sample library do not match, an interface recognition result is generated based on a non-game interface label, if the target terminal interface matches the game interface sample library, a target game type corresponding to the target terminal interface is obtained, and an interface recognition result is generated based on the target game type and a game interface label. The target game type corresponding to the target terminal interface is determined, the content of the terminal interface can be further recognized through the explicit game type, the possibility of misjudgment is reduced, the target object can be prompted and guided individually according to the game type, and an interface recognition result output by the interface recognition model is obtained. The efficiency of the matching processing is improved through the model participating in the matching processing process of the terminal interface and the game interface, and the accuracy of the game interface recognition is also improved. If the single continuous duration of the game state is greater than a first preset duration, a first rest prompt voice is output, and if the cumulative continuous duration of the game state is greater than a second preset duration, a second rest prompt voice is output. The single continuous duration and the cumulative continuous duration are reminded, the target object is prevented from playing games for a long time, the total game time of the target object within a period of time is controlled, the game and the learning are balanced, and the eyes of the target object are prevented from being damaged, the learning of the target object is prevented from being delayed, and the target object is prevented from resting. If the target game type exists in a game type blacklist, an alarm prompt voice is output. The game type played by the target object is limited, and the target object is prevented from contacting content that affects mental health or produces a negative value.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory or a random access memory, etc.

[0151] The above only describes the preferred embodiments of the present application, and cannot limit the scope of the present application. Any equivalent changes made according to the claims of the present application are still within the scope of the present application.

[0152] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the monitoring images and target terminal interfaces involved in the present application are obtained under sufficient authorization.

Claims

1. A terminal interface detection method, characterized in that: The method comprises: Use a camera to capture the target terminal interface operated by the target object; Matching the target terminal interface with a game interface sample library to determine whether the target terminal interface is a game interface; If the target terminal interface is a game interface, it is determined that the target object is in a game state, and a prompt voice is output based on the game state.

2. The method according to claim 1, characterized in that The matching process of the target terminal interface with a game interface sample library to determine whether the target terminal interface is a game interface includes: Inputting the target terminal interface into an interface recognition model; Using the interface recognition model, matching the target terminal interface with the game interface sample library, the game interface sample library including at least one game category and a game interface corresponding to each game category in the at least one game category; Obtaining the interface recognition result output by the interface recognition model.

3. The method according to claim 2, characterized in that The using the interface recognition model to match the target terminal interface with the game interface sample library includes: Using the interface recognition model, obtaining target feature elements of the target terminal interface; The interface recognition model is used to match the target feature element with the game interface sample library.

4. The method according to claim 2, characterized in that Before obtaining the interface recognition result output by the interface recognition model, the method further includes: If the target terminal interface does not match the game interface sample library, generating an interface recognition result based on the non-game interface tag; If the target terminal interface matches the game interface sample library, the target game type corresponding to the target terminal interface is obtained, and an interface recognition result is generated based on the target game type and the game interface label.

5. The method according to claim 2, characterized in that Before inputting the target terminal interface into the interface recognition model, the method further includes: Create an initial interface recognition model, obtain a sample terminal interface, a sample interface recognition result corresponding to the sample terminal interface, and a game interface sample library; Inputting the sample terminal interface into the initial interface recognition model to obtain a training interface recognition result output by the initial interface recognition model; Based on the training interface recognition result and the sample interface recognition result, the initial interface recognition model is subjected to parameter adjustment processing until the model training is completed to obtain an interface recognition model.

6. The method according to claim 1, characterized in that The outputting of a prompt voice based on the game status includes: If the single duration of the game state is longer than a first preset duration, outputting a first rest prompt voice; If the accumulated duration of the game state is greater than a second preset duration, a second rest prompt voice is output.

7. The method according to claim 4, characterized in that The method further comprises: If the target game type exists in the game type blacklist, an alarm prompt voice is output.

8. A terminal interface detection device, characterized in that: The device comprises: An interface acquisition unit, configured to use a camera to acquire the target terminal interface operated by the target object; An interface judgment unit, configured to match the target terminal interface with a game interface sample library to determine whether the target terminal interface is a game interface; The voice prompt unit is used to determine that the target object is in a game state if the target terminal interface is a game interface, and output a prompt voice based on the game state.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps according to any one of claims 1 to 7.

Images