Screen control method and device, terminal, storage medium and program product

Through camera body recognition and continuous multi-frame image prediction, intelligent control of the screen status of the smart terminal is solved, and the problem of low intelligence in the existing technology is achieved, and more accurate user intention judgment and screen status management are achieved.

CN118939104APending Publication Date: 2024-11-12伟光有限公司(CN)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310473789.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, the intelligent degree of screen status switching of smart terminals is low, which affects the normal use of users, and it is difficult to accurately determine whether the user has the intention to use the terminal.

Method used

Images are collected by the camera for human recognition, and use behavior prediction is performed based on the recognition results of continuous multi-frame images. The screen is controlled to be in a state of illuminating or off to ensure that the state switching is in line with the user's intention.

Benefits of technology

It improves the accuracy of user usage behavior prediction, improves the intelligence of screen status control, and ensures that the screen status is more in line with user usage needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939104A_ABST
    Figure CN118939104A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a screen control method and device, a terminal, a storage medium and a program product, and belongs to the technical field of human-computer interaction. The method comprises the following steps: performing human body recognition on an image collected by a camera to obtain a human body recognition result; using behavior prediction is carried out on the basis of the human body recognition results of the continuous multi-frame images, a using behavior prediction result is obtained, and the using behavior prediction result is used for representing whether there is the intention of using the terminal or not; under the condition that the use behavior prediction result represents that the intention of using the terminal exists, a screen is controlled to be in a screen-on state; and under the condition that the use behavior prediction result represents that the intention of using the terminal does not exist, controlling the screen to be in a screen-off state. By adopting the scheme provided by the embodiment of the invention, behavior prediction can be performed according to the acquired continuous multi-frame images, and whether the user has the intention of using the terminal or not can be accurately predicted, so that the screen state is controlled to accord with the user intention.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of human-computer interaction technology, and in particular to a screen control method, device, terminal, storage medium and program product. Background Art

[0002] Nowadays, smart terminals are indispensable in life, and users often hope that the screen of the smart terminal will automatically turn off or on according to the user's usage intention.

[0003] In the related art, if the terminal does not receive any user operation on the screen within a preset screen-off time interval, it automatically switches to the screen-off state. Or the user actively lights up the screen and then unlocks it when there is a need for use.

[0004] However, the intelligence level of controlling screen status switching in the related art is low, which often affects the user's normal use of the smart terminal when the screen is switched. Summary of the invention

[0005] The embodiments of the present application provide a screen control method, device, terminal, storage medium and program product. The technical solution is as follows:

[0006] On the one hand, an embodiment of the present application provides a screen control method, the method comprising:

[0007] Perform human body recognition on the images collected by the camera to obtain human body recognition results;

[0008] Performing usage behavior prediction based on the human body recognition results of the continuous multiple frames of images to obtain a usage behavior prediction result, wherein the usage behavior prediction result is used to characterize whether there is an intention to use the terminal;

[0009] When the usage behavior prediction result indicates that there is an intention to use the terminal, controlling the screen to be in a bright screen state;

[0010] When the usage behavior prediction result indicates that there is no intention to use the terminal, the screen is controlled to be in an off state.

[0011] On the other hand, an embodiment of the present application provides a screen control device, the device comprising:

[0012] The recognition module is used to perform human body recognition on the images collected by the camera to obtain human body recognition results;

[0013] A prediction module, used to predict the use behavior based on the human body recognition results of the continuous multiple frames of images to obtain a use behavior prediction result, wherein the use behavior prediction result is used to characterize whether there is an intention to use the terminal;

[0014] A control module, configured to control the screen to be in a bright screen state when the usage behavior prediction result indicates that there is an intention to use the terminal;

[0015] The control module is further configured to control the screen to be in an off state when the usage behavior prediction result indicates that there is no intention to use the terminal.

[0016] On the other hand, an embodiment of the present application provides a terminal, which includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the screen control method as described in the above aspect.

[0017] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one program code is stored, and the program code is loaded and executed by a processor to implement the screen control method as described in the above aspects.

[0018] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the screen control method provided in various optional implementations of the above aspects.

[0019] In the embodiment of the present application, the terminal performs human body recognition on the images captured by the camera, and then predicts the use behavior of the human body recognition results of the continuous multi-frame images, and controls the screen state according to whether the user has the intention to use the screen. The use behavior prediction using the human body recognition results of the continuous multi-frame images can avoid the wrong prediction of the user's use behavior when the user accidentally appears in front of the screen but has no intention to use the screen, and improves the accuracy of the prediction of the user's use behavior, so that the screen state can be controlled to be more in line with the user's intention, and the degree of intelligence is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A schematic diagram of a hardware structure for executing a screen state control solution provided by an exemplary embodiment of the present application is shown;

[0022] Figure 2A flowchart of a screen control method provided by an exemplary embodiment of the present application is shown;

[0023] Figure 3 A flowchart showing a screen control process in a flat state provided by an exemplary embodiment of the present application is shown;

[0024] Figure 4 A schematic diagram of switching human feature recognition frequency provided by an exemplary embodiment of the present application is shown;

[0025] Figure 5 An example diagram of the proportion of a human body in an image provided by an exemplary embodiment of the present application is shown;

[0026] Figure 6 A schematic diagram showing a target range provided by an exemplary embodiment of the present application is shown;

[0027] Figure 7 A schematic diagram showing a change in the position of a human face provided by an exemplary embodiment of the present application is shown;

[0028] Figure 8 A flowchart of a screen control strategy provided by an exemplary embodiment of the present application is shown;

[0029] Fig. 9 A structural block diagram of a screen control device provided by an embodiment of the present application is shown;

[0030] Fig.10 A structural block diagram of a terminal provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0032] It should be noted that before the application captures images through the camera and in the process of collecting relevant data of the user, a prompt interface, pop-up window or voice prompt information can be displayed. The prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that the application only starts to execute the relevant steps of obtaining the relevant data of the user after obtaining the confirmation operation issued by the user to the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user to the prompt interface or pop-up window is not obtained), the relevant steps of obtaining the relevant data of the user are terminated, that is, the relevant data of the user is not obtained. In other words, all user data collected by this application are collected with the consent and authorization of the user, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0033] The embodiments of the present application are executed by a terminal, which may be an electronic device such as a smart phone, a foldable screen mobile phone, a tablet computer, a wearable electronic device, etc. Figure 1 FIG. 1 shows a schematic diagram of a hardware structure for executing a screen state control solution provided by an exemplary embodiment of the present application. Figure 1 As shown, the hardware structure for executing screen state control mainly includes: a camera 101 , an AON (Always ON) module 102 , a screen control module 103 and a screen 104 .

[0034] It should be noted that Figure 1 Only the main components used to execute the embodiments of the present application are shown. In addition, the terminal executing the embodiments of the present application should also include components such as memory, sensors, speakers, etc. Those skilled in the art can configure more components for the terminal to achieve other functions, and the embodiments of the present application are not limited to this.

[0035] Typically, a terminal includes multiple cameras, and the multiple cameras are arranged on different sides of the terminal. In the embodiment of the present application, camera 101 refers to a front camera located on the side where screen 104 is located, which is used to capture user behavior, and then perform human body recognition and usage behavior prediction on the collected images through AON module 102.

[0036] In some embodiments, the embedded system is divided into a hardware layer, a driver layer, an operating system layer and an application layer, wherein the driver layer is used to implement the terminal driver program, the AON module 102 belongs to the driver layer, the AON module 102 has low power consumption, and the power consumption of the process in the application layer is high.

[0037] The AON module 102 is connected to the camera 101, and the AON module 102 can send a control signal to the camera 101 through the connection, and obtain the image frame data captured by the camera 101 through the connection. The AON module 102 can be used to perform signal processing, and process the image frame data captured by the camera 101. In addition, the power consumption of the AON module 102 for image data processing is low. In some embodiments, the AON module 102 can control the camera 101 to start up around the clock.

[0038] In addition, the AON module 102 is also connected to the screen control module 103, and the screen control module 103 can change the screen state without application layer control. The AON module 102 sends a screen state control instruction to the screen control module 103, and the screen state control instruction is used to instruct the screen control module 103 to control the screen state.

[0039] The screen 104 is connected to the screen control module 103. The screen state is controlled by the screen control module 103. The screen state includes a screen-on state and a screen-off state. After receiving the control signal from the screen control module 103, the screen 104 is converted into a corresponding screen state.

[0040] Indicatively, Figure 1 As shown, the AON module 102 controls the camera 101 to continuously collect the environmental image in front of the screen 104. Subsequently, the AON module 102 obtains the collected image from the camera 101 and performs human body recognition. When a human body is recognized, the human body recognition results in multiple consecutive frames of images are predicted for usage behavior. When it is predicted that the user has no intention of using the terminal, the AON module 102 sends a screen-off instruction to the screen control module so that the screen control module controls the screen 104 to be in an off state; when it is predicted that the user has the intention of using the terminal, the AON module 102 sends a screen-on instruction to the screen control module so that the screen control module controls the screen 104 to be in a bright state.

[0041] Figure 2 A flowchart of a screen control method provided by an exemplary embodiment of the present application is shown, and the method comprises the following steps:

[0042] Step 201, performing human body recognition on the image captured by the camera to obtain a human body recognition result.

[0043] In the embodiment of the present application, the AON module continuously collects environmental images through the camera. Normally, when the user intends to use the terminal, he will look at the mobile phone screen, and the camera will capture the human body. Therefore, the terminal can perform human body recognition on the image collected by the camera and determine whether the user has the intention to use the terminal based on the human body recognition result.

[0044] In a possible implementation, the terminal can perform human body recognition on images captured by the camera through a neural network model, specifically, it can be RCNN (Region-Convolution Neural Network), R-FCN (Region-based Fully Convolutional Networks), yolov2 algorithm, etc.

[0045] In the embodiment of the present application, the human body recognition result indicates whether there is a human body in the image captured by the camera. If a human body is captured, the terminal further determines whether the user has the intention to use the terminal. If a human body is not captured by the camera, the terminal can directly determine that the user does not have the intention to use the terminal and continue to collect images through the camera.

[0046] Step 202 , performing usage behavior prediction based on the human body recognition results of the continuous multiple-frame images to obtain a usage behavior prediction result.

[0047] The usage behavior prediction result is used to characterize whether there is an intention to use the terminal.

[0048] In actual application scenarios, a user may briefly appear in the camera's shooting range, but quickly leave the camera's shooting range. In this case, the user most likely does not have the intention to use the terminal. Therefore, in order to avoid the terminal's incorrect prediction of the intention to use the terminal based on the human body recognition results in this case, after the terminal recognizes the human body, it also needs to perform usage behavior prediction based on multiple consecutive frames of human body recognition results to predict whether the user has the intention to use the terminal.

[0049] In a possible implementation, when the human body recognition results of multiple consecutive frames of images all indicate that there is a human body in the captured images, the terminal obtains a usage behavior prediction result representing the intention to use the terminal; when the human body recognition results of multiple consecutive frames of images indicate that there is a human body that does not exist in the captured images, the terminal obtains a usage behavior prediction result representing that there is no intention to use the terminal.

[0050] In another possible implementation, when there are more images without human bodies than images with human bodies in the human body recognition results of multiple consecutive frames of images, the terminal obtains a usage behavior prediction result indicating that there is no intention to use the terminal; and when there are fewer images without human bodies than images with human bodies in the human body recognition results of multiple consecutive frames of images, the terminal obtains a usage behavior prediction result indicating that there is an intention to use the terminal.

[0051] Step 203 , when the usage behavior prediction result indicates that there is an intention to use the terminal, the screen is controlled to be in a bright state.

[0052] When the current screen is in a light-on state and the behavior prediction result indicates that the user has an intention to use the terminal, the screen is controlled to remain in the light-on state.

[0053] When the current screen is in an off state, and when the behavior prediction result indicates that the user has an intention to use the terminal, the screen is controlled to be switched from the off state to the on state.

[0054] Optionally, if the current screen is in the off state, the terminal needs to verify the user in front of the screen before switching to the bright screen state to ensure safety. In one possible implementation, the terminal performs human body recognition while performing feature recognition on the image captured by the camera. When the human body in the image matches the target human body pre-stored in the terminal, a human body recognition result is obtained indicating that the target user has been identified. When the prediction result indicates that there is an intention to use the terminal and the target user has been identified, the screen is controlled to be in the bright screen state.

[0055] Step 204 , when the usage behavior prediction result indicates that there is no intention to use the terminal, the screen is controlled to be in an off state.

[0056] When the current screen is in the on state, and the behavior prediction result indicates that the user has no intention to use the terminal, the screen is controlled to switch to the off state.

[0057] When the current screen is in an off state and the behavior prediction result indicates that the user has no intention to use the terminal, the screen is controlled to remain in an off state.

[0058] In summary, in the embodiment of the present application, the terminal performs human body recognition on the images captured by the camera, and then predicts the use behavior of the human body recognition results of multiple consecutive frames of images, and controls the screen state according to whether the user has the intention to use the screen. Using the human body recognition results of multiple consecutive frames of images to predict the use behavior can avoid the error in predicting the user's use behavior when the user accidentally appears in front of the screen but has no intention to use the screen, and improves the accuracy of the prediction of the user's use behavior, so that the screen state can be controlled to be more in line with the user's intention, and the degree of intelligence is high.

[0059] In the application scenario, there are two possible terminal postures, one of which is a flat state and the other is a vertical state. The flat state means that the terminal is placed horizontally, and the vertical state means that the terminal is placed vertically.

[0060] In some scenarios, the terminal is not placed strictly horizontally or vertically. For example, the terminal is placed at an angle, that is, the screen of the terminal is facing a certain angle with the horizontal plane, and this angle is taken as the tilt angle. When the tilt angle is greater than the tilt angle threshold, the terminal is in a vertical state. When the tilt angle is less than the tilt angle threshold, the terminal is in a vertical state.

[0061] Optionally, the terminal can determine the terminal posture through a built-in gravity sensor. Schematically, the gravity sensor built into the terminal includes three parts: a sensor, a processor, and a controller. The gravity sensor determines the terminal posture by determining the magnitude of the component forces in two directions orthogonal to the gravity of the built-in weight.

[0062] Since the user behaviors that can be captured by the camera in different terminal postures are different, the strategies for judging user behaviors should also be different in different terminal postures.

[0063] First, the human body recognition strategy is determined based on the terminal posture, which includes horizontal and vertical states. Different terminal postures correspond to different human body recognition strategies. Then, human body recognition is performed on the images collected by the camera based on the human body recognition strategy to obtain human body recognition results.

[0064] In the application scenario, in the horizontal posture, when the user has no intention to use the terminal, it is difficult to recognize the human body in the continuous multiple frames of images taken by the camera. In the vertical state, it is very likely that the user is still active within the camera's shooting range when the user has no intention to use the terminal. Therefore, it is necessary to perform human body recognition on the head portrait collected by the camera according to different human body recognition strategies to obtain human body recognition results for use behavior prediction. The following will use two exemplary embodiments to respectively explain the strategies for performing human body recognition in different terminal postures.

[0065] Figure 3 A flowchart of a screen control process in a flat state provided by an exemplary embodiment of the present application is shown. The process includes:

[0066] Step 301, performing human feature recognition on an image captured by a camera to obtain a first human recognition result.

[0067] The first human body recognition result is used to indicate whether a human body exists in the image.

[0068] Human feature recognition mainly refers to the terminal's recognition of the inherent physiological features or human behavior features of the human body, wherein the inherent physiological features include fingerprints, irises, facial features, auricles, etc., and human behavior features include human body detection, posture estimation, etc., wherein the terminal can determine whether there is a human body in the image captured by the camera by performing human body detection, and through fingerprints, irises, facial features and posture estimation, it can be determined whether the human body features captured in the image match the target human body features pre-stored in the terminal. The first human body recognition result is also used to characterize whether there is a target human body in the image.

[0069] Step 302: when the first human body recognition result of the continuous multiple-frame images indicates the presence of a human body, determining the usage behavior prediction result as the presence of an intention to use the terminal.

[0070] Optionally, when the first human body recognition result of the continuous multiple frames of images indicates that there are more image frames with human bodies than image frames without human bodies, the usage behavior prediction result is determined as the presence of an intention to use the terminal.

[0071] In a possible implementation, there may be a situation where the first human body recognition results of multiple consecutive frames of images indicate the presence of a human body, but the human body is not looking at the screen. For example, a user is sitting in front of a table looking at other devices or things, and the terminal is placed flat on the table. At this time, the distance between the terminal and the user is relatively close, and a human body exists in the image taken by the terminal, but the user has no intention of using the terminal at this time.

[0072] In this case, the terminal can obtain the eye gaze result by performing eye gaze recognition on the continuous multiple frames of images captured, and the eye gaze result is used to characterize whether there is a human eye looking at the screen. When the first human body recognition result of the continuous multiple frames of images characterizes the existence of a human body, and the eye gaze result characterizes the existence of human eyes looking at the screen, the usage behavior prediction result is determined to be the existence of an intention to use the terminal. When the first human body recognition result of the continuous multiple frames of images characterizes the absence of a human body, and the eye gaze result characterizes the absence of human eyes looking at the screen, the usage behavior prediction result is determined to be the absence of an intention to use the terminal.

[0073] Step 303 : when the first human body recognition result of the continuous multiple-frame images indicates that there is no human body, determining the usage behavior prediction result as the absence of the intention to use the terminal.

[0074] Optionally, when the first human body recognition result of the continuous multiple frames of images indicates that the number of image frames in which the human body exists is less than the number of image frames in which the human body does not exist, the usage behavior prediction result is determined to be that there is no intention to use the terminal.

[0075] Step 304 : when the first human body recognition result of the continuous multiple-frame images indicates that there is no human body, the human body feature recognition frequency is reduced.

[0076] When the first human body recognition result of the continuous multi-frame image indicates that there is no human body, the terminal determines that the usage behavior prediction result is that there is no intention to use the terminal. In order to reduce power consumption, the terminal can reduce the frequency of the first human body feature recognition. In addition, the terminal can maintain a low recognition frequency until there is a human body recognition result of a frame image indicating that there is a human body, and the terminal can increase the human body feature recognition frequency again to identify whether there is a human body in the continuous multi-frame image.

[0077] For example, Figure 4A schematic diagram of switching the human feature recognition frequency provided by an exemplary embodiment of the present application is shown, wherein the terminal performs human feature recognition in two modes: high frequency mode and low power consumption mode. In high frequency mode, the terminal can perform human feature recognition on more image frames per unit time, but due to the large number of image frames recognized per unit time, the power consumption is large. In low power consumption mode, the terminal recognizes fewer image frames per unit time, so the power consumption is small. As shown in the figure, in high frequency mode, the frequency of human feature recognition by the terminal is 10 frames / second, and in low power consumption mode, the frequency of human feature recognition by the terminal is 5 frames / second. In the initial state, the terminal is in low power consumption mode (i.e., 5 frames / second) for human feature recognition. At this time, the frame rate captured by the camera is synchronized with the frequency of human feature recognition by the terminal, and the system resources occupied are small. If the first human body recognition result of a frame of image indicates that a human body exists, the terminal switches to high frequency mode for human body feature recognition. And when the first human body recognition result of multiple consecutive frames of images indicates that a human body exists, it is determined that the use behavior prediction result is the intention to use the terminal. If the first human body recognition result of multiple consecutive frames of images indicates that there is no human body, the human body feature recognition frequency is reduced and switched to low power consumption mode to reduce power consumption.

[0078] In an embodiment of the present application, when the terminal is in a flat state, the human body recognition results of multiple consecutive frame images are used to predict the usage behavior, which can more accurately determine whether the user has the intention to use the terminal. In addition, when the human body is not recognized in the multiple consecutive frame images, the frequency of human body feature recognition is reduced, which can achieve the purpose of reducing power consumption and is beneficial to saving system resources of the terminal.

[0079] Because when the terminal is in a vertical position, the user may not intend to use the terminal, but the terminal still detects the presence of a human body in the captured image. Therefore, when the terminal performs human body recognition on the image captured by the camera, it is necessary to perform human body posture recognition on the image captured by the camera to obtain a second human body recognition result. The second human body recognition result is used to characterize the human body posture of the human body in the image.

[0080] Since the terminal is in a vertical position, the user may be far away from the terminal. At this time, the user has no intention to use the terminal, but the camera will still capture the human body. Therefore, the terminal performs human body proportion recognition on the image captured by the camera to obtain a third-party human body recognition result. The third-party human body recognition result is used to characterize the proportion of the human body in the image, so as to determine the user's distance from the terminal and predict usage behavior.

[0081] It is understandable that the closer the human body is to the terminal, the larger the proportion of the human body in the image captured by the camera, and the farther the human body is from the terminal, the smaller the proportion of the human body in the image captured by the camera.

[0082] Indicative, Figure 5 An example diagram of the proportion of human body in an image provided by an exemplary embodiment of the present application is shown, wherein the distances d1, d2, and d3 are the straight-line distances between a human body 501 and a terminal 502, the terminal is in a vertical state, and d1>d2>d3, and the first image 503, the second image 504, and the third image 505 are images captured by the camera at the distances d1, d2, and d3, respectively. When the user is at a distance d1 from the terminal, in the first image 503 captured by the camera, the proportion of human body is the smallest because the human body is farthest from the terminal; when the user is at a distance d3 from the terminal, in the third image 505 captured by the camera, the proportion of human body is the largest because the human body is closest to the terminal; when the user is at a distance d2 from the terminal, compared to the distance d1, the user is closer to the terminal, and the proportion of human body in the second image 504 captured by the camera is larger than that in the first image 503, and compared to the distance d3, the user is farther from the terminal, and the proportion of human body in the second image 504 captured by the camera is smaller than that in the third image 505.

[0083] When the third human body recognition result of multiple consecutive frames of images indicates that the proportion of the human body in the image does not decrease continuously and the proportion is greater than the proportion threshold, human body posture recognition is performed on the image captured by the camera to obtain a second human body recognition result.

[0084] The ratio threshold is a ratio set to trigger human gesture recognition of images captured by the camera. By setting the ratio threshold, human gesture recognition is triggered when the terminal is within a certain range of the user entering the terminal.

[0085] The third human body recognition result of multiple consecutive frames of images indicates that the proportion of the human body in the image has not continuously decreased, that is, the user has not moved away from the terminal, and if the proportion is greater than the proportion threshold, the distance between the user and the terminal is relatively close. At this time, the user is more likely to intend to use the terminal. Therefore, the terminal performs human posture recognition on the images captured by the camera to further predict the usage behavior through the second human body recognition result.

[0086] In the embodiment of the present application, human posture recognition is performed on multiple consecutive frames of images, which can improve the accuracy of human posture recognition and thus improve the accuracy of usage behavior prediction. On the other hand, it can avoid human posture recognition on images captured by the camera when the user is unlikely to have the intention to use the terminal, and human posture recognition is performed when the user is close to the terminal, which is beneficial to reduce the power consumption of the terminal for human body recognition while ensuring the accuracy of usage behavior prediction.

[0087] In a possible implementation, in order to further reduce the amount of computation required for human gesture recognition by the terminal and thus reduce power consumption, the terminal segments a portrait area from the image captured by the camera based on the third human gesture recognition result, and then performs human gesture recognition on the portrait area to obtain a second human gesture recognition result.

[0088] Optionally, the portrait area can be segmented out from the image captured by the camera by using image segmentation technology. Specifically, the portrait area can be segmented out from the image captured by the camera by using graph theory-based segmentation methods, such as normalized segmentation, graphic segmentation, etc., or clustering-based segmentation methods, such as Meanshift algorithm, etc., or other learning models and algorithms capable of image segmentation.

[0089] In a possible implementation, the current screen state is the screen-off state. When the third-party human body recognition results of multiple consecutive frames of images indicate that the proportion of the human body in the image continues to increase and the proportion is less than the proportion threshold, the foreground application before the screen is turned off will be woken up and the screen will remain in the off state.

[0090] Among them, the third-party human body recognition results of multiple consecutive frames of images indicate that the proportion of the human body in the image continues to increase, that is, the user is gradually approaching the terminal, but because the proportion is less than the proportion threshold, the user is far away from the terminal and cannot trigger the terminal to perform human posture recognition. In this case, since the user is gradually approaching the terminal, the user may have the intention to use the terminal. At this time, the terminal wakes up the foreground application before the screen is turned off. When the user gets closer to the terminal, that is, the proportion is greater than the proportion threshold, and the behavior prediction result indicates that the user has the need to use the terminal, the original foreground application process can be quickly restored, and the terminal can directly display the foreground application before the screen is turned off to the user.

[0091] When the prerequisites for human posture recognition are met, the terminal can identify the images captured by the camera in three dimensions: human posture recognition, eye gaze detection, and face position recognition, thereby predicting whether the user intends to use the terminal in different dimensions.

[0092] In a possible implementation, when the human body recognition results of the continuous multiple frames of images indicate that the human body posture is unstable, the terminal determines the human body movement trend based on the human body recognition results of the continuous multiple frames of images, wherein the human body movement trend is used to indicate the human body movement range.

[0093] In actual applications, the angle of the image that a camera can capture is usually determined by the camera. However, when a user is using a terminal, the angle is usually not greatly offset from the terminal screen orientation. If the angle between the line connecting the user and the terminal and the terminal screen orientation is large, the user's experience of using the terminal is poor. Usually, the user will avoid this situation when using the terminal. Therefore, the terminal can predict whether the user intends to use the terminal based on the range of human body motion.

[0094] When the human body movement trend indicates that the human body movement range is within the target range, the usage behavior prediction result is determined to be that there is an intention to use the terminal; when the human body movement trend indicates that the human body movement range is outside the target range, the usage behavior prediction result is determined to be that there is no intention to use the terminal.

[0095] The target range is the field of view of the target angle centered on the terminal orientation.

[0096] Indicatively, Figure 6 As shown, the field of view of the target angle α centered on the terminal 601 is the target range, and the field of view captured by the camera 602 is β, which is greater than α. When the user is at point A, the human body is within the target range, and the terminal determines that the use behavior prediction result is that there is an intention to use the terminal. When the user is at point B or point C, the human body is not within the target range, and the use behavior prediction result is determined to be that there is no intention to use the terminal. The area with a distance d from the screen in the figure can make the proportion of the human body in the image collected by the camera equal to the proportion threshold, that is, when the distance between the user and the screen is d, the proportion of the human body in the image is equal to the proportion threshold, and the terminal starts to perform human posture recognition. When the user is at point D, the proportion of the human body in the collected image is less than the proportion threshold, then the terminal will not perform human posture recognition at this time, so the use behavior prediction result is determined to be that there is no intention to use the terminal.

[0097] It can be understood that the target range can be a vertical field of view centered on the terminal orientation, which is similar to the above Figure 6 The form shown in is similar and will not be repeated again.

[0098] In a possible implementation, the human body recognition results of multiple consecutive frames of images indicate that the human body posture is stable. In order to further determine whether there is an intention to use the terminal, the terminal performs eye gaze detection on the images captured by the camera to obtain eye gaze detection results. And, when the eye gaze detection result indicates that there is a human eye gaze on the terminal screen, the use behavior prediction result is determined to be the intention to use the terminal; when the eye gaze detection result indicates that there is no human eye gaze on the terminal screen, the use behavior prediction result is determined to be the intention not to use the terminal. Among them, the eye gaze detection result is used to characterize whether there is a human eye gaze on the terminal screen.

[0099] In the application scenario, the user may be close to the terminal and within the camera shooting range, but the user is not looking at the screen, such as looking at other things or the user's eyes are not open. At this time, the user has no intention to use the terminal. Therefore, it is determined whether the user is looking at the screen by performing eye gaze detection. If the user is looking at the screen, it is determined that the user has the intention to use the terminal. Eye gaze detection can be performed by locating the center of the eyeball and analyzing the degree of closure of the eye area.

[0100] Optionally, while performing eye gaze detection, human gesture recognition can be suspended, which can reduce the energy consumption of human gesture recognition on images. In addition, if the eye gaze detection result indicates that there is no human eye gaze at the terminal screen, the use behavior prediction result is determined to be the absence of the intention to use the terminal, and the human gesture recognition is restarted; if the eye gaze detection result indicates that there is a human eye gaze at the terminal screen, the use behavior prediction result is determined to be the intention to use the terminal, the screen is controlled to be in a bright screen state, the eye gaze detection is stopped, and the human gesture recognition is restarted.

[0101] In a possible implementation, when it is determined that the human eye is looking at the screen, it can be determined that the user has the intention to use the terminal, and at this time, the terminal control screen is in a light-on state. When the human eye gaze detection result indicates that the human eye is looking at the terminal screen, human body posture recognition and human eye gaze detection are stopped, and face position recognition is performed to obtain a face position recognition result, wherein the face position recognition result is used to characterize the position of the face in the image, and the complexity of face position recognition is less than that of human body posture recognition and human eye gaze detection.

[0102] After determining that the user has a need to use the device, the screen needs to be controlled after the user no longer has a need to use the device. Therefore, when the face position recognition results of multiple consecutive frames of images indicate that the change in the face position in the image is greater than an amplitude threshold, human posture recognition is performed again.

[0103] The face position results of multiple consecutive frames of images represent that if the change in the position of the face in the image is greater than the amplitude threshold, there is a probability that the user has no intention to use the device. At this time, the terminal re-performs human posture recognition to ensure the accuracy of usage behavior prediction.

[0104] Indicative, Figure 7 A schematic diagram of a face position change provided by an exemplary embodiment of the present application is shown, wherein image 701, image 702, and image 703 are three consecutive frames of images captured by a camera, and in the process from image 701 to image 702, the face position in the image changes from the upper left corner to the right side, and the change amplitude is d1, and in the process from image 702 to image 703, the face position in the image changes from the right side to the lower left corner, and the change amplitude is d2. Assuming that the change amplitude threshold is D, when d1 and d2 are greater than the change amplitude threshold D, the terminal re-performs human posture recognition.

[0105] Optionally, when human body posture recognition is performed again, the terminal stops performing face position recognition, thereby avoiding unnecessary power consumption caused by performing face position recognition.

[0106] In the embodiment of the present application, since the complexity of face position recognition is lower than that of body posture recognition and eye gaze recognition, the power consumption of face position recognition is relatively small. Therefore, when it is determined that the eyes are looking at the screen, body posture recognition and eye gaze detection are stopped, and face position detection is performed. This can reduce power consumption and save terminal computing resources while ensuring that the screen control meets the user's intention of using the terminal.

[0107] Figure 8 The flowchart of the screen control strategy provided by an exemplary embodiment of the present application is shown. First, the terminal will check its own posture, and the terminal adopts different human body recognition strategies in the horizontal state and the vertical state.

[0108] In the flat state, the terminal performs human feature recognition on the image captured by the camera. If there is a human body in the image and the first human body recognition result in multiple consecutive frames indicates the existence of a human body and the usage behavior prediction result is determined to be the intention to use the terminal, the terminal is controlled to be in a screen-on state; otherwise, the usage behavior prediction result is determined to be the absence of the intention to use the terminal, and the terminal is controlled to be in a screen-off state. If the first human body recognition result in multiple consecutive frames indicates the absence of a human body, the frequency of human body feature recognition is reduced, and the terminal is controlled to be in a screen-off state.

[0109] In the vertical state, the terminal first performs human posture recognition, and determines the human motion trajectory based on the continuous multi-frame human motion recognition results. When the human motion trajectory exceeds the target range, the usage behavior prediction result is determined to be that there is no intention to use the terminal. When the human motion trajectory is within the target range, it is further detected whether the human eye is looking at the screen. When the human eye gaze detection result indicates that there is no human eye looking at the terminal screen, the usage behavior prediction result is determined to be that there is no intention to use the terminal. When the human body movement trajectory is within the target range and the human eye is looking at the screen, the terminal further performs face position recognition. When the amplitude of the change in the position of the face for multiple consecutive frames is less than the amplitude threshold, it is determined that there is an intention to use the terminal, and the terminal controls the screen to be in a bright screen state. When the amplitude of the change in the position of the face for multiple consecutive frames is not less than the amplitude threshold, the terminal re-performs human posture recognition and re-determines whether there is an intention to use.

[0110] In the embodiment of the present application, the terminal user behavior is determined by human feature recognition in the horizontal state, and the energy consumption is relatively low. In the vertical state, the image is gradually and deeply analyzed from three dimensions: human posture recognition, human eye gaze detection, and face position recognition, so that the terminal can accurately predict the user's usage behavior. In addition, when performing face position recognition, human posture recognition and human eye gaze detection are stopped, which can reduce the power consumption of the terminal for screen control while ensuring the accuracy of the prediction of the user's usage behavior.

[0111] In a possible implementation, the terminal controls the screen to be in a bright screen state through the driver layer without passing through the application layer, and controls the screen to be in a dark screen state through the driver layer without passing through the application layer. In some embodiments, the AON module can run low-power processes in the background. The AON module is in the driver layer. When the screen control state is decided by the AON module, it can avoid controlling through the interface of the application layer, and directly control the screen at the driver layer. The purpose of screen control can be achieved without waking up certain processes in the application layer, effectively reducing the power consumption of the system in the terminal.

[0112] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0113] Please refer to Fig. 9 , which shows a structural block diagram of a screen control device provided by an embodiment of the present application. The device may include:

[0114] The recognition module 901 is used to perform human body recognition on the images collected by the camera to obtain human body recognition results;

[0115] A prediction module 902 is used to perform usage behavior prediction based on the human body recognition results of the continuous multiple frames of images to obtain a usage behavior prediction result, wherein the usage behavior prediction result is used to characterize whether there is an intention to use the terminal;

[0116] A control module 903, configured to control the screen to be in a bright screen state when the usage behavior prediction result indicates that there is an intention to use the terminal;

[0117] The control module 903 is further configured to control the screen to be in an off state when the usage behavior prediction result indicates that there is no intention to use the terminal.

[0118] Optionally, the identification module 901 is used to:

[0119] Determining a human body recognition strategy based on a terminal posture, wherein the terminal posture includes a horizontal state and a vertical state, and different terminal postures correspond to different human body recognition strategies;

[0120] Based on the human body recognition strategy, human body recognition is performed on the image captured by the camera to obtain the human body recognition result.

[0121] Optionally, the terminal posture is the flat state;

[0122] The recognition module 901 is used to perform human feature recognition on the image captured by the camera to obtain a first human recognition result, where the first human recognition result is used to indicate whether there is a human body in the image;

[0123] The prediction module 902 is used to:

[0124] When the first human body recognition result of the continuous multiple frames of images indicates the presence of a human body, determining the usage behavior prediction result as the presence of an intention to use the terminal;

[0125] When the first human body recognition result of the continuous multiple frames of images indicates that a human body does not exist, the usage behavior prediction result is determined to be that there is no intention to use the terminal.

[0126] Optionally, the control module 903 is further used to:

[0127] When the first human body recognition result of the continuous multiple frames of images indicates that a human body does not exist, the frequency of human body feature recognition is reduced.

[0128] Optionally, the terminal posture is the vertical state;

[0129] The recognition module 901 is used to perform human posture recognition on the image captured by the camera to obtain a second human posture recognition result, where the second human posture recognition result is used to characterize the human posture of the human body in the image.

[0130] Optionally, the prediction module 902 is used to:

[0131] When the human body recognition results of the continuous multiple frames of images indicate that the human body posture is unstable, determining the human body movement trend based on the human body recognition results of the continuous multiple frames of images;

[0132] In a case where the human body movement trend indicates that the human body movement range is within a target range, determining that the usage behavior prediction result is that there is an intention to use the terminal, the target range being a field of view range of a target angle centered on the terminal orientation;

[0133] In a case where the human body movement trend indicates that the human body movement range is outside the target range, it is determined that the usage behavior prediction result is that there is no intention to use the terminal.

[0134] Optionally, the prediction module 902 is further used for:

[0135] When the human body recognition results of the continuous multiple frames of images indicate that the human body posture is stable, performing human eye gaze detection on the images captured by the camera to obtain a human eye gaze detection result, wherein the human eye gaze detection result is used to indicate whether there is human eye gaze on the terminal screen;

[0136] In a case where the eye gaze detection result indicates that a person is gazing at a terminal screen, determining that the usage behavior prediction result is that there is an intention to use the terminal;

[0137] In a case where the human eye gaze detection result indicates that there is no human eye gaze at the terminal screen, the usage behavior prediction result is determined as no intention to use the terminal.

[0138] Optionally, the control module 903 is further used to:

[0139] When the eye gaze detection result indicates that a person is looking at the terminal screen, human posture recognition and eye gaze detection are stopped, and face position recognition is performed to obtain a face position recognition result, wherein the face position recognition result is used to characterize the position of the face in the image, and the complexity of face position recognition is less than that of human posture recognition and eye gaze detection;

[0140] When the face position recognition results of the continuous multiple frames of images indicate that the change amplitude of the face position in the image is greater than the amplitude threshold, human body posture recognition is performed again.

[0141] Optionally, the recognition module 901 is further used to perform human body proportion recognition on the image captured by the camera to obtain a third human body recognition result, where the third human body recognition result is used to characterize the proportion of the human body in the image;

[0142] The recognition module 901 is also used to perform human posture recognition on the image captured by the camera to obtain the second human recognition result when the third human body recognition result of multiple consecutive frames of images indicates that the proportion of the human body in the image has not continuously decreased and the proportion is greater than the proportion threshold.

[0143] Optionally, the identification module 901 is used to:

[0144] Segmenting a portrait area from the image captured by the camera based on the third human body recognition result;

[0145] Performing human body posture recognition on the portrait area to obtain the second human body recognition result.

[0146] Optionally, the control module 903 is further used to:

[0147] When the third human body recognition result of multiple consecutive frames of images indicates that the proportion of the human body in the image continues to increase and the proportion is less than the proportion threshold, the foreground application before the screen is turned off will be woken up and the screen will remain off.

[0148] Optionally, the control module 903 is used to:

[0149] Without going through the application layer, the screen is controlled to be in a bright state through the driver layer;

[0150] Without going through the application layer, the screen is controlled to be in the off state through the driver layer.

[0151] In the embodiment of the present application, the terminal performs human body recognition on the images captured by the camera, and then predicts the use behavior of the human body recognition results of the continuous multi-frame images, and controls the screen state according to whether the user has the intention to use the screen. The use behavior prediction using the human body recognition results of the continuous multi-frame images can avoid the wrong prediction of the user's use behavior when the user accidentally appears in front of the screen but has no intention to use the screen, and improves the accuracy of the prediction of the user's use behavior, so that the screen state can be controlled to be more in line with the user's intention, and the degree of intelligence is high.

[0152] Please refer to Fig.10 , which shows a block diagram of a terminal provided by an exemplary embodiment of the present application. The terminal 1000 can be implemented as the terminal in each of the above embodiments. The terminal 1000 may include one or more of the following components: a processor 1010 and a memory 1020.

[0153] The processor 1010 may include one or more processing cores. The processor 1010 uses various interfaces and lines to connect the various parts of the entire terminal 1000, and executes various functions and processes data of the terminal 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1020, and calling data stored in the memory 1020. Optionally, the processor 1010 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 1010 can integrate one or more combinations of central processing unit (Central Processing Unit, CPU), image processor (Graphics Processing Unit, GPU), neural network processor (Neural-network Processing Unit, NPU) and modem. Among them, the CPU mainly processes the operating system, user interface and application program; the GPU is responsible for rendering and drawing the content to be displayed on the touch display; the NPU is used to realize the artificial intelligence (Artificial Intelligence, AI) function; the modem is used to process wireless communication. It is understandable that the above-mentioned modem may not be integrated into the processor 1010, but may be implemented by a separate chip.

[0154] The memory 1020 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1020 includes a non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data (such as audio data, a phone book), etc., created according to the use of the terminal 1000.

[0155] In addition, those skilled in the art will appreciate that the structure of the terminal 1000 shown in the above drawings does not constitute a limitation on the terminal, and the terminal may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the terminal 1000 also includes a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a Bluetooth module, and other components, which will not be described in detail here.

[0156] An embodiment of the present application further provides a computer-readable storage medium, which stores at least one program code, and the program code is loaded and executed by a processor to implement the screen control method described in the above embodiments.

[0157] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the screen control method provided in various optional implementations of the above aspects.

[0158] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0159] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A screen control method, characterized in that: The method comprises: Perform human body recognition on the images collected by the camera to obtain human body recognition results; Performing usage behavior prediction based on the human body recognition results of the continuous multiple frames of images to obtain a usage behavior prediction result, wherein the usage behavior prediction result is used to characterize whether there is an intention to use the terminal; When the usage behavior prediction result indicates that there is an intention to use the terminal, controlling the screen to be in a bright screen state; When the usage behavior prediction result indicates that there is no intention to use the terminal, the screen is controlled to be in an off state.

2. The method according to claim 1, characterized in that The performing human body recognition on the image captured by the camera to obtain the human body recognition result includes: Determining a human body recognition strategy based on a terminal posture, wherein the terminal posture includes a horizontal state and a vertical state, and different terminal postures correspond to different human body recognition strategies; Based on the human body recognition strategy, human body recognition is performed on the image captured by the camera to obtain the human body recognition result.

3. The method according to claim 2, characterized in that The terminal posture is the flat state; The performing human body recognition on the image collected by the camera based on the human body recognition strategy to obtain the human body recognition result includes: Performing human feature recognition on the image captured by the camera to obtain a first human recognition result, where the first human recognition result is used to indicate whether there is a human body in the image; The method of performing usage behavior prediction based on the human body recognition result of the continuous multiple-frame images to obtain the usage behavior prediction result includes: When the first human body recognition result of the continuous multiple frames of images indicates the presence of a human body, determining the usage behavior prediction result as the presence of an intention to use the terminal; When the first human body recognition result of the continuous multiple frames of images indicates that a human body does not exist, the usage behavior prediction result is determined to be that there is no intention to use the terminal.

4. The method according to claim 3, characterized in that The method further comprises: When the first human body recognition result of the continuous multiple frames of images indicates that a human body does not exist, the frequency of human body feature recognition is reduced.

5. The method according to claim 2, characterized in that: The terminal posture is the vertical state; The performing human body recognition on the image collected by the camera based on the human body recognition strategy to obtain the human body recognition result includes: Human body posture recognition is performed on the image captured by the camera to obtain a second human body recognition result, where the second human body recognition result is used to characterize the human body posture of the human body in the image.

6. The method according to claim 5, characterized in that The method of performing usage behavior prediction based on the human body recognition result of the continuous multiple-frame images to obtain the usage behavior prediction result includes: When the human body recognition results of the continuous multiple frames of images indicate that the human body posture is unstable, determining the human body movement trend based on the human body recognition results of the continuous multiple frames of images; In a case where the human body movement trend indicates that the human body movement range is within a target range, determining that the usage behavior prediction result is that there is an intention to use the terminal, the target range being a field of view range of a target angle centered on the terminal orientation; In a case where the human body movement trend indicates that the human body movement range is outside the target range, it is determined that the usage behavior prediction result is that there is no intention to use the terminal.

7. The method according to claim 6, characterized in that The performing usage behavior prediction based on the human body recognition result of the continuous multiple frames of images to obtain the usage behavior prediction result also includes: When the human body recognition results of the continuous multiple frames of images indicate that the human body posture is stable, performing human eye gaze detection on the images captured by the camera to obtain a human eye gaze detection result, wherein the human eye gaze detection result is used to indicate whether there is human eye gaze on the terminal screen; In a case where the eye gaze detection result indicates that a person is gazing at a terminal screen, determining that the usage behavior prediction result is that there is an intention to use the terminal; In a case where the human eye gaze detection result indicates that there is no human eye gaze at the terminal screen, the usage behavior prediction result is determined as no intention to use the terminal.

8. The method according to claim 7, characterized in that The method further comprises: When the eye gaze detection result indicates that a person is looking at the terminal screen, human posture recognition and eye gaze detection are stopped, and face position recognition is performed to obtain a face position recognition result, wherein the face position recognition result is used to characterize the position of the face in the image, and the complexity of face position recognition is less than that of human posture recognition and eye gaze detection; When the face position recognition results of the continuous multiple frames of images indicate that the change amplitude of the face position in the image is greater than the amplitude threshold, human body posture recognition is performed again.

9. The method according to claim 5, characterized in that Before performing human posture recognition on the image captured by the camera to obtain a second human recognition result, the method further includes: Performing human body proportion recognition on the image captured by the camera to obtain a third human body recognition result, wherein the third human body recognition result is used to characterize the proportion of the human body in the image; The performing human posture recognition on the image captured by the camera to obtain a second human body recognition result includes: When the third human body recognition result of multiple consecutive frames of images indicates that the proportion of the human body in the image does not decrease continuously and the proportion is greater than the proportion threshold, human body posture recognition is performed on the image captured by the camera to obtain the second human body recognition result.

10. The method according to claim 9, characterized in that The performing human posture recognition on the image captured by the camera to obtain a second human body recognition result includes: Segmenting a portrait area from the image captured by the camera based on the third human body recognition result; Performing human body posture recognition on the portrait area to obtain the second human body recognition result.

11. The method according to claim 9, characterized in that The terminal is currently in a screen-off state, and after performing human body proportion recognition on the image captured by the camera and obtaining a third human body recognition result, the method further includes: When the third human body recognition result of multiple consecutive frames of images indicates that the proportion of the human body in the image continues to increase and the proportion is less than the proportion threshold, the foreground application before the screen is turned off will be woken up and the screen will remain off.

12. The method according to any one of claims 1 to 11, characterized in that: The control screen is in a bright screen state, including: Without going through the application layer, the screen is controlled to be in a bright state through the driver layer; The control screen is in a screen-off state, including: Without going through the application layer, the screen is controlled to be in the off state through the driver layer.

13. A screen control device, characterized in that: The device comprises: The recognition module is used to perform human body recognition on the images collected by the camera to obtain human body recognition results; A prediction module, used to predict the use behavior based on the human body recognition results of the continuous multiple frames of images to obtain a use behavior prediction result, wherein the use behavior prediction result is used to characterize whether there is an intention to use the terminal; A control module, configured to control the screen to be in a bright screen state when the usage behavior prediction result indicates that there is an intention to use the terminal; The control module is further configured to control the screen to be in an off state when the usage behavior prediction result indicates that there is no intention to use the terminal.

14. A terminal, characterized in that: The terminal includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the screen control method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: At least one program code is stored in the computer-readable storage medium, and the program code is loaded and executed by the processor to implement the screen control method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the screen control method as described in any one of claims 1 to 12.