Screen display system and method based on scene self-adaption

Abstract

The invention belongs to the technical field of intelligent display, provides a screen display system and method based on scene self-adaption, and aims to solve the problem of low screen display control accuracy of intelligent equipment in the prior art. Environment information corresponding to the surrounding environment of the preset screen is collected, whether the environment information shows that the surrounding environment is in an environment idle scene or not is judged, if yes, the current time and the operation time corresponding to the final operation of the user on the preset screen are determined, and then the time interval between the operation time and the current time and the current interaction state are determined; the method comprises the steps of obtaining a time interval of a user, obtaining preset historical behavior information data of the user, judging whether the user has a historical interaction behavior habit similar to a current interaction state or not according to the time interval and the preset historical behavior information data, and if yes, maintaining a preset screen in a lighted state, so that the control flexibility and accuracy of screen display can be improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent display technology, and in particular to a scene-adaptive screen display system and method. Background Art

[0002] With the development of smart devices, the screen display of smart devices has received more and more attention. In order to provide users with a better user experience, the relevant parties of smart devices have been studying how to achieve intelligent and automatic display of smart device screens, so that the screen display of smart devices can more accurately adapt to the different needs of different users.

[0003] In traditional technology, the screen display is generally controlled by setting a fixed-time lock screen mode, detecting the ambient light of the smart device, or monitoring the screen status. For example, according to the user's settings, the screen is locked after 30 seconds or 5 minutes, or the screen brightness is automatically adjusted accordingly as the brightness of the light around the smart device changes, or the screen is turned off when it is detected that there is no operation on the screen, so as to predict user needs and control the automatic display of the screen.

[0004] However, the inventors realized that in traditional technologies, due to the complexity, variability and different interference of the surrounding environment of different users' smart devices, the user's conditions, etc., the screen control corresponding to the screen display based on the above-mentioned different methods is often inconsistent with the user's current actual needs, resulting in low control accuracy of the screen display. Not only does it not bring convenience to the user, but there is a risk of interference with the user's use, which brings a bad experience to the user.

[0005] Therefore, how to improve the control accuracy of screen display has become an urgent problem to be solved in the field of smart device display. Summary of the Invention

[0006] The technical problem solved by the present invention is to solve the problem of low screen display control accuracy of smart devices in traditional technologies.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: monitor whether the preset screen is in a dynamic operating state, and if not, collect environmental information corresponding to the surrounding environment of the preset screen; judge whether the environmental information indicates that the surrounding environment is in an idle scene, and if so, determine the operation time corresponding to the current time and the user's last operation on the preset screen; determine the time interval between the operation time and the current time and the current interaction state; obtain the user's preset historical behavior information data; based on the time interval and the preset historical behavior information data, judge whether the user has a historical interaction behavior habit similar to the current interaction state; if the above judgment is yes, maintain the preset screen in a lit state.

[0008] As a preferred solution of the scene-adaptive screen display method described in the present invention, judging whether the user has historical interaction behavior habits similar to the current interaction state based on the time interval and the preset historical behavior information data, includes: determining several historical time intervals corresponding to the preset historical behavior information data; determining the maximum value among all the historical time intervals; determining the historical time interval interval based on the maximum value; judging whether the time interval is distributed in the historical time interval interval; and if the time interval is distributed in the historical time interval interval, judging that the user has historical interaction behavior habits similar to the current interaction state.

[0009] The present invention also provides a screen display system based on scene adaptation, including: a first monitoring module, used to monitor whether the preset screen is in a dynamic operating state, and if not, collect environmental information corresponding to the surrounding environment of the preset screen; a first judgment module, used to judge whether the environmental information indicates that the surrounding environment is in an idle scene, and if so, determine the operation time corresponding to the current time and the user's last operation on the preset screen; a first determination module, used to determine the time interval between the operation time and the current time and the current interaction state; a first acquisition module, used to obtain the user's preset historical behavior information data; a second judgment module, used to judge whether the user has a historical interaction behavior habit similar to the current interaction state based on the time interval and the preset historical behavior information data; a lighting module, used to maintain the preset screen in a lit state if the above judgment is yes.

[0010] The beneficial effects of the present invention are as follows: by combining different dimensional information such as the current operating status of the preset screen, the surrounding environment, the current interaction status corresponding to the user's current operation behavior information, and the user's historical interaction behavior habits, the interaction behavior and interaction status between the smart device corresponding to the preset screen and the user, that is, the current scene corresponding to the smart device, are predicted and judged, and the screen display of the smart device is controlled accordingly to realize screen control under scene adaptation, which can better adapt to the actual screen display needs of different users in different scenes, realize personalized control and customized control of the display of the smart device screen to adapt to different users, and can improve the control flexibility and accuracy of the screen display, thereby improving the convenience of users operating the preset screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a flow chart of a scene-adaptive screen display method provided by an embodiment of the present invention; Figure 2A schematic diagram of the overall process of a scene-adaptive screen display method provided by an embodiment of the present invention; Figure 3 A schematic diagram of the first sub-process of the scene-adaptive screen display method provided by an embodiment of the present invention; Figure 4 A schematic diagram of a second sub-process of the scene-adaptive screen display method provided by an embodiment of the present invention; Figure 5 A schematic block diagram of a scene-adaptive screen display system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0013] Embodiments of the present invention provide a scene-adaptive screen display method and system. The method and system can be applied to devices including but not limited to smartphones, tablet computers, and vehicle-mounted center consoles, and can be used when performing scene-adaptive screen display.

[0014] In response to the technical problem of low screen display control accuracy of smart devices in traditional technologies, the inventors have proposed a scene-adaptive screen display method according to an embodiment of the present invention. The core concept of the embodiment of the present invention is to monitor the current operating status, surrounding environment, and current user operation behavior of the smart device, and combine the user's historical interaction behavior habits to determine the current scene corresponding to the smart device, and control the display state of the smart device screen according to the current scene. By combining information from different dimensions such as the surrounding environment information of the smart device, the current operating status of the smart device itself, and the user's current operation behavior information and corresponding historical interaction behavior habit information, the interaction behavior and interaction state between the smart device and the user can be accurately predicted and judged, that is, the current scene corresponding to the smart device. The screen display of the smart device is controlled accordingly, realizing scene-adaptive screen control, which can better adapt to the actual screen display needs of different users in different scenarios, realize personalized and customized control of the smart device screen display to adapt to different users, and improve the control flexibility and accuracy of the screen display.

[0015] The present invention is described in detail below through specific examples.

[0016] Example 1, please refer to Figure 1 and Figure 2 , Figure 1A schematic diagram of a flow chart of a scene-adaptive screen display method provided by an embodiment of the present invention. Figure 2 The overall flow diagram of the scene-adaptive screen display method provided by the embodiment of the present invention is as follows. Figure 1 As shown, in this embodiment, the method includes but is not limited to the following steps S101-S111: S101: Monitor whether the preset screen is in a dynamic operation state.

[0017] Explanatory, in response to the screen display control instructions of the smart device, such as the startup screen display control of a smartphone, or the startup screen display control of a car center console, since the preset screen is in a dynamic operation, it means that the user is using the preset screen. Therefore, the preset screen is generally in a lit state. Therefore, when the preset screen of the smart device is in a lit state, it is monitored whether the preset screen is in a dynamic operation state. The dynamic operation state describes the operation state of the preset screen dynamically executing a certain task. The dynamic operation state includes but is not limited to the operation state of dynamic processing corresponding to audio playback, video playback, and game play.

[0018] S102, when the preset screen is in a dynamic operation state, maintaining the preset screen in a lighting state; S103 : When the preset screen is not in a dynamic operation state, collect environmental information corresponding to the surrounding environment of the preset screen.

[0019] Explanatoryally, when the preset screen is in a dynamic operation state, it indicates that the user is using the preset screen. At this time, the preset screen is maintained in a lit state to facilitate the user to perform corresponding operations.

[0020] When the preset screen is not in a dynamic operating state, the preset screen is in a blank screen state or a static operating state, wherein the blank screen state indicates that the preset screen has been started but no application is open, such as the state of just turning on the computer, that is, the state of closing all applications; the static operating state describes the operating state of the preset screen where a certain application has been opened but is in a static display. The static operating state includes but is not limited to the static state corresponding to audio playback pause or suspension, video playback pause or suspension, web page opening, application opening, and WORD opening. In this case, the environmental information corresponding to the surrounding environment of the preset screen is collected. The environmental information includes but is not limited to the audio information, image information, and light corresponding to the surrounding environment. The audio information includes but is not limited to voice. The image information includes but is not limited to the image corresponding to the face or portrait under visible light and the image corresponding to the portrait under infrared light.

[0021] S104, determining whether the environmental information indicates that the surrounding environment is in an idle environment scene; S105: When the environmental information does not indicate that the surrounding environment is in an idle scene, turn off the preset screen.

[0022] Explanatoryly, based on the audio or portrait corresponding to the environmental information, it is identified whether there are other people in the surrounding environment, that is, whether the surrounding environment is in an idle scene. The idle scene indicates that the surrounding environment is an idle scene with no sound and no people. Based on this, it is judged whether the surrounding environment is safe, thereby identifying whether there is a risk of leakage of the display content of the preset screen in the surrounding environment. Moreover, when the environmental information does not indicate that the surrounding environment is in an idle scene, that is, when the environmental information indicates that the surrounding environment is not in an idle scene, that is, when the environmental information indicates that there are other people in the surrounding environment, it indicates that there is a risk of leakage of the display content of the preset screen, resulting in insecurity, and the preset screen is turned off, thereby ensuring the privacy and security of the display content of the preset screen. Among them, screen off means controlling the preset screen to enter low-power sleep or always-on display function (such as AOD, standby). Screen off includes but is not limited to completely turning off the screen, always-on display (AOD), sleep / standby (Sleep Mode), software-level screen off (only black screen), and reduced screen brightness. Among them, completely turning off the screen means that the screen backlight / pixels are completely powered off and there is no display (such as a pure black OLED screen). The device may still be running in the background (such as a mobile phone lock screen); screen off means that the main screen is off, but basic information (time, notifications, etc.) is partially displayed, and low-power operation is performed (such as an OLED only lights up some pixels); sleep / standby means that the screen is turned off, but the device maintains low-power operation (such as the memory is not cleared) and can be quickly awakened; software-level screen off means that the screen displays a pure black picture, but the backlight / pixels are still powered (the LCD screen may leak light), which is common in software simulation of the "off" effect.

[0023] S106: When the environment information indicates that the surrounding environment is in an idle scene, determine an operation time corresponding to the current time and the last operation of the user on the preset screen.

[0024] Explanatoryally, when the environment information indicates that the surrounding environment is in an idle scene, the current time corresponding to the preset screen is determined, and the operation time corresponding to the user's last operation on the preset screen is determined.

[0025] S107: Determine the time interval between the operation time and the current time and the current interaction state.

[0026] Explanatory, as described above, when the environmental information indicates that the surrounding environment is in an idle scene, that is, when the environmental information indicates that there are no other people in the surrounding environment, it indicates that there is no risk of leakage of the display content of the preset screen causing insecurity. In this case, the time interval between the operation time corresponding to the user's last operation on the preset screen and the current time is calculated. The time interval (Time Interval) refers to the duration length between the operation time corresponding to the above-mentioned last operation and the current time, usually expressed in fixed units (such as seconds, minutes, etc.) to measure the time span.

[0027] The time interval reflects how long there has been no interaction between the user and the preset screen, which can be called the current interaction state. The current interaction state can also be called the user's current interaction behavior or current behavior. The above names all represent the user's operation and interaction on the preset screen. The current interaction state can be expressed based on the above time interval. For example, the current interaction state is expressed as "no interaction for 5 minutes", where 5 minutes represents the above time interval. At the same time, the current interaction state can also be called the user's interaction behavior. Regular interaction behavior implies the user's behavioral habits or behavioral patterns. For example, under regular behavioral habits, the user leaves the display screen due to work reports, tea breaks, etc., resulting in a regular period of non-interaction between the user and the preset screen, which implies the user's regular behavior or habits. Therefore, the time interval can reflect the user's behavioral habits or behavioral patterns to a certain extent.

[0028] S108: Obtaining preset historical behavior information data of the user; S109: Determine, based on the time interval and the preset historical behavior information data, whether the user has a historical interaction behavior habit similar to the current interaction state.

[0029] Explanatory, the user's preset historical behavior information data is determined. The preset historical behavior information data represents the behavior data corresponding to the user's past operations on the preset screen. The preset historical behavior information data includes but is not limited to several historical time intervals, historical interaction states, and historical interaction behavior habits. The historical time interval represents the historical duration length between two adjacent historical operations in the user's past historical behavior. Similarly, the preset historical behavior information data reflects the historical behavior habits or historical behavior patterns corresponding to the user's past operations on the preset screen. The historical time interval generally represents the historical duration length greater than or equal to the preset time length threshold, so as to filter out the time interval generated by the user's normal use of the preset screen, and only retain the historical time interval with a longer historical duration length between two adjacent historical operations that is sufficient to cause screen display control. The historical time interval can better reflect the user's historical behavior habits, thereby making the screen display control more in line with the behavior habits caused by the user's life, work, etc. behavior habits. Among them, the preset time length threshold can be an empirical value, or it can be obtained through statistics based on the user's historical behavior information. As mentioned above, the historical interaction state can also be expressed based on historical time intervals. The historical interaction state can also be called historical interaction behavior or historical behavior. The above names all represent the user's historical operations and interactions on the preset screen. Several identical, similar or similar historical interaction states form a historical interaction behavior habit.

[0030] Based on the above description and conception, according to the time interval and the user's preset historical behavior information data, it is determined whether the user has a historical interaction behavior habit similar to the current interaction state. The historical interaction behavior habit similar to the current interaction state represents several historical interaction states that are the same or close to the current interaction state. Among them, several historical interaction states represent historical interaction behavior habits. The historical interaction states and the current interaction state are different only in the limited interaction state time, and other relevant meanings or contents are the same. By comparing the time interval corresponding to the current interaction state with the historical time interval, it is determined whether the user has a historical interaction behavior habit similar to the current interaction state. The judgment can be made from the following two aspects: 1) Determine whether there is a historical time interval that is the same, similar, or close to the time interval. If so, it is determined that the user has a historical interaction behavior habit similar to the current interaction state; 2) Determine the number of historical time intervals that are the same, similar, or close to the time interval. The number is positively correlated with the behavioral pattern or behavioral habit. The larger the number, the stronger and more obvious the behavioral pattern or behavioral habit corresponding to the time interval and the historical time interval. In addition, the above two aspects can be judged separately or in combination.

[0031] Among them, historical interaction behavior habits also refer to the user's historical behavior habits or historical behavior patterns, and generally also refer to the user's behavior habits or behavior patterns. For example, the user's work behavior habits caused by the nature and arrangement of the work, and the resulting use habits of the preset screen, for example, staff who often need to ask for instructions and report will often and habitually be interrupted from using the display screen due to the request and report, and the duration of this interruption also has corresponding regularity; the user's life behavior habits caused by living habits, physical health reasons, etc., for example, when it is time to prepare a meal, they will temporarily need to leave the display screen.

[0032] S110: If the user does not have a historical interaction behavior habit similar to the current interaction state, turning off the preset screen; S111. When the user has a historical interaction behavior habit similar to the current interaction state, maintain the preset screen in a lit state.

[0033] Explanatory note: if a user has similar historical interaction behavior habits, indicating that this may be a continuation of the user's historical behavior and historical habits, and the surrounding environment is in a safe space corresponding to an idle environment, in this case, the preset screen is maintained in a lit state to facilitate the user's continued use of the preset screen. For example, if a user is using a laptop or desktop computer, and the user has similar historical interaction behavior habits and is in a safe space, such as a solitary office or home, the laptop or desktop computer's display screen can be controlled to remain lit to facilitate the user's continued use of the aforementioned computer device, avoiding the tedious operations and inconvenience caused by repeated screen shutdown and startup.

[0034] Similarly, when the user does not have similar historical interactive behavior habits, it indicates that this is not a continuation of the user's historical behavior and historical habits. In this case, the user's next usage expectations cannot be predicted, and when the corresponding time interval is exceeded, the preset screen will be turned off to ensure reduced energy consumption and meaningless loss and waste of computer equipment. It can also ensure the privacy and security of the content displayed on the user's preset screen under certain circumstances.

[0035] In summary, combining the user's historical interaction behavior habits or historical behavior patterns to achieve adaptive control of the screen display in different scenarios, and realize personalized and customized control of the display of the smart device screen to adapt to different users, can improve the control flexibility and accuracy of the screen display, thereby improving the convenience of users operating the preset screen.

[0036] In an embodiment of the present invention, whether a preset screen is in a dynamic operating state is monitored. If not, environmental information corresponding to the surrounding environment of the preset screen is collected, and then it is determined whether the environmental information indicates that the surrounding environment is in an idle scene. If so, the operation time corresponding to the current time and the user's last operation on the preset screen is determined, and then the time interval between the operation time and the current time and the current interaction state are determined, and the user's preset historical behavior information data is obtained. Then, based on the time interval and the preset historical behavior information data, it is determined whether the user has a historical interaction behavior habit similar to the current interaction state. If the above judgment is yes, the preset screen is maintained in a lit state. If the above judgment is no, the preset screen is turned off to implement The current operating status of the preset screen, the surrounding environment, the current interaction status corresponding to the user's current operation behavior information, and the user's historical interaction behavior habits and other different dimensional information are now combined to predict and judge the interaction behavior and interaction status between the smart device corresponding to the preset screen and the user, that is, the current scene corresponding to the smart device. Based on this, the screen display of the smart device is controlled to achieve screen control under scene adaptation, which can better adapt to the actual screen display needs of different users in different scenes, and realize personalized control and customized control of the smart device screen to suit different users, which can improve the control flexibility and accuracy of the screen display, thereby improving the convenience of users operating the preset screen.

[0037] In one embodiment, see Figure 3 , Figure 3 This is a schematic diagram of the first sub-flow of the scene-adaptive screen display method provided by an embodiment of the present invention. Figure 3 As shown, in this embodiment, judging whether the user has a historical interaction behavior habit similar to the current interaction state based on the time interval and the preset historical behavior information data includes: S301, determining a number of historical time intervals corresponding to the preset historical behavior information data; S302: Determine the maximum value among all the historical time intervals; S303: Determine a historical time interval based on the maximum value; S304, determining whether the time interval is distributed within the historical time interval; S305: If the time interval is distributed within the historical time interval, determining that the user has a historical interaction behavior habit similar to the current interaction state; S306: If the time interval is not distributed within the historical time interval, determine that the user does not have a historical interaction behavior habit similar to the current interaction state.

[0038] Explanation: Based on the historical behavior information data of the user, several historical time intervals are determined. The historical time intervals are as described above, and the maximum value among all historical time intervals is determined. Then, based on the maximum value, the historical time interval interval is determined. The historical time interval interval represents the range of the length of the historical duration between two adjacent historical operations in the user's past historical behavior. The historical time interval interval includes but is not limited to the following [0, maximum value], [minimum value, maximum value], where the minimum value refers to the minimum value among all historical time intervals. Then, it is judged whether the time interval is distributed in the historical time interval interval. In the case that the time interval is distributed in the historical time interval interval, it indicates that the operation behavior corresponding to the user's last operation on the preset screen is very likely to be similar to the user's historical operation behavior, that is, it indicates that the user's behavior this time is very likely to be similar to the user's historical behavior, thereby determining that the user has a historical interaction state similar to the current interaction state. Interaction behavior habits. Similarly, when the time interval is not distributed in the historical time interval interval, it indicates that the possibility that the operation behavior corresponding to the user's last operation on the preset screen is similar to the user's historical operation behavior is relatively small, that is, it indicates that the possibility that the user's current behavior is similar to the user's historical behavior is relatively small, and thus it is determined that the user does not have a historical interaction behavior habit similar to the current interaction state, and then the preset screen is maintained in a lit state or turned off accordingly, to achieve a combination of the user's historical operation behavior and a consistent prediction of the corresponding implicit historical behavior, and based on this concept, the preset screen is displayed under corresponding scene adaptive control, which can make the screen display better adapt to the actual screen display needs of different users in different scenes, and make the screen display control more in line with the user's life, work and other behavioral habits caused by the operation behavior habits and the behavioral habits corresponding to their life and work.

[0039] In an embodiment of the present invention, by determining several historical time intervals corresponding to preset historical behavior information data, determining the maximum value among all the historical time intervals, and then determining the historical time interval interval based on the maximum value, and then determining whether the time intervals are distributed within the historical time interval interval, if the above judgment is yes, it is determined that the users have similar historical interaction behavior habits; if the above judgment is no, it is determined that the users do not have similar historical interaction behavior habits. This achieves the goal of combining the personalized historical interaction behavior habits (i.e., historical behavior habits) of different users to predict and determine the interaction behavior and interaction state between the smart device and the user corresponding to the preset screen, and then controlling the screen display of the smart device accordingly. This can make the screen display better adapt to the actual screen display needs of different users in different scenarios, make the screen display control more closely fit the user's operating behavior habits caused by behavioral habits such as life and work, and achieve personalized and customized control of the smart device screen display adapted to different users. This can improve the control flexibility and accuracy of the screen display, thereby improving the convenience of users operating the preset screen.

[0040] In one embodiment, see Figure 4 , Figure 4 The second sub-flow diagram of the scene-adaptive screen display method provided by the embodiment of the present invention. Figure 4 As shown, in this embodiment, determining the maximum value of all the historical time intervals includes: S401, clustering all the historical time intervals based on a preset time interval clustering model to obtain a number of clusters; S402, sorting all the clusters according to the order of the historical time intervals contained in the clusters from large to small to obtain a cluster sequence; S403: Counting the proportion of the number of the historical time intervals included in the cluster to all the historical time intervals according to the order of the cluster sequence from front to back; S404: When the ratio is greater than or equal to a preset ratio threshold, taking the maximum historical time interval included in the cluster as the maximum value of all the historical time intervals.

[0041] Explanatoryally, a time interval clustering model is pre-set, that is, a preset time interval clustering model. The preset time interval clustering model represents a model for clustering all historical time intervals. The preset time interval clustering model includes but is not limited to a K-Means algorithm and a DBSCAN algorithm.

[0042] According to the above conception and setting, based on the preset time interval clustering model, all historical time intervals are clustered to obtain several cluster clusters. For example, all historical time intervals are grouped according to similar time intervals, such as "short interval", "medium interval", and "long interval", and then the clustered clusters 1 (short interval), cluster 2 (medium interval), and cluster 3 (long interval) are obtained.

[0043] Then, all clusters are sorted in descending order according to the historical time intervals contained in the clusters to obtain a cluster sequence, such as {cluster 3, cluster 2, cluster 1} in the above example. Then, according to the order of the cluster sequence from front to back, that is, from cluster 3 to cluster 1, the proportion of the number of historical time intervals contained in a single cluster to all historical time intervals is cyclically counted. When the proportion is greater than or equal to the preset proportion threshold, it indicates that the historical time interval contained in the single cluster can reflect the user's historical behavior habits. The maximum historical time interval contained in the cluster is taken as the maximum value of all historical time intervals. When the proportion is less than the preset proportion threshold, it indicates that the historical time interval contained in the single cluster cannot reflect the user's historical behavior habits and may be just a special case in a certain situation. In this case, the proportion of the number of historical time intervals contained in the next cluster to all historical time intervals is counted until the proportion corresponding to a cluster contained in the cluster sequence is greater than or equal to the preset proportion threshold. This avoids extreme cases from being misjudged as the user's historical interaction behavior habits, thereby improving the accuracy of the maximum value, and can further make the screen display control more closely aligned with the actual operation behavior habits caused by the user's life, work, and other behavioral habits, without being interfered with and affected by extreme exceptions, thereby achieving personalized and customized control of the display of the smart device screen to adapt to different users, and can further improve the control flexibility and accuracy of the screen display.

[0044] In an embodiment of the present invention, all historical time intervals are clustered based on a preset time interval clustering model to obtain a number of cluster clusters, and all cluster clusters are sorted according to the order of the historical time intervals contained in the cluster clusters from large to small to obtain a cluster cluster sequence, and then according to the order of the cluster cluster sequence from front to back, the proportion of the number of historical time intervals contained in a single cluster in all historical time intervals is counted, and when the proportion is greater than or equal to a preset proportion threshold, the maximum historical time interval contained in the cluster cluster is taken as the maximum value of all historical time intervals, which can filter out the interference of the extreme historical operation behavior of the user contained in the historical time interval, so that the maximum value of all historical time intervals can reflect the historical interaction behavior habits, and all historical time intervals can be filtered out. The maximum value in the interval is consistent with the user's historical behavioral habits, and then combined with the personalized historical interaction behavioral habits of different users (i.e., historical behavioral habits) to predict and judge the interaction behavior and interaction state between the smart device corresponding to the preset screen and the user, and then control the screen display of the smart device accordingly, which can make the screen display better adapt to the actual needs of screen display of different users in different scenarios, and make the control of the screen display more in line with the operating behavior habits caused by the user's life, work, etc., without being interfered with and affected by extreme exceptions, and realize personalized control and customized control of the display of the smart device screen to different users, which can further improve the control flexibility and accuracy of the screen display, thereby improving the convenience of users operating the preset screen.

[0045] In one embodiment, determining the historical time interval based on the maximum value includes: determining a minimum value among all of said historical time intervals; The minimum value and the maximum value form a historical time interval.

[0046] Explanatory, the minimum value in all historical time intervals is determined, and the minimum value and the maximum value are combined to form a historical time interval interval. Since the minimum value and the maximum value both reflect the historical behavioral habits corresponding to the user's preset historical behavior information data, that is, the historical time interval interval reflects the user's historical behavioral habit interval. Based on this, combined with the personalized historical interaction behavior habits (that is, historical behavior habits) of different users, the interaction behavior and interaction status between the smart device corresponding to the preset screen and the user are predicted and judged, and the screen display of the smart device is controlled. This can make the screen display better adapt to the actual screen display needs of different users in different scenarios, and make the control of the screen display further more in line with the actual operation behavior habits caused by the user's life, work and other behavioral habits.

[0047] In an embodiment of the present invention, by determining several historical time intervals corresponding to preset historical behavior information data, determining the minimum and maximum values in all historical time intervals, and forming a historical time interval interval with the minimum and maximum values, it is then determined whether users have similar historical interaction behavior habits based on the historical time interval interval. Since both the minimum and maximum values reflect the historical behavior habits corresponding to the user's preset historical behavior information data, that is, the historical time interval interval reflects the user's historical behavior habit interval, based on this, the interaction behavior and interaction state between the smart device corresponding to the preset screen and the user are predicted and determined in combination with the personalized historical interaction behavior habits of different users, and the screen display of the smart device is controlled. This can make the screen display better adapt to the actual screen display needs of different users in different scenarios, make the screen display control further more in line with the user's operating behavior habits caused by behavioral habits such as life and work, realize personalized and customized control of the smart device screen display adapted to different users, improve the control flexibility and accuracy of the screen display, and thus improve the convenience of users operating the preset screen.

[0048] In one embodiment, collecting environmental information corresponding to the surrounding environment of the preset screen includes at least one of the following: Based on a preset audio collection device, collecting audio information corresponding to the surrounding environment of the preset screen; Based on a preset image acquisition device, image information corresponding to the surrounding environment of the preset screen is acquired.

[0049] Explanatoryally, the pre-set audio acquisition device is a preset audio acquisition device, which refers to a device for collecting audio in the surrounding environment of the preset screen. The preset audio acquisition device can be a part or component of the smart device corresponding to the preset screen. The preset audio acquisition device can include but is not limited to a microphone and a professional sound card (such as Focusrite Scarlett).

[0050] A pre-set image acquisition device, i.e., a preset image acquisition device, refers to a device for acquiring images of the surrounding environment of a preset screen. The preset image acquisition device may be a component or assembly of the smart device corresponding to the preset screen. The preset image acquisition device may include but is not limited to a camera, an industrial camera, or an image sensor, wherein the image sensor includes but is not limited to infrared, thermal imaging, and ToF (time of flight) cameras.

[0051] According to the above-mentioned conception and setting, based on the preset audio acquisition device, the audio information corresponding to the surrounding environment of the preset screen is collected to obtain the sound information corresponding to the surrounding environment, and based on the sound information, it is monitored whether the surrounding environment is in an idle environment scene. Moreover, based on the preset image acquisition device, the image information corresponding to the surrounding environment of the preset screen is collected to obtain the image information corresponding to the surrounding environment, and based on the image information, it is monitored whether the surrounding environment is in an idle environment scene. In this way, audio and images of different dimensions can be combined to monitor whether the surrounding environment is in an idle environment scene, which can improve the flexibility, accuracy and convenience of monitoring the surrounding environment.

[0052] Furthermore, based on a preset image acquisition device, image information corresponding to the surrounding environment of the preset screen is acquired, including at least one of the following: Based on a preset visible light image acquisition device, collecting image information of a person corresponding to the surrounding environment of the preset screen; Based on a preset non-visible light image acquisition device, human body information corresponding to the surrounding environment of the preset screen is acquired.

[0053] Specifically, a visible light image acquisition device is pre-set, that is, a preset visible light image acquisition device. The preset visible light image acquisition device refers to a device corresponding to collecting visible light-based images in the surrounding environment. The preset visible light image acquisition device includes but is not limited to a camera and an industrial camera.

[0054] A non-visible light image acquisition device is pre-set, that is, a preset non-visible light image acquisition device. The preset non-visible light image acquisition device refers to a device corresponding to collecting images based on non-visible light in the surrounding environment. The preset non-visible light image acquisition device includes but is not limited to infrared sensors, thermal imaging sensors, and ToF (time of flight) cameras.

[0055] According to the above-mentioned conception and setting, based on the preset visible light image acquisition device, the image information of the personnel corresponding to the surrounding environment of the preset screen is collected, and based on the image information of the personnel, whether the surrounding environment is in an idle scene is monitored. Similarly, based on the preset non-visible light image acquisition device, the human body information corresponding to the surrounding environment of the preset screen is collected, and based on the human body information, whether the surrounding environment is in an idle scene is monitored. In this way, images of different dimensions can be combined to monitor whether the surrounding environment is in an idle scene, which can improve the flexibility, accuracy and convenience of monitoring the surrounding environment.

[0056] In an embodiment of the present invention, audio information corresponding to the surrounding environment of a preset screen is collected based on a preset audio collection device, and / or image information corresponding to the surrounding environment of the preset screen is collected based on a preset image collection device, and the surrounding environment of the preset screen is predicted and judged based on the environmental information corresponding to the audio information and / or image information. Then, screen display control is achieved according to different surrounding environments and in combination with factors of other different dimensions. The preset screen can be displayed while ensuring the safety of the surrounding environment of the preset screen, that is, without leaking the privacy of the content displayed on the preset screen. If the surrounding environment of the preset screen is not safe enough, that is, there is a possibility of leaking the privacy of the content displayed on the preset screen, the preset screen can be turned off accordingly. Therefore, when it is predicted that the user will not use the preset screen, not only the privacy and security of the content displayed on the preset screen can be guaranteed, but also the power consumption of the preset screen can be reduced, and the energy consumption of the smart device corresponding to the preset screen can be reduced.

[0057] In one embodiment, determining whether the environmental information indicates that the surrounding environment is in an idle environment scene includes at least one of the following: According to the audio information and based on a preset human voice recognition model, if no human voice information is detected, determining that the surrounding environment is in an idle scene; According to the image information and based on a preset portrait recognition model, when no portrait is detected, it is determined that the surrounding environment is in an idle scene.

[0058] Explanatoryally, a pre-set human voice recognition model is a preset human voice recognition model. The preset human voice recognition model refers to a model used to recognize human sounds including but not limited to speech, snoring, singing, typing and other human-related sounds. The preset human voice recognition model includes but is not limited to support vector machine (SVM), random forest, CNN-based model, and RNN / Transformer-based model.

[0059] The pre-set portrait recognition model is a preset portrait recognition model. The preset portrait recognition model refers to an image model related to humans, such as a human portrait, face, and body, and the preset portrait recognition model includes but is not limited to a convolutional neural network (CNN) and a generative adversarial network (GAN).

[0060] According to the above conception and setting, based on the audio information and the preset human voice recognition model, it is detected whether the surrounding environment of the preset screen contains human voice. When no human voice information is detected, it generally means that no effective human voice input is detected or the human voice in the environment is lower than the preset sound threshold. It is determined that the surrounding environment does not contain human voice, and thus it is determined that the surrounding environment is in an idle environment scene. Otherwise, it is determined that the surrounding environment is not in an idle environment scene.

[0061] According to image information, such as images or videos taken with visible light, or according to the detection of infrared light, and based on a preset portrait recognition model, portrait detection is performed on the surrounding environment to determine whether there is a portrait in the surrounding environment. If no portrait is detected, it is determined that there are no relevant persons in the surrounding environment, and the surrounding environment is determined to be in an idle scene. Otherwise, it is determined that the surrounding environment is not in an idle scene.

[0062] In another example, based on audio information and a preset voice recognition model, it is detected whether the surrounding environment of the preset screen contains human voices, and based on image information and a preset portrait recognition model, portrait detection is performed on the surrounding environment. If no voice information and no portrait are detected, it is determined that no relevant person exists in the surrounding environment, and thus it is determined that the surrounding environment is in an idle scene. Otherwise, it is determined that the surrounding environment is not in an idle scene, which can improve the accuracy of determining that the surrounding environment is in an idle scene.

[0063] In an embodiment of the present invention, by determining the audio and / or image of the surrounding environment, the surrounding environment is monitored to determine whether it contains people, thereby identifying the surrounding environment of a preset screen, and thereby determining whether the surrounding environment is safe. In a case where the surrounding environment does not contain people, that is, when it is determined that the surrounding environment does not contain human sounds and / or human bodies, the surrounding environment is determined to be in an idle scene, indicating that the surrounding environment is safe, that is, when the privacy of the content displayed on the preset screen will not be leaked. In addition, the screen display is controlled by combining factors of other different dimensions, so that the preset screen is displayed while ensuring the safety of the surrounding environment of the preset screen. In a case where the surrounding environment of the preset screen is not safe enough, that is, when there is a possibility of leaking the privacy of the content displayed on the preset screen, the preset screen is accordingly turned off, thereby achieving adaptive consideration and control of the screen display based on environmental factors, thereby achieving personalized and customized control of the screen display of the smart device that is adapted to different users, which can further improve the control flexibility, accuracy and adaptability of the screen display, thereby improving the convenience of users operating the preset screen.

[0064] It should be noted that the scene-adaptive screen display method described in the above embodiments can recombine the technical features contained in different embodiments as needed to obtain a combined implementation plan, but all are within the scope of protection required by the present invention.

[0065] In one embodiment, a scene-adaptive screen display system is provided. The scene-adaptive screen display system corresponds to the scene-adaptive screen display method in the above embodiment. Figure 5 , Figure 5Schematic block diagram of a scene-adaptive screen display system provided by an embodiment of the present invention. Figure 5 As shown, the scene-adaptive screen display system 50 includes a first monitoring module 51, a first judgment module 52, a first determination module 53, a first acquisition module 54, a second judgment module 55 and a lighting module 56. The above functional modules are described in detail as follows: The first monitoring module 51 is used to monitor whether the preset screen is in a dynamic operating state, and if not, to collect environmental information corresponding to the surrounding environment of the preset screen; A first determination module 52 is configured to determine whether the environmental information indicates that the surrounding environment is in an idle state, and if so, determine an operation time corresponding to a current time and a last operation of the user on the preset screen; A first determining module 53 is configured to determine the time interval between the operation time and the current time and the current interaction state; A first acquisition module 54 is used to acquire preset historical behavior information data of the user; A second judgment module 55 is configured to judge whether the user has a historical interaction behavior habit similar to the current interaction state based on the time interval and the preset historical behavior information data; The lighting module 56 is configured to maintain the preset screen in a lighting state if the above determination is yes.

[0066] In one embodiment, the screen display system 50 further includes: The first screen-off module is configured to turn off the preset screen when the user does not have any historical interaction behavior habits similar to the current interaction state.

[0067] In one embodiment, the screen display system 50 further includes: The second screen-off module is configured to turn off the preset screen when the environmental information does not indicate that the surrounding environment is in an idle scene.

[0068] In one embodiment, the second determining module 55 includes: A first determining submodule is configured to determine a number of historical time intervals corresponding to the preset historical behavior information data; A second determining submodule is configured to determine a maximum value among all the historical time intervals; A third determining submodule is configured to determine a historical time interval based on the maximum value; A first judgment submodule is configured to judge whether the time interval is distributed within the historical time interval; The first determination submodule is configured to determine, when the time interval is distributed within the historical time interval, whether the user has a historical interaction behavior habit similar to the current interaction state.

[0069] In one embodiment, the second determining submodule includes: A clustering submodule, configured to cluster all the historical time intervals based on a preset time interval clustering model to obtain a plurality of clusters; A sorting submodule, configured to sort all the clusters according to the order of the historical time intervals contained in the clusters from large to small, to obtain a cluster sequence; a statistics submodule, configured to count the proportion of the number of the historical time intervals contained in the cluster to the proportion of all the historical time intervals according to the order of the cluster sequence from front to back; The fourth determining submodule is configured to, when the ratio is greater than or equal to a preset ratio threshold, take the maximum historical time interval included in the cluster as the maximum value of all the historical time intervals.

[0070] In one embodiment, the third determining submodule includes: a fifth determining submodule, configured to determine a minimum value among all the historical time intervals; The composition submodule is used to combine the minimum value and the maximum value into a historical time interval.

[0071] In one embodiment, the first monitoring module 51 includes at least one of the following: A first acquisition submodule is configured to acquire audio information corresponding to the surrounding environment of the preset screen based on a preset audio acquisition device; The second acquisition submodule is configured to acquire image information corresponding to the surrounding environment of the preset screen based on a preset image acquisition device.

[0072] In one embodiment, the second acquisition submodule includes at least one of the following: A third acquisition submodule is configured to acquire, based on a preset visible light image acquisition device, image information of a person corresponding to the surrounding environment of the preset screen; The fourth acquisition submodule is configured to acquire human body information corresponding to the surrounding environment of the preset screen based on a preset non-visible light image acquisition device.

[0073] In one embodiment, the first determination module 52 includes at least one of the following: A second determination submodule is configured to determine, based on the audio information and a preset human voice recognition model, that the surrounding environment is in an idle scene when no human voice information is detected; The third determination submodule is configured to determine, based on the image information and a preset portrait recognition model, that the surrounding environment is in an idle scene when no portrait is detected.

[0074] An embodiment of the present invention provides a screen display system based on scene adaptation, which monitors whether a preset screen is in a dynamic operating state. If not, it collects environmental information corresponding to the surrounding environment of the preset screen, and then determines whether the environmental information indicates that the surrounding environment is in an idle scene. If so, it determines the operation time corresponding to the current time and the user's last operation on the preset screen, and then determines the time interval between the operation time and the current time and the current interaction state, and obtains the user's preset historical behavior information data. Then, based on the time interval and the preset historical behavior information data, it determines whether the user has a historical interaction behavior habit similar to the current interaction state. If the above judgment is yes, the preset screen is maintained in a lit state. If the above judgment is no, the preset screen is turned off. Set the screen to be off, and combine different dimensions of information such as the current operating status of the preset screen, the surrounding environment, the current interaction status corresponding to the user's current operation behavior information, and the user's historical interaction behavior habits to predict and judge the interaction behavior and interaction status between the smart device corresponding to the preset screen and the user, that is, the current scene corresponding to the smart device, and control the screen display of the smart device accordingly to achieve scene-adaptive screen control, which can better adapt to the actual screen display needs of different users in different scenes, and realize personalized and customized control of the smart device screen display to suit different users, which can improve the control flexibility and accuracy of the screen display, thereby improving the convenience of users operating the preset screen.

[0075] For the specific definition of the scene-adaptive screen display system, please refer to the definition of the scene-adaptive screen display method above, which will not be repeated here. The various modules in the above-mentioned scene-adaptive screen display system can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0076] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0077] The relevant data collection in the embodiments of the present invention complies with the requirements of relevant laws and regulations, such as China's Personal Information Protection Law, GDPR (EU General Data Protection Regulation) or information security standards of other countries and regions.

[0078] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A screen display method based on scene adaptation, characterized in that: include: monitoring whether the preset screen is in a dynamic operating state, and if not, collecting environmental information corresponding to the surrounding environment of the preset screen; Determining whether the environmental information indicates that the surrounding environment is in an idle scene, and if so, determining an operation time corresponding to a current time and a last operation of the user on the preset screen; Determine the time interval between the operation time and the current time and the current interaction state; Obtaining preset historical behavior information data of the user; Determining whether the user has a historical interaction behavior habit similar to the current interaction state based on the time interval and the preset historical behavior information data; If the above judgment is yes, the preset screen is maintained in a lit state.

2. The scene-adaptive screen display method according to claim 1, wherein: The method further comprises: In a case where the user does not have a historical interaction behavior habit similar to the current interaction state, the preset screen is turned off.

3. The scene-adaptive screen display method according to claim 1, wherein: The method further comprises: When the environmental information does not indicate that the surrounding environment is in an idle scene, the preset screen is turned off.

4. The scene-adaptive screen display method according to any one of claims 1 to 3, wherein: Determining, based on the time interval and the preset historical behavior information data, whether the user has a historical interaction behavior habit similar to the current interaction state includes: Determining a number of historical time intervals corresponding to the preset historical behavior information data; determining a maximum value among all of said historical time intervals; Determining a historical time interval based on the maximum value; Determining whether the time interval is distributed within the historical time interval; When the time intervals are distributed within the historical time intervals, it is determined that the user has historical interaction behavior habits that are similar to the current interaction state.

5. The scene-adaptive screen display method according to claim 4, wherein: Determine the maximum value among all of the historical time intervals, including: Based on a preset time interval clustering model, clustering all the historical time intervals to obtain a number of clusters; Sorting all the clusters according to the order of the historical time intervals contained in the clusters from large to small to obtain a cluster sequence; According to the order of the cluster sequence from front to back, counting the proportion of the number of the historical time intervals contained in the cluster to all the historical time intervals; When the ratio is greater than or equal to a preset ratio threshold, the maximum historical time interval included in the cluster is taken as the maximum value of all the historical time intervals.

6. The scene-adaptive screen display method according to claim 4, wherein: Determining a historical time interval based on the maximum value includes: determining a minimum value among all of said historical time intervals; The minimum value and the maximum value form a historical time interval.

7. The scene-adaptive screen display method according to any one of claims 1 to 3, wherein: Collecting environmental information corresponding to the surrounding environment of the preset screen includes at least one of the following: Based on a preset audio collection device, collecting audio information corresponding to the surrounding environment of the preset screen; Based on a preset image acquisition device, image information corresponding to the surrounding environment of the preset screen is acquired.

8. The scene-adaptive screen display method according to claim 7, wherein: The method includes collecting image information corresponding to the surrounding environment of the preset screen based on a preset image acquisition device, including at least one of the following: Based on a preset visible light image acquisition device, collecting image information of a person corresponding to the surrounding environment of the preset screen; Based on a preset non-visible light image acquisition device, human body information corresponding to the surrounding environment of the preset screen is acquired.

9. The scene-adaptive screen display method according to claim 7, wherein: Determining whether the environmental information indicates that the surrounding environment is in an idle environment scenario includes at least one of the following: According to the audio information and based on a preset human voice recognition model, if no human voice information is detected, determining that the surrounding environment is in an idle scene; According to the image information and based on a preset portrait recognition model, when no portrait is detected, it is determined that the surrounding environment is in an idle scene.

10. A scene-adaptive screen display system, characterized in that: include: A first monitoring module is used to monitor whether the preset screen is in a dynamic operation state, and if not, to collect environmental information corresponding to the surrounding environment of the preset screen; A first determination module is configured to determine whether the environmental information indicates that the surrounding environment is in an idle scene, and if so, determine an operation time corresponding to a current time and a last operation of the user on the preset screen; A first determining module, configured to determine a time interval between the operation time and the current time and a current interaction state; A first acquisition module is used to obtain preset historical behavior information data of the user; A second judgment module is used to judge whether the user has a historical interaction behavior habit similar to the current interaction state based on the time interval and the preset historical behavior information data; The lighting module is used to maintain the preset screen in a lighting state if the above judgment is yes.

Images