Television application recommendation method and system based on multi-physiological signal fusion, television and storage medium

Abstract

The invention discloses a television application recommendation method and system based on multi-physiological signal fusion, a television and a storage medium, and the method comprises the steps: obtaining a physiological signal of a user collected by a remote controller, and generating a standardized physiological feature vector according to the physiological signal; acquiring behavior features, time features and scene features, and inputting the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end side calculation model for fusion calculation to obtain user identity data; obtaining a multi-modal feature and a user historical use record, constructing a user portrait according to the user identity data, the multi-modal feature and the user historical use record, and obtaining user portrait data; and performing application prediction recommendation according to the user portrait data, and displaying a recommended application when an active recommendation triggering condition is met. According to the method, personalized services are provided according to the characteristics of different users, and the prediction precision of the recommended content is improved, so that real non-inductive identification and active prediction are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent television and human-computer interaction technology, and particularly relates to a television application recommendation method and system based on multi-physiological signal fusion, a television and a computer readable storage medium. BACKGROUND

[0002] Current home televisions are usually shared by multiple people, and different family members have obvious differences in age, interests, use time period and viewing preferences. The existing television recommendation system mainly depends on single account or device use records, and cannot identify "who is the current user", resulting in generalized and lack of targeted recommendation results, and poor user experience. For example, parents mainly watch news or series in the evening, while children use educational or entertainment applications during the weekend.

[0003] The traditional system cannot automatically switch the recommended object, and the frequent manual login and account switching operation is cumbersome, which does not conform to the natural interaction habit of family shared television, and cannot realize true "unconscious identification and active prediction".

[0004] That is, in the family television system, the existing identity recognition method is cumbersome and inefficient, and cannot quickly and automatically identify the identity of the operator, resulting in the inability to provide personalized services according to the characteristics of different users. The existing technology mainly depends on simple historical data for user behavior prediction, resulting in low prediction accuracy of recommended content, lack of personalization and scene adaptability. The current television system recommendation service is mostly static recommendation, which cannot actively recommend according to the dynamic changes of user behavior and different states of the scene, and the user experience is insufficient.

[0005] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0006] The main purpose of the present application is to provide a television application recommendation method and system based on multi-physiological signal fusion, a television and a computer readable storage medium, which aims to solve the problem of the inability to provide personalized services according to the characteristics of different users and the low prediction accuracy of recommended content in the prior art.

[0007] To achieve the above purpose, the present application provides a television application recommendation method based on multi-physiological signal fusion, which comprises the following steps: Acquiring physiological signals of a user collected by a plurality of sensor units of a remote controller, and generating a standardized physiological feature vector according to the physiological signals; Acquiring behavior features, time features and scene features, inputting the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side calculation model for fusion calculation to obtain user identity data; acquire multi-modal features and user historical usage records, construct a user portrait according to the user identity data, the multi-modal features and the user historical usage records, and obtain user portrait data; perform application prediction recommendation according to the user portrait data, and display a recommended application when a proactive recommendation trigger condition is met.

[0008] Optionally, the television application recommendation method based on multi-physiological signal fusion, wherein the physiological signals include skin electric response data, skin temperature data, holding pressure data and bioimpedance data. The acquisition of the remote controller acquires physiological signals of a user through multiple sensor units, and generates a standardized physiological feature vector according to the physiological signals, specifically including: When the user holds the remote controller, skin electric response data reflecting differences in skin conductive characteristics of the individual is acquired through a skin point sensing unit, skin temperature data of the individual temperature is acquired through a temperature sensor, holding pressure data about hand shape, holding force and holding posture differences is acquired through a pressure array unit, and bioimpedance data about measured skin resistance and phase difference is acquired through a bioimpedance detection unit. When the data acquisition time lasts for a preset time, the average value, variance, change rate and spectral features are extracted according to the skin electric response data, the skin temperature data, the holding pressure data and the bioimpedance data, and a standardized physiological feature vector is generated.

[0009] Optionally, the television application recommendation method based on multi-physiological signal fusion, wherein the acquisition of the behavior features, time features and scene features, inputting the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side calculation model for fusion calculation to obtain user identity data, specifically including: The behavior features are acquired according to the key rhythm, focus moving speed and commonly used application distribution, the time features are acquired according to the use period, and the scene features are acquired according to the television state. The standardized physiological feature vector, the behavior features, the time features and the scene features are inputted into a lightweight end-side calculation model, and the lightweight end-side calculation model performs fusion calculation to identify the current operator identity to obtain user identity data.

[0010] Optionally, the television application recommendation method based on multi-physiological signal fusion, wherein the acquisition of the multi-modal features and the user historical usage records, the construction of a user portrait according to the user identity data, the multi-modal features and the user historical usage records, and the obtaining of user portrait data, specifically including: Combining features from multiple different types of data to obtain multi-modal features, obtaining user historical usage records according to history data of user operations and application usage on a television system; Constructing a user portrait according to the user identity data, the multi-modal features and the user historical usage records to generate user portrait data, wherein the user portrait includes behavior patterns, application usage preferences, time period regularities and scene adaptation features.

[0011] Optionally, the television application recommendation method based on multi-physiological signal fusion, wherein the application prediction recommendation according to the user portrait data displays recommended applications when a proactive recommendation trigger condition is met, and specifically includes: Obtaining time period features, user behavior features, scene state features, physiological signal features and application usage features according to the user portrait data and inputting them to a hierarchical fusion model, wherein the hierarchical fusion model includes a cycle layer and a context layer; The cycle layer calculates a long-term weight based on time regularities, the context layer calculates a short-term weight based on current operation states, and a comprehensive score Score(App) of an application is calculated according to the long-term weight and the short-term weight: Score(App) = α × S_cycle(App) + β × S_context(App); Wherein S_cycle(App) represents the long-term weight, S_context(App) represents the short-term weight, and α and β represent two dynamically changeable weight coefficients, α is used to control the influence degree of long-term habits, and β is used to control the influence degree of current behavior and current television state. The top three applications are obtained according to the comprehensive scores of different applications, and the recommended applications are displayed if the top three applications meet the proactive recommendation trigger condition.

[0012] Optionally, the television application recommendation method based on multi-physiological signal fusion, wherein the calculation of the long-term weight is: S_cycle(App) = λ × S_time + γ × S_similar + δ × S_holiday; Wherein S_time represents the weight of the time dimension, S_similar represents the compensation weight of similar time periods, S_holiday represents the holiday weight, λ represents an exponential decay factor for adjusting the weight of long-term data, γ represents a similar time period compensation factor for compensating the similarity between different time periods, and δ represents a holiday correction factor for adjusting the weight of special dates.

[0013] Optionally, the television application recommendation method based on multi-physiological signal fusion, wherein the active recommendation trigger condition is that the cumulative use time length of the current period is greater than a first time, the use time span is greater than a second time, and the current time is located within a third time before the start of the historical interval to a fourth time before the end.

[0014] In addition, to achieve the above object, the present application also provides a television application recommendation system based on multi-physiological signal fusion, wherein the television application recommendation system based on multi-physiological signal fusion comprises: a physiological signal acquisition module, configured to acquire physiological signals of a user collected by a plurality of sensor units of a remote controller, and generate a standardized physiological feature vector according to the physiological signals; a multi-modal fusion recognition module, configured to acquire behavior features, time features and scene features, input the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side calculation model for fusion calculation to obtain user identity data; a user portrait modeling module, configured to acquire multi-modal features and user historical use records, construct a user portrait according to the user identity data, the multi-modal features and the user historical use records, and obtain user portrait data; an application prediction recommendation module, configured to perform application prediction recommendation according to the user portrait data, and display recommended applications when an active recommendation trigger condition is met.

[0015] In addition, to achieve the above object, the present application also provides a television, wherein the television comprises a memory, a processor, and a television application recommendation program based on multi-physiological signal fusion stored on the memory and capable of running on the processor, and the television application recommendation program based on multi-physiological signal fusion implements the steps of the television application recommendation method based on multi-physiological signal fusion when executed by the processor.

[0016] In addition, to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a television application recommendation program based on multi-physiological signal fusion, and the television application recommendation program based on multi-physiological signal fusion implements the steps of the television application recommendation method based on multi-physiological signal fusion when executed by a processor.

[0017] In the present application, the physiological signals of the user are collected by a plurality of sensor units, and a standardized physiological feature vector is generated according to the physiological signals; the behavior feature, time feature and scene feature are obtained, and the standardized physiological feature vector, the behavior feature, the time feature and the scene feature are input into a lightweight end-side calculation model for fusion calculation to obtain user identity data; multi-modal features and user historical use records are obtained, and a user portrait is constructed according to the user identity data, the multi-modal features and the user historical use records, and user portrait data is obtained; application prediction recommendation is performed according to the user portrait data, and when the active recommendation trigger condition is met, the recommended application is displayed. The present application can automatically identify the user identity without login, and predict the application that the user may operate next, provide personalized services according to the characteristics of different users, improve the prediction accuracy of the recommended content, and thus realize real no-sensing recognition and active prediction. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flowchart of a preferred embodiment of the television application recommendation method based on multi-physiological signal fusion of the present application; Figure 2 is a schematic diagram of the application recommendation process completed by each module in the preferred embodiment of the television application recommendation method based on multi-physiological signal fusion of the present application; Figure 3 is a schematic diagram of data collection by the physiological signal collection module in the preferred embodiment of the television application recommendation method based on multi-physiological signal fusion of the present application; Figure 4 is a schematic diagram of data fusion calculation by the multi-modal fusion recognition module in the preferred embodiment of the television application recommendation method based on multi-physiological signal fusion of the present application; Figure 5 is a schematic diagram of user portrait construction by the user portrait modeling module in the preferred embodiment of the television application recommendation method based on multi-physiological signal fusion of the present application; Figure 6 is a schematic diagram of application recommendation by the application prediction recommendation module in the preferred embodiment of the television application recommendation method based on multi-physiological signal fusion of the present application; Figure 7 is a structural diagram of a preferred embodiment of the television application recommendation system based on multi-physiological signal fusion of the present application; Figure 8 is a structural diagram of a preferred embodiment of the television of the present application. DETAILED DESCRIPTION

[0019] In order to make the objects, technical solutions and advantages of the present application clearer and more apparent, the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not to limit the present application.

[0020] The television application recommendation method based on multi-physiological signal fusion according to the preferred embodiment of the present application, as shown in Figure 1 and Figure 2 includes the following steps: Step S10, acquiring physiological signals of a user collected by a plurality of sensor units of a remote controller, and generating a standardized physiological feature vector according to the physiological signals.

[0021] Specifically, as shown in Figure 2 Step S10 is completed by a physiological signal acquisition module in the remote controller, the remote controller collects physiological signals of a user through a plurality of sensor units, and the physiological signals include skin galvanic response data, skin temperature data, holding pressure data and bioimpedance data; skin galvanic response: detecting skin electrical activity through conductivity changes, reflecting individual skin conductivity characteristic differences. Skin temperature: measured by thermal or infrared sensors, individual temperature distribution is stable, and can be used to distinguish users. Holding pressure distribution: pressure array sensor collects hand shape, grip strength and grip posture differences. Bioimpedance: micro-current measures skin resistance and phase difference, which is used to distinguish characteristics such as body fat rate and moisture content.

[0022] As shown in Figure 3 When the user holds the remote controller, the skin galvanic response data reflecting individual skin conductivity characteristic differences is collected through a skin point sensing unit, the skin temperature data of individual temperature is collected through a temperature sensor, the holding pressure data about hand shape, grip strength and grip posture differences is collected through a pressure array unit, and the bioimpedance data about measuring skin resistance and phase difference is collected through a bioimpedance detection unit; after the data collection time lasts for a preset time (for example, 3-5 seconds), the physiological signal acquisition module extracts average value, variance, change rate and spectral features and other parameters according to the skin galvanic response data, the skin temperature data, the holding pressure data and the bioimpedance data, generates a standardized physiological feature vector, and sends the standardized physiological feature vector to a multi-modal fusion recognition module of a television set.

[0023] Step S20, acquiring behavior features, time features and scene features, inputting the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side computing model for fusion calculation to obtain user identity data.

[0024] Specifically, as shown in Figure 2As shown, step S20 is completed by a multi-modal fusion recognition module in the television, which acquires behavior features, time features, and scene features. The behavior features are: key rhythm, focus moving speed, and common application distribution. The time features are: time period (morning / night / weekend / holiday). The scene features are: television state (power on, playing, pause, voice interaction). Figure 4 As shown, the behavior features are collected according to the key rhythm, focus moving speed, and common application distribution. The time features are collected according to the time period. The scene features are collected according to the television state. The standardized physiological feature vector, the behavior features, the time features, and the scene features are input into a lightweight end-side computing model. The lightweight end-side computing model performs fusion calculation to identify the current operator identity, obtains user identity data, and sends the user identity data to a user portrait modeling module of the television.

[0025] Step S30: Acquire multi-modal features and user historical use records. Construct a user portrait according to the user identity data, the multi-modal features, and the user historical use records, and obtain user portrait data.

[0026] Specifically, as shown in Figure 2 Step S30 is completed by a user portrait modeling module in the television. Multi-modal features refer to a combination of features from multiple different types of data, which are independently collected by different modules, different hardware, and different data sources. That is, multi-modal features are obtained by combining features from multiple different types of data. User historical use records are obtained according to the history data of user operations and application use on the television system. Figure 5 As shown, a user portrait is constructed according to the user identity data, the multi-modal features, and the user historical use records, and user portrait data is generated. The user portrait includes behavior patterns, application use preferences, time period rules, and scene adaptation features (which refer to the history data of user operations and application use on the television system). The user portrait data is sent to an application prediction and recommendation module of the television.

[0027] Step S40: Perform application prediction and recommendation according to the user portrait data. When a proactive recommendation trigger condition is met, display a recommended application.

[0028] Specifically, as shown in Figure 2 Step S40 is completed by an application prediction and recommendation module in the television, as shown in Figure 6As shown, the time period features, user behavior features, scene state features, physiological signal features, and application usage features are obtained according to the user portrait data and input into a hierarchical fusion model. The time period features include intra-day time period, intra-week distribution, and 8-day usage regularity. The user behavior features include last application usage, stay duration, and operation rhythm. The scene state features include TV state, brightness, volume, and external connection. The physiological signal features include grip strength, temperature, conductivity, and impedance change rate. The application usage features include per-minute usage record and weight matrix. The hierarchical fusion model includes a cycle layer and a context layer. The cycle layer calculates long-term weights based on time regularity, and the context layer calculates short-term weights (i.e., short-term correction score, mainly calculated according to recent operation behavior and usage habits of the user) based on current operation state. The comprehensive score Score(App) of the application is calculated according to the long-term weights and the short-term weights: Score(App) = α × S_cycle(App) + β × S_context(App); wherein S_cycle(App) represents the long-term weights, S_context(App) represents the short-term weights, and α and β represent two dynamically changeable weight coefficients, which are dynamically adjusted by a scene adaptive mechanism. α is used to control the influence degree of long-term habits, and β is used to control the influence degree of current behavior and current TV state.

[0029] The calculation of the long-term weights is as follows: S_cycle(App) = λ × S_time + γ × S_similar + δ × S_holiday; wherein S_time represents the weight of time dimension, S_similar represents the compensation weight of similar time period, S_holiday represents the holiday weight, λ represents an exponential decay factor for adjusting the weight of long-term data, γ represents a similar time period compensation factor for compensating the similarity between different time periods, and δ represents a holiday correction factor for adjusting the weight of special dates.

[0030] λ: The importance of historical data gradually decreases over time, which is used to adjust the weight of long-term data to ensure that the model pays more attention to recent behavior.

[0031] γ: Used to compensate for the similarity between different time periods. This factor adjusts the weight according to certain specific time period regularity.

[0032] δ: User behavior may be significantly different on holidays than on weekdays, which is used to adjust the weight of special dates to ensure that holiday data is not ignored.

[0033] Among them, the similar time period compensation refers to that the behavior rule of a user in some time period may be repeated in other similar time periods, instead of being limited to the same specific time point, the time periods with similar user habits are connected as a supplementary weight to make the recommendation more accurate. The holiday correction refers to that the use habits of a user during holidays may be different from those during ordinary times, and the recommendation needs to be specially adjusted to make the result more in line with the actual needs during holidays.

[0034] The time dimension: for example, the weight of the last ±30 minutes is set to 3; the weight of ±90-180 minutes is set to 2; and the weight of other times is set to 1. The date dimension: the weight of the previous day is set to 3; the weight of the previous two days or the same day of the previous week is set to 2; and the weight of earlier times is set to 1. The greater the weight, the greater the influence on the final score.

[0035] The context layer: if continuous switching or frequent exit is detected, the weight of the last two applications is increased (that is, when the current behavior feature of continuous switching or frequent exit is detected, the scores of these applications in the short-term weight S_context (App) are improved); the television state correction: the application before the last shutdown in the boot state; the similar category application in the playing state; the voice interaction based on semantic keyword matching.

[0036] According to the comprehensive scores of different applications, the top three applications (that is, the Top-3 applications) are obtained, and if the top three applications meet the active recommendation trigger condition, the recommended application is displayed, the active recommendation trigger condition is that the cumulative use time of the application in the current period is greater than a first time (for example, 60 minutes), the use time span is greater than a second time (for example, 30 minutes), and the current time is located within a third time (for example, 5 minutes) before the start of the historical interval to a fourth time (for example, 15 minutes) before the end of the historical interval, and the three conditions need to be met at the same time to perform the recommendation, so as to ensure that the current behavior of the user is a stable behavior and avoid false triggering of the active recommendation.

[0037] In addition, self-learning and dynamic weight updating are set, for example, the feature weight is increased when the recommendation hits, the feature influence is reduced when the recommendation fails, and the weight is temporarily reduced (shifted by 3-6 positions) when the user manually cancels the recommendation.

[0038] The application is aimed at identity recognition and user behavior prediction in a television system, can more accurately determine the user based on various physiological signals and operation behaviors, emphasizes the comprehensive judgment of the time period, the behavior feature and the scene state to predict the next step of the user, and provides an individualized, scene-adaptive active recommendation service. A technical solution of integrating multiple data sources is provided, the recommendation system is dynamically adjusted, and the user experience is optimized.

[0039] The application has the following beneficial effects: (1) Multi-source fusion recognition accuracy: combining physiological, behavioral and time signals to achieve efficient and accurate user identity recognition.

[0040] (2) Periodic prediction recommendation: based on the time regularity, actively predict the application to be opened by the user.

[0041] (3) Strong context adaptability: the algorithm can automatically adjust the weight according to the TV state, environment and operation mode.

[0042] (4) Privacy protection and low delay: recognition and reasoning are performed locally on the TV without cloud transmission.

[0043] Further, as shown in Figure 7 based on the above-mentioned television application recommendation method based on multi-physiological signal fusion, the present application also correspondingly provides a television application recommendation system based on multi-physiological signal fusion, wherein the television application recommendation system based on multi-physiological signal fusion comprises: a physiological signal acquisition module 51 for acquiring physiological signals of a user collected by a plurality of sensor units of a remote controller, and generating a standardized physiological feature vector according to the physiological signals; a multi-modal fusion recognition module 52 for acquiring behavior features, time features and scene features, inputting the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side calculation model for fusion calculation to obtain user identity data; a user portrait modeling module 53 for acquiring multi-modal features and user historical use records, constructing a user portrait according to the user identity data, the multi-modal features and the user historical use records, and obtaining user portrait data; an application prediction recommendation module 54 for performing application prediction recommendation according to the user portrait data, and displaying recommended applications when the active recommendation trigger condition is met.

[0044] Further, as shown in Figure 8 based on the above-mentioned television application recommendation method and system based on multi-physiological signal fusion, the present application also correspondingly provides a television, which comprises a processor 10, a memory 20 and a display 30. Figure 8 Only part of the components of the television are shown, but it should be understood that all the shown components are not required to be implemented, and more or fewer components can be alternatively implemented.

[0045] The memory 20 can be an internal storage unit of the television in some embodiments, such as a hard disk or a memory of the television. The memory 20 can also be an external storage device of the television in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 20 can include both an internal storage unit and an external storage device of the television. The memory 20 is used to store application software and various data installed on the television, such as program codes of the television, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a television application recommendation program based on multi-physiological signal fusion 40, which can be executed by the processor 10 to implement the television application recommendation method based on multi-physiological signal fusion in the present application.

[0046] The processor 10 can be a Central Processing Unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the television application recommendation method based on multi-physiological signal fusion, etc.

[0047] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the television and to display a visualized user interface. The processor 10, the memory 20 and the display 30 of the television communicate with each other through a system bus.

[0048] In an embodiment, the following steps are implemented when the processor 10 executes the television application recommendation program based on multi-physiological signal fusion 40 in the memory 20: Acquire physiological signals of a user collected by a plurality of sensor units of a remote controller, and generate a standardized physiological feature vector according to the physiological signals; Acquire behavior features, time features and scene features, input the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side calculation model for fusion calculation to obtain user identity data; acquire multi-modal features and user historical usage records, construct a user portrait according to the user identity data, the multi-modal features and the user historical usage records, and obtain user portrait data; perform application prediction recommendation according to the user portrait data, and display a recommended application when a proactive recommendation trigger condition is met.

[0049] The physiological signals include electrodermal response data, skin temperature data, grip pressure data and bioimpedance data. The acquisition of the remote controller acquires physiological signals of the user through a plurality of sensor units, and generates a standardized physiological feature vector according to the physiological signals, specifically including: When the user holds the remote controller, the electrodermal response data reflecting the difference in skin conductive characteristics of the individual is acquired through a skin point sensing unit, the skin temperature data of the individual temperature is acquired through a temperature sensor, the grip pressure data about the hand shape, grip strength and grip posture difference is acquired through a pressure array unit, and the bioimpedance data about the measurement of skin resistance and phase difference is acquired through a bioimpedance detection unit. When the data acquisition time lasts for a preset time, the average value, variance, change rate and spectral features are extracted according to the electrodermal response data, the skin temperature data, the grip pressure data and the bioimpedance data, and a standardized physiological feature vector is generated.

[0050] The behavior features, time features and scene features are acquired, the standardized physiological feature vector, the behavior features, the time features and the scene features are input into a lightweight end-side computing model for fusion calculation to obtain user identity data, specifically including: The behavior features are acquired according to the key rhythm, focus moving speed and commonly used application distribution, the time features are acquired according to the use period, and the scene features are acquired according to the television state. The standardized physiological feature vector, the behavior features, the time features and the scene features are input into a lightweight end-side computing model, and the lightweight end-side computing model performs fusion calculation to identify the current operator identity to obtain user identity data.

[0051] The multi-modal features and user historical usage records are acquired, a user portrait is constructed according to the user identity data, the multi-modal features and the user historical usage records, and user portrait data is obtained, specifically including: The features derived from a plurality of different types of data are combined to obtain multi-modal features, and the user historical usage records are obtained according to the historical data of the user's operation and application use on the television system. construct a user portrait according to the user identity data, the multi-modal features and the user historical usage records, and generate user portrait data, wherein the user portrait comprises a behavior pattern, an application usage preference, a time period regularity and a scene adaptation feature.

[0052] The application prediction recommendation is performed according to the user portrait data, and a recommended application is displayed when a proactive recommendation trigger condition is met, and specifically comprises the following steps. The time period feature, the user behavior feature, the scene state feature, the physiological signal feature and the application usage feature are obtained according to the user portrait data, and are input into a hierarchical fusion model, wherein the hierarchical fusion model comprises a cycle layer and a context layer. The cycle layer calculates a long-term weight based on a time regularity, the context layer calculates a short-term weight based on a current operation state, and a comprehensive score Score(App) of an application is calculated according to the long-term weight and the short-term weight. Score(App) = α × S_cycle(App) + β × S_context(App); Wherein, S_cycle(App) represents a long-term weight, S_context(App) represents a short-term weight, and α and β represent two dynamically changeable weight coefficients, α is used to control the influence degree of long-term habits, and β is used to control the influence degree of current behavior and current television state. The top three applications are obtained according to the comprehensive scores of different applications, and the recommended application is displayed if the top three applications meet the proactive recommendation trigger condition.

[0053] The long-term weight is calculated as follows. S_cycle(App) = λ × S_time + γ × S_similar + δ × S_holiday; Wherein, S_time represents a weight of a time dimension, S_similar represents a compensation weight of a similar period, S_holiday represents a holiday weight, λ represents an exponential decay factor, which is used to adjust the weight of long-term data, γ represents a similar period compensation factor, which is used to compensate the similarity between different time periods, and δ represents a holiday correction factor, which is used to adjust the weight of special dates.

[0054] The proactive recommendation trigger condition is that the cumulative usage time length of the application in the current period is greater than a first time, the usage time span is greater than a second time, and the current time is located within a third time before the start of the historical interval to a fourth time before the end of the historical interval.

[0055] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a television application recommendation program based on multi-physiological signal fusion, and the television application recommendation program based on multi-physiological signal fusion, when executed by a processor, implements the steps of the television application recommendation method based on multi-physiological signal fusion.

[0056] In summary, the application provides a television application recommendation method, system, television and computer readable storage medium based on multi-physiological signal fusion, the method comprising: acquiring physiological signals of a user collected by a plurality of sensor units of a remote controller, and generating a standardized physiological feature vector according to the physiological signals; acquiring behavior features, time features and scene features, inputting the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side calculation model for fusion calculation to obtain user identity data; acquiring multi-modal features and user historical use records, constructing a user portrait according to the user identity data, the multi-modal features and the user historical use records, and obtaining user portrait data; and performing application prediction recommendation according to the user portrait data, and displaying recommended applications when a proactive recommendation triggering condition is met. The application can automatically identify user identity without login, and predict applications that the user is likely to operate next, provide personalized services according to characteristics of different users, improve prediction accuracy of recommended content, and thus realize real no-sensing identification and proactive prediction.

[0057] It should be noted that in this document, the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or televisions including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or televisions. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or television comprising the element.

[0058] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware (such as a processor, a controller, etc.) to complete, and the program can be stored in a computer readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer readable storage medium can be a memory, a disk, an optical disk, etc.

[0059] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes should be within the protection scope of the appended claims of the application.

Claims

1. A method for recommending TV applications based on multi-physiological signal fusion, characterized in that, The television application recommendation method based on multi-physiological signal fusion comprises: acquiring physiological signals of a user collected by a plurality of sensor units of a remote controller, and generating a standardized physiological feature vector according to the physiological signals; acquiring behavior features, time features and scene features, inputting the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side calculation model for fusion calculation to obtain user identity data; acquiring multi-modal features and user historical use records, constructing a user portrait according to the user identity data, the multi-modal features and the user historical use records, and obtaining user portrait data; performing application prediction recommendation according to the user portrait data, and displaying recommended applications when a proactive recommendation triggering condition is met. 2.The method of claim 1, wherein, The physiological signals comprise skin galvanic response data, skin temperature data, holding pressure data and bioimpedance data. The acquiring of the physiological signals of the user collected by the plurality of sensor units of the remote controller, and the generating of the standardized physiological feature vector according to the physiological signals specifically comprise: when the user holds the remote controller, collecting skin galvanic response data reflecting differences in skin conductive characteristics of the individual through a skin point sensing unit, collecting skin temperature data of the individual temperature through a temperature sensor, collecting holding pressure data about hand shape, holding force and holding posture differences through a pressure array unit, and collecting bioimpedance data about measured skin resistance and phase difference through a bioimpedance detection unit; after the time for collecting data lasts for a preset time, extracting mean value, variance, change rate and spectral features according to the skin galvanic response data, the skin temperature data, the holding pressure data and the bioimpedance data to generate a standardized physiological feature vector. 3.The method of claim 1, wherein, The acquiring of the behavior features, the time features and the scene features, and the inputting of the standardized physiological feature vector, the behavior features, the time features and the scene features into the lightweight end-side calculation model for fusion calculation to obtain the user identity data specifically comprise: acquiring behavior features according to key rhythm, focus moving speed and commonly used application distribution, acquiring time features according to use time period, and acquiring scene features according to television state; inputting the standardized physiological feature vector, the behavior features, the time features and the scene features into the lightweight end-side calculation model, and the lightweight end-side calculation model performing fusion calculation to identify the current operator identity to obtain the user identity data. 4.The method of claim 1, wherein, The acquiring of the multi-modal features and the user historical use records, and the constructing of the user portrait according to the user identity data, the multi-modal features and the user historical use records to obtain the user portrait data specifically comprise: combining features derived from a plurality of different types of data to obtain multi-modal features, and obtaining user historical use records according to historical data of operations and application use of the user on the television system; constructing the user portrait according to the user identity data, the multi-modal features and the user historical use records to generate user portrait data, wherein the user portrait comprises behavior patterns, application use preferences, time period regularities and scene adaptation features. 5.The method of claim 1, wherein, The application prediction recommendation is made according to the user portrait data, and a recommended application is displayed when a proactive recommendation trigger condition is met, and specifically includes the following steps: The time period feature, the user behavior feature, the scene state feature, the physiological signal feature and the application use feature are obtained according to the user portrait data and input into a hierarchical fusion model, and the hierarchical fusion model includes a cycle layer and a context layer; The cycle layer calculates a long-term weight based on a time rule, the context layer calculates a short-term weight based on a current operation state, and a comprehensive score Score(App) of an application is calculated according to the long-term weight and the short-term weight: Score(App) = α × S_cycle(App) + β × S_context(App); Wherein, S_cycle(App) represents the long-term weight, S_context(App) represents the short-term weight, and α and β represent two dynamically changeable weight coefficients, α is used to control the influence degree of long-term habits, and β is used to control the influence degree of current behavior and current television state; The top three applications are obtained according to the comprehensive scores of different applications, and if the top three applications meet the proactive recommendation trigger condition, the recommended application is displayed. 6.The method for television application recommendation based on multi-physiological signal fusion according to claim 5, characterized in that, The calculation of the long-term weight is as follows: S_cycle(App) = λ × S_time + γ × S_similar + δ × S_holiday; Wherein, S_time represents the weight of the time dimension, S_similar represents the compensation weight of the similar period, S_holiday represents the holiday weight, λ represents an exponential decay factor, which is used to adjust the weight of long-term data, γ represents a similar period compensation factor, which is used to compensate the similarity between different time periods, and δ represents a holiday correction factor, which is used to adjust the weight of special dates. 7.The method of claim 5, wherein, The proactive recommendation trigger condition is that the current period application cumulative use time is greater than a first time, the use time span is greater than a second time, and the current time is located in a third time before the start of the historical interval to a fourth time before the end.

8. A television application recommendation system based on multi-physiological signal fusion, characterized in that, The television application recommendation system based on multi-physiological signal fusion includes: A physiological signal acquisition module is configured to acquire physiological signals of a user collected by a plurality of sensor units of a remote controller, and generate a standardized physiological feature vector according to the physiological signals; A multi-modal fusion recognition module is configured to acquire behavior features, time features and scene features, input the standardized physiological feature vector, the behavior features, the time features and the scene features into a lightweight end-side calculation model for fusion calculation to obtain user identity data; A user portrait modeling module is configured to acquire multi-modal features and user historical use records, construct a user portrait according to the user identity data, the multi-modal features and the user historical use records, and obtain user portrait data; An application prediction recommendation module is configured to make application prediction recommendation according to the user portrait data, and display a recommended application when a proactive recommendation trigger condition is met.

9. A television set, characterized in that The television comprises a memory, a processor, and a multi-physiological signal fusion based television application recommendation program stored in the memory and executable on the processor, and the multi-physiological signal fusion based television application recommendation program, when executed by the processor, implements the steps of the multi-physiological signal fusion based television application recommendation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a multi-physiological signal fusion based television application recommendation program, and the multi-physiological signal fusion based television application recommendation program, when executed by the processor, implements the steps of the multi-physiological signal fusion based television application recommendation method according to any one of claims 1-7.

Images