User personalized content processing method and device based on camera recognition

Abstract

The invention discloses a user personalized content processing method and device based on camera recognition. The method comprises the steps of controlling and collecting face information of a user in front of a smart television for recognition; when a new user is identified, creating a new virtual user account; recording operation habit data and playing data of a new user in a use process, establishing association with the virtual user account, and constructing a user portrait; when the current user in front of the smart television is identified as the user with the existing user portrait, calling out personalized contents or services corresponding to the user with the existing user portrait, and performing first recommendation display; and further performing action and expression recognition processing on the current user, further screening and filtering the content of the first recommendation display according to the action and expression recognition processing result to obtain personalized content or service matched with the action and expression of the current user, and performing second recommendation display. According to the invention, accurate identification of different family members is realized, so that contents and services more conforming to preferences are provided for the family members; and convenience is provided for the use of the user.

Description

Technical Field

[0001] The present invention relates to the technical field of smart terminals, and in particular to a method and device for processing user personalized content based on camera recognition, a smart television and a storage medium. Background Art

[0002] With the development of electronic technology and the continuous improvement of people's living standards, the use of various smart TVs such as smart TVs has become more and more popular, and smart TVs have become an indispensable household appliance in people's lives.

[0003] As a family-shared device, smart TVs are usually used by more than one user, including the elderly, young people, and young children. Different groups have very different demands for content. Internet TVs in the prior art also have some personalized content recommendation functions, but they are usually unable to provide very accurate services to each user group, which is sometimes inconvenient for users.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the problems and defects of the above-mentioned prior art, a user personalized content processing method, device, smart TV and storage medium based on camera recognition are provided, aiming to solve the problem that Internet TV in the prior art cannot provide very accurate services to each user group and is sometimes inconvenient for users to use.

[0006] The technical solution adopted by the present invention to solve the problem is as follows:

[0007] A user personalized content processing method based on camera recognition, comprising:

[0008] When the smart TV is detected to be started, the camera set on the smart TV is controlled to collect the face information of the user in front of the smart TV;

[0009] Based on the image recognition algorithm, the user's facial information is recognized and it is determined whether a new user is recognized;

[0010] When a new user is identified, a new virtual user account is created; the operation habit data and playback data of the new user during use are recorded, and associated with the virtual user account, and a user profile is gradually built;

[0011] When it is identified that the current user in front of the smart TV is a user with an existing user profile, personalized content or services corresponding to the user with the existing user profile are called out through a preset multi-user model for a first recommendation display;

[0012] During the first recommendation display process, the camera is further controlled to perform motion and expression recognition processing on the current user, and the content of the first recommendation display is further screened and filtered according to the motion and expression recognition processing results to obtain personalized content or services that match the current user's motion and expression, and perform a second recommendation display.

[0013] In the method for processing user personalized content based on camera recognition, the preset multi-user model is constructed by the following steps:

[0014] Pre-collect and obtain the operation habit data and playback data of each user during the smart TV playback process;

[0015] Based on each user's operation habit data and playback data, establish a corresponding relationship with the user and build user portraits for multiple users;

[0016] Based on the constructed user portraits of multiple users, a recommendation algorithm is used to generate personalized recommendation content corresponding to the users of each user portrait, and the constructed multi-user model is trained to obtain a preset multi-user model.

[0017] The method for processing user personalized content based on camera recognition, wherein before the step of detecting that the smart TV is started and controlling the camera set on the smart TV to collect the face information of the user in front of the smart TV, it also includes:

[0018] Users are classified in advance according to different ages: children, elderly and young users.

[0019] The method for processing user personalized content based on camera recognition, wherein before the step of detecting that the smart TV is started and controlling the camera set on the smart TV to collect the face information of the user in front of the smart TV, it also includes:

[0020] Personalized recommendations for children are set in advance with animation and educational content as the priority;

[0021] Personalized recommendations for elderly users are set in advance with health and wellness and opera content as the priority;

[0022] The personalized recommendation content set in advance for multiple users is to recommend content that multiple users like to watch in proportion, or to recommend family-friendly content;

[0023] Pre-set for children users, the control provides differentiated services such as automatic switching of eye protection mode and timely reminder of rest;

[0024] The control provides differentiated services for elderly users by presetting the size of the screen text to a preset value and the volume to a preset threshold.

[0025] The method for processing user personalized content based on camera recognition, wherein the steps of identifying the user's face information based on an image recognition algorithm and determining whether a new user is identified include:

[0026] Get the voice information of the user in front of the smart TV;

[0027] Based on the collected facial and voice information of the user, combined with the data analysis algorithm, the age of the user in front of the smart TV is determined, whether he is a child user, an elderly user or a young user.

[0028] The method for processing user personalized content based on camera recognition, wherein the step of calling out personalized content or services corresponding to the user with the existing user portrait through a preset multi-user model and performing a first recommendation display comprises:

[0029] Control the recommendation priority of animation and educational content for children's personalized recommendations;

[0030] Control the priority of personalized recommendations for elderly users to focus on health and wellness, and opera content;

[0031] The personalized recommended content for multiple users is controlled to recommend content that multiple users like to watch in proportion, or to recommend family-friendly content.

[0032] The method for processing user personalized content based on camera recognition, wherein the step of calling out personalized content or services corresponding to the user with the existing user portrait through a preset multi-user model and performing a first recommendation display comprises:

[0033] For child users, the control provides differentiated services such as automatic switching of eye protection mode and timely reminder of rest;

[0034] For elderly users, the control provides differentiated services such as enlarging the screen text to a preset value and increasing the volume to a preset threshold.

[0035] A user personalized content processing device based on camera recognition, wherein the device comprises:

[0036] A pre-setting module is used to pre-collect and obtain the operation habit data and playback data of each user during the playback of the smart TV; based on the operation habit data and playback data of each user, construct user portraits of multiple users; based on the constructed user portraits of multiple users, a recommendation algorithm is used to generate personalized recommendation content corresponding to the users of each user portrait, and the constructed multi-user model is trained to obtain a preset multi-user model;

[0037] A face collection module is used to detect that the smart TV is started and control the camera set on the smart TV to collect the face information of the user in front of the smart TV;

[0038] A new user identification module is used to identify the user's facial information based on an image recognition algorithm and determine whether a new user is identified;

[0039] A virtual user creation module is used to create a new virtual user account when a new user is identified; and record the operation habit data and playback data of the new user during use, establish an association with the virtual user account, and gradually build a user portrait;

[0040] A first recommendation control module, for, when recognizing that the current user in front of the smart TV is a user with an existing user profile, calling out personalized content or services corresponding to the user with the existing user profile through a preset multi-user model, performing a first recommendation display;

[0041] The second recommendation control module further controls the camera to perform motion and expression recognition processing on the current user during the first recommendation display process, and further filters the content of the first recommendation display according to the motion and expression recognition processing results to obtain personalized content or services that match the current user's motion and expression, and perform a second recommendation display.

[0042] A smart TV comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, including the method for executing any one of the methods described above.

[0043] A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the methods described above.

[0044] Beneficial effects of the present invention: The present invention provides a method, device, smart TV and storage medium for user personalized content processing based on camera recognition. The present invention realizes the recognition of individual users in the family based on the camera, creates virtual users, and provides differentiated content / services for users. Based on the multi-user model, big data is collected to build accurate portraits of multiple users. Based on differences in age, gender, and preferences (big data portraits), more accurate content recommendations and considerate services are provided to users. The present invention realizes accurate identification of different family members, thereby providing them with content and services that are more in line with their preferences; it provides convenience for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 It is a flowchart of a method for processing user personalized content based on camera recognition provided by an embodiment of the present invention.

[0047] Figure 2 It is a flowchart of a method for processing user personalized content based on camera recognition provided by an embodiment of the present invention.

[0048] Figure 3 It is a principle block diagram of an embodiment of a user personalized content processing device based on camera recognition provided by the present invention.

[0049] Figure 4 It is a block diagram of the internal structure principle of the smart TV provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0052] In recent years, more and more TVs have been equipped with cameras to meet users' needs for video calls, etc. The camera's image recognition capability has also been greatly improved, and it can easily and accurately identify the user's gender, age, emotions and other information.

[0053] The present invention provides a user personalized content processing method based on camera recognition. Through the camera capability, accurate recognition of different family members is achieved, so as to provide them with content and services that are more in line with their preferences.

[0054] like Figure 1 As shown, a user personalized content processing method based on camera recognition in embodiment 1 of the present invention includes the following steps:

[0055] Step S100: Detecting that the smart TV is started, controlling the camera set on the smart TV to collect the face information of the user in front of the smart TV;

[0056] In the embodiment of this step, a smart TV is taken as an example for explanation. When the user turns on the TV, the camera is called; when it is detected that the smart TV is turned on, the camera set on the smart TV is controlled to collect the facial information of the user in front of the smart TV.

[0057] Specifically, the smart TV of the embodiment of the present invention will automatically detect the surrounding environment when it is turned on to determine whether there is a user within its field of view. This process may involve the use of motion sensors or infrared sensors. Once a user is detected, the smart TV will automatically activate its built-in camera. The camera will start to collect the user's facial image for subsequent identity recognition. The smart TV of the present invention can extract the user's facial features through image processing technology and compare them with pre-stored user data. This process usually involves machine learning and computer vision technology.

[0058] For example, when multiple members of a family use the same smart TV, every time the TV is turned on, it will identify which family member is watching through the camera.

[0059] The benefit of this step is that users can be accurately identified, and face recognition technology can be used for home security, ensuring that only authorized users can access specific content or settings to prevent unauthorized access. Users do not need to log in manually or enter passwords, the system can automatically identify and log in to the user account through the camera, saving time and effort.

[0060] Step S200: Based on the image recognition algorithm, recognize the user's facial information and determine whether a new user is recognized;

[0061] In this step, the image recognition algorithm is based on a deep learning model that can extract facial features from images captured by a camera, including a convolutional neural network (CNN), which can effectively identify and classify objects in the image.

[0062] In specific implementation, the present invention will compare the captured facial image with the user's facial features stored in the database. By matching the feature points, it is possible to determine whether the current user is a known user, that is, to determine whether the user currently watching TV is an old user whose viewing data has been collected before.

[0063] In the embodiment of the present invention, if the system cannot find facial features matching the current user in the database, it will mark the user as a new user. This process not only involves identification, but also includes recording and storing the new user information for future identification.

[0064] For example, when a smart TV is used in a family with parents and two children, every time the TV is turned on, the system analyzes the facial features of the current user through an image recognition algorithm. If Xiao Ming (one of the children) is watching, the system will recognize his facial features and recommend cartoons based on his viewing history in the following steps.

[0065] However, if Xiao Ming's friend Xiao Hua comes to the house and uses the TV, the present invention will recognize that Xiao Hua's facial features do not match any records in the database, so it marks him as a new user. At this time, the present invention can ask Xiao Hua whether he wants to create a new user account to record his viewing preferences. In this way, the smart TV can not only provide personalized content recommendations, but also effectively manage the usage of family members and improve the overall user experience.

[0066] The benefit of this step is that by identifying users, smart devices can provide more personalized services and content recommendations, improving user satisfaction. And they can effectively manage the preferences and settings of multiple users to ensure that each user can get content that suits their interests. By identifying new users, unauthorized access can be prevented to ensure family privacy and security, especially when sensitive content is involved.

[0067] Step S300: When a new user is identified, a new virtual user account is created; and the operation habit data and playback data of the new user during use are recorded, associated with the virtual user account, and a user profile is gradually constructed;

[0068] In this step, when a new user is identified, a virtual user account is automatically generated. This account does not require the user to manually enter information, but is automatically created by the system to ensure that the user can quickly start using the device. And record the operating habit data of the new user. During the user's use of the smart device, the present invention will continuously monitor and record the user's operating habits. This includes the type of programs watched by the user, the viewing time, the selected channels, the searched content, etc.

[0069] In addition to operating habits, the present invention also records the user's playback data, such as the specific program watched, playback time, frequency of pause and fast forward, etc. These data are helpful for analyzing the user's preferences.

[0070] The specific embodiment of the present invention can gradually build a user portrait through comprehensive analysis of operation habits and playback data. This portrait includes the user's interests, preferences and behavior patterns, helping the system to better understand user needs.

[0071] For example, when a new user Xiaohua uses a smart TV for the first time. After the method embodiment of the present invention recognizes Xiaohua, it automatically creates a virtual user account. In the next few days, Xiaohua watched several science fiction movies and some music programs. The embodiment of the present invention records his viewing time, preferred types, and the keywords he entered in the search bar. As Xiaohua uses it, the present invention gradually builds his user portrait and records that he has a strong interest in science fiction movies and music programs. Based on these data, the smart TV of the present invention can recommend more similar science fiction movies and music-related content to Xiaohua in the following steps.

[0072] The present invention also records that Xiaohua likes to watch programs at night, so during the evening period, the system will give priority to pushing newly released science fiction movies and live concerts. This personalized recommendation not only improves Xiaohua's viewing experience, but also makes the use of smart TV more efficient and enjoyable.

[0073] The benefit of this step is that by establishing a user profile, the smart device can provide more accurate content recommendations and improve the user's viewing experience. It can also adjust the interface and function settings according to the user's operating habits, making it more convenient and comfortable for the user to use. And over time, the system can continuously update and improve the user profile to ensure that the recommended content always meets the user's latest interests and needs.

[0074] In a specific embodiment, when a user turns on the TV, the present invention calls the camera to identify the basic features of one or more users currently watching TV. When a new user is identified, a virtual user is created for the new user. After that, the user's viewing, clicking and other operation behaviors on the TV end will be associated with the virtual user, and the user's portrait will be gradually built. During the gradual use of the user, a complete family user portrait can be gradually built.

[0075] Step S400: When it is identified that the current user in front of the smart TV is a user with an existing user profile, personalized content or services corresponding to the user with the existing user profile are called out through a preset multi-user model, and a first recommendation display is performed;

[0076] In this step, the smart TV uses facial recognition technology or user login information to confirm whether the current user is a registered user. If the recognition result is a known user, the relevant data will be immediately mobilized for processing.

[0077] In the embodiment of the present invention, preset models for different users are built into the system, and these models are established based on the user's viewing habits, activity, preference type, etc. Through these models, the system can accurately match the relationship between the user and his or her portrait.

[0078] In the embodiment of the present invention, once the current user is confirmed and based on the corresponding user portrait, the system will extract personalized content and services related to the user, such as recommended movies, programs, applications or other functions. The present invention will display the filtered personalized recommended content on the user interface. This display usually occurs when the user uses the TV for the first time or revisits it after a long period of non-use.

[0079] For example, when user Xiao Ming often watches action movies on smart TV, and the present invention records this preference in the user portrait in the previous step, Xiao Ming comes to the living room that day, and the TV instantly recognizes that he is logged in. When Xiao Ming opens the interface behind the TV, the present invention can quickly call up a series of personalized action movies based on Xiao Ming's portrait, which may include newly released blockbusters, classics with excellent film reviews, etc.

[0080] In addition, the present invention can also push related film and television services based on his previous viewing history, such as subscribed action movie channels, or game applications related to action movies, etc. This personalized content recommendation allows Xiao Ming to quickly find his favorite movies, improves the user experience, and also allows him to experience the wisdom and convenience of smart TVs.

[0081] The benefit of this step is that through personalized recommendations, users can quickly find content that interests them, avoid getting lost in a sea of ​​information, and significantly shorten the search time. Personalized content recommendations can make users feel understood, thereby increasing their reliance on the platform and frequency of use, and increasing user retention. And when users can easily find content they like, the overall viewing experience is improved, which generates positive feedback in the user's mind, improves user satisfaction, and provides convenience for user use.

[0082] In a further embodiment, the present invention further comprises the following steps before being implemented:

[0083] S11, pre-collecting and obtaining the operation habit data and playback data of each user during the playback of the smart TV;

[0084] That is, the present invention will automatically record the user's operating habits and playback data during the use of the smart TV. These data include the programs watched by the user, the stay time, the search history, the evaluation, the collection, etc. This information provides a basis for subsequent analysis.

[0085] S12, based on the operation habit data and playback data of each user, establish a corresponding relationship with the user and construct user portraits of multiple users;

[0086] The operating habits and playback data collected in this step will be used to establish a corresponding relationship with the user. Through data analysis, the present invention can extract information such as each user's interest preferences, viewing habits, active time, etc. to form a user portrait. The user portrait is a comprehensive description of the user's characteristics, which can reflect their inclination and preference in content selection.

[0087] S13. Based on the constructed user portraits of multiple users, a recommendation algorithm is used to generate personalized recommendation content corresponding to the users of each user portrait, and the constructed multi-user model is trained to obtain a preset multi-user model.

[0088] Based on the constructed user portraits, the present invention will use the recommendation algorithm in machine learning to analyze the portrait features of different users, thereby generating personalized recommended content. In this process, the present invention will train multiple user models, optimize the recommendation algorithm, ensure that the recommended content is the most appropriate for each user, and finally obtain a multi-user model that can be deployed.

[0089] For example, a specific scenario is used to illustrate this process: when user Xiao Li is a teenager who loves action movies, the present invention records Xiao Li's viewing behavior in step S11, such as he often watches action movies, likes to watch them on weekend nights, and always shows a strong interest in newly released movies when watching.

[0090] Then in step S12, the present invention analyzes these behavioral data to obtain a user portrait of Xiao Li: he is a young user who loves action movies, prefers specific types of actors, and rarely watches documentaries or romance movies.

[0091] Then, in step S13, the present invention uses this portrait to generate personalized recommended content for Xiao Li through a recommendation algorithm. When Xiao Li turns on the smart TV, he can see a series of newly released action movie recommendations, recommendations similar to the movies he has watched, and a list of related movies generated based on his browsing history. Through such accurate recommendations, Xiao Li can not only quickly find the content he likes, but also have a higher satisfaction and expectation for it, further increasing his frequency of using the smart TV.

[0092] Step S500: During the first recommendation display process, the camera is further controlled to perform motion and expression recognition processing on the current user, and the content of the first recommendation display is further screened and filtered according to the motion and expression recognition processing results to obtain personalized content or services matching the current user's motion and expression, and perform a second recommendation display.

[0093] This step embodiment refers to that when a user first contacts the recommended content, the smart TV uses a camera to recognize the user's actions and expressions in real time, so as to further optimize the recommended content.

[0094] In the embodiment of this step, regarding the motion expression recognition processing, specifically, the built-in camera of the smart TV can be used to capture the user's facial expressions and body movements in real time, and the user's immediate reaction to the recommended content can be obtained by analyzing the user's emotional state (such as happiness, surprise, boredom, etc.) and body language (such as nodding, shaking head, etc.).

[0095] Then the recommended content is filtered. Specifically, according to the recognized user expressions and actions, the present invention will further filter the first recommended content. For example, if the user shows an excited expression when watching a recommended movie trailer, the present invention will further prioritize the full movie or similar type of movie related to the movie. The second recommendation display is performed. Specifically, after filtering in this step, personalized content that matches the user's current emotions and actions will be displayed. This recommendation process is dynamic and can be adjusted in real time to ensure that the user obtains the viewing experience that best suits his current state.

[0096] For example, when user Xiaohua is watching smart TV, the present invention recommends a newly released science fiction movie for the first time according to the previous steps. If Xiaohua's expression is detected to show obvious excitement and expectation when watching the trailer. At this time, the smart TV of the present invention captures this emotion through the camera and analyzes it. Based on Xiaohua's excited reaction, the embodiment of the present invention will decide to further filter the recommended content and filter and push behind-the-scenes footage, film reviews, and other similar types of science fiction movies related to the science fiction movie. Finally, a new recommendation list is displayed on the smart TV, which contains content that Xiaohua may be interested in, such as "Science Fiction Movie Rankings" or "Upcoming Science Fiction Blockbusters".

[0097] It can be seen that the recommendation method based on the user's real-time emotions of the present invention not only allows Xiaohua to feel the intelligence and humanity of the smart TV, but also makes his viewing experience richer and more enjoyable, providing convenience for the user.

[0098] The present invention also has the following benefits: 1) Enhanced interactivity: Through real-time motion and expression recognition, the interaction between the user and the smart TV becomes more natural and smooth, thereby enhancing the user's sense of participation.

[0099] 2) Improving recommendation accuracy: The present invention can adjust the recommended content according to the user's immediate reaction, ensuring that the pushed content is more in line with the user's current emotions and interests, thereby improving the accuracy and satisfaction of the recommendation.

[0100] 3) Deepening of personalized experience: This emotion-based recommendation mechanism makes the personalized experience more in-depth. Users can not only get recommendations based on historical data, but also enjoy personalized services brought by real-time feedback.

[0101] In another embodiment of the present invention, the following steps are also included before the specific implementation:

[0102] Users are classified in advance according to different ages: children, elderly and young users.

[0103] Among them, child users refer to users aged between 0 and 12. The content preferences of this group are mainly concentrated on cartoons, children's educational programs, games and interactive content. The viewing time and type of child users are often influenced by their parents, and the content needs to be in line with their cognitive level and interests.

[0104] Young users: refers to users between the ages of 13 and 35. Users in this age group are usually curious about new things and like to watch popular TV series, movies, variety shows, and online videos. Their viewing habits may be influenced by social media and popular culture.

[0105] Elderly users: refers to users aged 60 and above. Elderly users' content preferences may be more inclined towards classic movies, news, health and lifestyle programs, etc. In addition, they may face some operational challenges when using smart TVs, so they need content presentation methods with user-friendly interfaces and simple operations.

[0106] In another embodiment of the present invention, the following steps are also included before the specific implementation:

[0107] Personalized recommendations for children are set in advance with animation and educational content as the priority;

[0108] Personalized recommendations for elderly users are set in advance with health and wellness and opera content as the priority;

[0109] The personalized recommendation content set in advance for multiple users is to recommend content that multiple users like to watch in proportion, or to recommend family-friendly content;

[0110] Pre-set for children users, the control provides differentiated services such as automatic switching of eye protection mode and timely reminder of rest;

[0111] The control provides differentiated services for elderly users by presetting the size of the screen text to a preset value and the volume to a preset threshold.

[0112] Through the above-mentioned pre-setting, it is convenient for the present invention to achieve a faster response speed to the user's personalized recommendation content and differentiated services during specific implementation.

[0113] In another embodiment of the present invention, the step of identifying the user's face information based on the image recognition algorithm and determining whether a new user is identified in step S200 includes:

[0114] S201, obtaining voice information of a user in front of a smart TV;

[0115] In the embodiment of this step, the smart TV can collect the user's voice information through the built-in microphone, such as the user's voice commands, conversations or other sounds when watching TV. By analyzing these sound data, the smart TV can obtain the user's language characteristics, intonation, speech speed and other information. This information is crucial in subsequent user analysis because the voice characteristics can reflect the user's emotions, gender and potential age group.

[0116] S202: Based on the collected facial information and voice information of the user and in combination with a data analysis algorithm, determine the age of the user in front of the smart TV, whether the user is a child user, an elderly user, or a young user.

[0117] In this step embodiment, the smart TV combines the sound information collected in step S201 with the face information obtained through the camera. By using a data analysis algorithm (such as a machine learning model), the user's age can be estimated and whether the user is a child, a young person or an elderly user can be determined. Specifically, the algorithm analyzes facial features (such as wrinkles, facial structure) and voice features (such as tone, speech speed) to draw conclusions.

[0118] For example, when a smart TV is playing a program, users Xiao Zhang (30 years old) and Xiao Ming (7 years old) are watching in front of the TV. The smart TV collects their voice information through the microphone and obtains their facial features through the camera.

[0119] For Xiao Zhang: the smart TV analyzes his voice characteristics and finds that his voice is low-pitched and his speech speed is fast. Combined with the face recognition result, the algorithm determines that he is a young user. Based on this judgment, the present invention can recommend the latest TV series and movies in the following steps to meet Xiao Zhang's viewing needs.

[0120] For Xiao Ming: the smart TV detects that Xiao Ming's voice is high and his speech speed is slow, and his facial features show that he is a child. Therefore, the system classifies Xiao Ming as a child user. In the following steps, the present invention can recommend cartoons and children's educational programs suitable for him to ensure the safety and fun of the content.

[0121] In this way, smart TVs can not only provide personalized content, but also ensure the safety and enjoyment of users during the viewing process, thereby improving the overall user experience.

[0122] By accurately identifying the user's age group, smart TVs can provide more personalized content recommendations. For example, child users can get cartoons and educational programs suitable for them, while elderly users can watch health and lifestyle programs. And for child users, the smart TV of the present invention can automatically filter out content that is not suitable for them to watch, ensuring that their viewing environment is safer. This classification can effectively reduce the impact of inappropriate content on teenagers. And according to the user's age group, smart TVs can push more relevant advertisements to specific groups, thereby improving the conversion rate of advertisements and user satisfaction. And through sound and face recognition, smart TVs can achieve more natural user interaction. For example, users can more conveniently control the TV through voice commands to improve the user experience.

[0123] In another embodiment of the present invention, the step of calling out personalized content or services corresponding to the user with the existing user portrait through a preset multi-user model and performing a first recommendation display in step S400 includes:

[0124] S401, controlling the recommendation priority of personalized recommended content for child users to be animation and educational content;

[0125] In the embodiment of the present invention, the smart TV is controlled to recommend content suitable for the child users' age, mainly focusing on cartoons and educational resources. Animation can not only attract children's attention, but also easily impart knowledge.

[0126] The benefit of this step is that it can ensure that the recommended content does not contain any content that is inappropriate for children, thus protecting their mental health. It can also stimulate children's interest in learning and improve their cognitive abilities through entertaining and educational means.

[0127] For example, cartoon A and cartoon B are recommended for 5-year-old children, and simple math and language learning applications such as "Language Enlightenment" or "Mathematics Enlightenment" are also recommended.

[0128] S402, controlling the recommendation priority of personalized recommendation content for elderly users to be health and wellness and opera content;

[0129] This step controls the smart TV to provide suitable health and cultural content for elderly users, focusing on health and wellness knowledge and traditional opera, etc., to meet their life needs and cultural interests.

[0130] For example, the present invention can recommend health encyclopedia knowledge and health recipes to users over 60 years old, and play classic clips of Peking opera or Kunqu opera to help them understand and appreciate traditional culture. By recommending health knowledge, it helps elderly users obtain correct health information, thereby improving their quality of life; by recommending traditional cultural content such as opera, it enhances the elderly users' sense of cultural identity and belonging.

[0131] S403: Control the personalized recommended content for multiple users to recommend content that the multiple users like to watch in proportion, or recommend family-friendly content.

[0132] This step is for multiple users in a family using the smart TV of the embodiment of the present invention, and priority is given to recommending content that all members can accept or like, so the recommended content should take into account different age groups and interests. For example, in a family including parents and children, the personalized recommendation of the embodiment of the present invention can recommend movies suitable for the whole family, such as movie A and movie B, and recommend family games, such as game A or game B, so that all family members can participate.

[0133] The benefit of this step is that it promotes interaction and communication among family members and increases the fun of watching together through the recommendation of family-friendly content. It can also meet the needs of different users and ensure that each user can find content of interest.

[0134] Yes, the present invention can better meet the interests and needs of users and improve user satisfaction through personalized recommendations for specific user groups. The content required by users of different age groups has different focuses. Through effective recommendation strategies, it is possible to ensure the suitability of the content while enhancing the user experience and promoting family harmony and interaction.

[0135] In another embodiment of the present invention, the step S400 uses a preset multi-user model to call out personalized content or services corresponding to the user with the existing user portrait, and performs the first recommendation display step includes:

[0136] S411. For child users, the control provides differentiated services such as automatic switching of eye protection mode and timely reminder of rest;

[0137] This step is intended to protect the eyesight health of child users. Controlling the smart TV of the present invention to automatically switch to the eye protection mode can reduce the damage to the eyes caused by the blue light on the screen, and the timely reminder to rest encourages children to take regular breaks and avoid staring at the screen for a long time. For example, a children's learning application automatically switches to the eye protection mode at 6 o'clock in the evening, and the screen changes to a soft yellow tone. At the same time, every 30 minutes, the application will pop up a reminder to remind the child to rest his eyes and do 5 minutes of eye relaxation activities.

[0138] The benefit of this step is that the eye protection mode can effectively reduce blue light radiation, reduce eye fatigue, and protect children's vision development. Regular rest reminders can help children develop good eye habits and avoid health problems caused by long-term use of electronic devices.

[0139] S412: For elderly users, the control provides differentiated services such as enlarging the screen text to a predetermined value and increasing the volume to a predetermined threshold.

[0140] This step is intended to improve the convenience of use for elderly users. By controlling the smart TV of the present invention to recognize elderly users, the screen text is enlarged and the volume is increased to ensure that the elderly users can read and hear the content more clearly, thereby improving their use experience.

[0141] For example, when the smart TV of the present invention recognizes an elderly user, it will automatically adjust the text size to 18 points and set the volume to 75% when the user uses it for the first time. When the user watches a health video, clear large-font titles will be displayed on the interface to ensure that they can read easily, accompanied by clear audio narration to help understand the content.

[0142] The advantage of this step embodiment is that the text can be enlarged to make it easier for elderly users to read, reduce visual fatigue, and enhance the efficiency of information acquisition. In addition, increasing the volume helps elderly users better understand and enjoy the content, especially for users with hearing impairments.

[0143] The present invention can effectively improve the user experience and satisfaction of children and elderly users by providing differentiated services for them. Eye protection measures and rest reminders for children help protect eyesight and cultivate healthy habits, while text and volume adjustment for elderly users improve readability and ease of use. These measures not only focus on the health of users, but also enhance their interaction with the application and improve the overall experience.

[0144] Embodiment 2, as Figure 2 As shown, the second embodiment provides a method for processing user personalized content based on camera recognition, including the following steps:

[0145] Step S10, start, turn on the TV, and go to step S11;

[0146] In this specific embodiment, the user turns on the television and prepares to enter the process of user identification and personalized service.

[0147] Step S11, calling the camera and proceeding to step S12;

[0148] Control the smart TV to actively enable the built-in camera to capture the user's image, which is prepared for subsequent user recognition.

[0149] Step S12: Based on image recognition, determine whether a new user is recognized, if so, proceed to step S13, if not, proceed to step S14;

[0150] The present invention uses an image recognition algorithm to analyze the captured image. If the new user identified is different from the existing user, the process proceeds to step S13. If no new user is identified, the process directly proceeds to step S14;

[0151] Step S13, create a new virtual user, and proceed to step S14;

[0152] In the embodiment of the present invention, when a new user is identified, a new virtual user profile is created for the user. The profile stores the basic information and preference settings of the user for subsequent use.

[0153] Step S14, setting the identified user as the current user, and proceeding to step S15;

[0154] In the embodiment of the present invention, whether it is a new user or an identified user, it will be set as the current operating user so that its behavior can be recorded and analyzed.

[0155] Step S15: other usage behaviors are associated with the current user, and the process proceeds to step S16;

[0156] In the embodiment of the present invention, various usage behaviors of the current user, such as programs watched, contents searched, etc., are monitored and these behaviors are connected with the current user profile, which helps to provide personalized services to the user later.

[0157] Step S16: forming a user portrait and proceeding to step S17;

[0158] In the embodiment of the present invention, a user portrait is generated based on the user's behavior data. This portrait includes information such as the user's interests and viewing habits, and is used to better understand the user's needs.

[0159] Step S17: providing personalized content or services based on existing user portraits;

[0160] In the embodiment of the present invention, the generated user portrait is used to analyze the user's preferences, so as to recommend personalized content or services to the user, so as to improve the user satisfaction and interactive experience.

[0161] Then, end the process.

[0162] It can be seen that in the embodiment of the present invention, the process of user identity recognition and personalized service provision is realized by image recognition technology. By automatically creating user profiles and tracking user behavior in real time, relevant content can be recommended to users more accurately, improving user experience. Such a process not only facilitates user use and reduces the steps of manual setting, but also improves the effect of personalized services.

[0163] Exemplary Devices

[0164] like Figure 3 As shown, an embodiment of the present invention provides a user personalized content processing device based on camera recognition, the device comprising:

[0165] The presetting module 310 is used to pre-collect and obtain the operation habit data and playback data of each user during the playback of the smart TV; construct user portraits of multiple users based on the operation habit data and playback data of each user; based on the constructed user portraits of the multiple users, use a recommendation algorithm to generate personalized recommendation content corresponding to the users of each user portrait, and train the constructed multi-user model to obtain a preset multi-user model;

[0166] The face collection module 320 is used to detect that the smart TV is started and control the camera set on the smart TV to collect the face information of the user in front of the smart TV;

[0167] A new user identification module 330 is used to identify the user's facial information based on an image recognition algorithm and determine whether a new user is identified;

[0168] The virtual user creation module 340 is used to create a new virtual user account when a new user is identified; and record the operation habit data and playback data of the new user during use, establish an association with the virtual user account, and gradually build a user portrait;

[0169] The first recommendation control module 350 is used to, when recognizing that the current user in front of the smart TV is a user with an existing user profile, call out personalized content or services corresponding to the user with the existing user profile through a preset multi-user model, and perform a first recommendation display;

[0170] The second recommendation control module 360, during the first recommendation display process, further controls the camera to perform motion and expression recognition processing on the current user, and further filters the content of the first recommendation display according to the motion and expression recognition processing results to obtain personalized content or services that match the current user's motion and expression, and perform a second recommendation display, as described above.

[0171] Based on the above embodiments, the present invention further provides a smart TV, whose principle block diagram can be shown as follows: Figure 4 As shown. The smart TV includes a processor, a memory, a network interface, a display screen, and a database connected through a system bus. Among them, the processor of the smart TV is used to provide computing and control capabilities. The memory of the smart TV includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the smart TV is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a user personalized content processing method based on camera recognition is implemented. The database of the smart TV is used to store a user personalized content processing program based on camera recognition.

[0172] Those skilled in the art will understand that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the smart TV to which the solution of the present invention is applied. A specific smart TV may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0173] In one embodiment, a smart TV is provided, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include instructions for performing the following operations:

[0174] When the smart TV is detected to be started, the camera set on the smart TV is controlled to collect the face information of the user in front of the smart TV;

[0175] Based on the image recognition algorithm, the user's facial information is recognized and it is determined whether a new user is recognized;

[0176] When a new user is identified, a new virtual user account is created; the operation habit data and playback data of the new user during use are recorded, and associated with the virtual user account, and a user profile is gradually built;

[0177] When it is identified that the current user in front of the smart TV is a user with an existing user profile, personalized content or services corresponding to the user with the existing user profile are called out through a preset multi-user model for a first recommendation display;

[0178] During the first recommendation display process, the camera is further controlled to perform motion and expression recognition processing on the current user, and the content of the first recommendation display is further screened and filtered according to the motion and expression recognition processing results to obtain personalized content or services that match the current user's motion and expression, and perform a second recommendation display, as described above.

[0179] The preset multi-user model is constructed by the following steps:

[0180] Pre-collect and obtain the operation habit data and playback data of each user during the smart TV playback process;

[0181] Based on each user's operation habit data and playback data, establish a corresponding relationship with the user and build user portraits for multiple users;

[0182] Based on the constructed user portraits of multiple users, a recommendation algorithm is used to generate personalized recommendation content corresponding to the users of each user portrait, and the constructed multi-user model is trained to obtain a preset multi-user model, as described above.

[0183] Wherein, before the step of detecting that the smart TV is started and controlling the camera set on the smart TV to collect the face information of the user in front of the smart TV, the step further includes:

[0184] Users are classified in advance according to different ages: child users, elderly users, and young users, as described above.

[0185] Wherein, before the step of detecting that the smart TV is started and controlling the camera set on the smart TV to collect the face information of the user in front of the smart TV, the step further includes:

[0186] Personalized recommendations for children are set in advance with animation and educational content as the priority;

[0187] Personalized recommendations for elderly users are set in advance with health and wellness and opera content as the priority;

[0188] The personalized recommendation content set in advance for multiple users is to recommend content that multiple users like to watch in proportion, or to recommend family-friendly content;

[0189] Pre-set for children users, the control provides differentiated services such as automatic switching of eye protection mode and timely reminder of rest;

[0190] The control provides differentiated services for elderly users by presetting the size of the screen text to a predetermined value and the volume to a predetermined threshold, as described above.

[0191] The step of identifying the user's facial information based on an image recognition algorithm and determining whether a new user is identified includes:

[0192] Get the voice information of the user in front of the smart TV;

[0193] Based on the collected facial and voice information of the user, combined with the data analysis algorithm, the age of the user in front of the smart TV is determined, whether he is a child user, an elderly user or a young user, as described above.

[0194] The step of calling out personalized content or services corresponding to the user with the existing user portrait through a preset multi-user model and performing a first recommendation display includes:

[0195] Control the recommendation priority of animation and educational content for children's personalized recommendations;

[0196] Control the priority of personalized recommendations for elderly users to focus on health and wellness, and opera content;

[0197] The personalized recommended content for multiple users is controlled to recommend content that multiple users like to watch in proportion, or to recommend family-friendly content, as described above.

[0198] The step of calling out personalized content or services corresponding to the user with the existing user portrait through a preset multi-user model and performing a first recommendation display includes:

[0199] For child users, the control provides differentiated services such as automatic switching of eye protection mode and timely reminder of rest;

[0200] For elderly users, the control provides differentiated services such as enlarging the screen text to a predetermined value and increasing the volume to a predetermined threshold, as described above.

[0201] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0202] In summary, the present invention provides a method, device, smart TV and storage medium for user personalized content processing based on camera recognition. The present invention realizes the recognition of individual users in the family based on the camera, creates virtual users, and provides differentiated content / services for users. Based on the multi-user model, big data is collected to build accurate portraits of multiple users. Based on differences in age, gender, and preferences (big data portraits), more accurate content recommendations and considerate services are provided to users. The present invention realizes accurate identification of different family members, thereby providing them with content and services that are more in line with their preferences; it provides convenience for users.

Claims

1. A user personalized content processing method based on camera recognition, characterized in that: include: When the smart TV is detected to be started, the camera set on the smart TV is controlled to collect the face information of the user in front of the smart TV; Based on the image recognition algorithm, the user's facial information is recognized and it is determined whether a new user is recognized; When a new user is identified, a new virtual user account is created; And record the operation habit data and playback data of the new user during use, establish association with the virtual user account, and gradually build a user portrait; When it is identified that the current user in front of the smart TV is a user with an existing user profile, personalized content or services corresponding to the user with the existing user profile are called out through a preset multi-user model for a first recommendation display; During the first recommendation display process, the camera is further controlled to perform motion and expression recognition processing on the current user, and the content of the first recommendation display is further screened and filtered according to the motion and expression recognition processing results to obtain personalized content or services that match the current user's motion and expression, and perform a second recommendation display.

2. The method for processing user personalized content based on camera recognition according to claim 1, characterized in that: The preset multi-user model is constructed by the following steps: Pre-collect and obtain the operation habit data and playback data of each user during the smart TV playback process; Based on each user's operation habit data and playback data, establish a corresponding relationship with the user and build user portraits for multiple users; Based on the constructed user portraits of multiple users, a recommendation algorithm is used to generate personalized recommendation content corresponding to the users of each user portrait, and the constructed multi-user model is trained to obtain a preset multi-user model.

3. The method for processing user personalized content based on camera recognition according to claim 1, characterized in that: Before the step of detecting that the smart TV is started and controlling the camera set on the smart TV to collect the face information of the user in front of the smart TV, the step also includes: Users are classified in advance according to different ages: children, elderly and young users.

4. The method for processing user personalized content based on camera recognition according to claim 1, characterized in that: Before the step of detecting that the smart TV is started and controlling the camera set on the smart TV to collect the face information of the user in front of the smart TV, the step also includes: Personalized recommendations for children are set in advance with animation and educational content as the priority; Personalized recommendations for elderly users are set in advance with health and wellness and opera content as the priority; The personalized recommendation content set in advance for multiple users is to recommend content that multiple users like to watch in proportion, or to recommend family-friendly content; Pre-set for children users, the control provides differentiated services such as automatic switching of eye protection mode and timely reminder of rest; The control provides differentiated services for elderly users by presetting the size of the screen text to a preset value and the volume to a preset threshold.

5. The method for processing user personalized content based on camera recognition according to claim 1, characterized in that: The step of identifying the user's face information based on the image recognition algorithm and determining whether a new user is identified includes: Get the voice information of the user in front of the smart TV; Based on the collected facial and voice information of the user, combined with the data analysis algorithm, the age of the user in front of the smart TV is determined, whether he is a child user, an elderly user or a young user.

6. The method for processing user personalized content based on camera recognition according to claim 1, characterized in that: The step of calling out personalized content or services corresponding to the user with the existing user portrait through the preset multi-user model and performing the first recommendation display comprises: Control the recommendation priority of animation and educational content for children's personalized recommendations; Control the priority of personalized recommendations for elderly users to focus on health and wellness, and opera content; The personalized recommended content for multiple users is controlled to recommend content that multiple users like to watch in proportion, or to recommend family-friendly content.

7. The method for processing user personalized content based on camera recognition according to claim 1, characterized in that: The step of calling out personalized content or services corresponding to the user with the existing user portrait through the preset multi-user model and performing the first recommendation display comprises: For child users, the control provides differentiated services such as automatic switching of eye protection mode and timely reminder of rest; For elderly users, the control provides differentiated services such as enlarging the screen text to a preset value and increasing the volume to a preset threshold.

8. A user personalized content processing device based on camera recognition, characterized in that: The device comprises: A pre-setting module is used to pre-collect and obtain the operation habit data and playback data of each user during the playback of the smart TV; based on the operation habit data and playback data of each user, construct user portraits of multiple users; based on the constructed user portraits of multiple users, a recommendation algorithm is used to generate personalized recommendation content corresponding to the users of each user portrait, and the constructed multi-user model is trained to obtain a preset multi-user model; A face collection module is used to detect that the smart TV is started and control the camera set on the smart TV to collect the face information of the user in front of the smart TV; A new user identification module is used to identify the user's facial information based on an image recognition algorithm and determine whether a new user is identified; A virtual user creation module is used to create a new virtual user account when a new user is identified; and record the operation habit data and playback data of the new user during use, establish an association with the virtual user account, and gradually build a user portrait; A first recommendation control module, for, when recognizing that the current user in front of the smart TV is a user with an existing user profile, calling out personalized content or services corresponding to the user with the existing user profile through a preset multi-user model, performing a first recommendation display; The second recommendation control module further controls the camera to perform motion and expression recognition processing on the current user during the first recommendation display process, and further filters the content of the first recommendation display according to the motion and expression recognition processing results to obtain personalized content or services that match the current user's motion and expression, and perform a second recommendation display.

9. A smart TV, characterized in that: The device comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include being used to execute the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as described in any one of claims 1 to 7.

Images