A method and system for emotional feedback based on content recognition

Abstract

The present invention proposes an emotional feedback method and system based on content recognition. The emotional feedback method based on content recognition includes: when a user browses online content, collecting the user's eye data information and facial data information in real time to obtain the user's eye data information and facial data information when browsing online content; making a primary emotional judgment on the user's browsing of online content based on the user's eye data information when browsing online content to obtain a primary emotional judgment result; judging whether to adjust the data collection frequency of the user's facial data information based on the primary emotional judgment result; when the primary emotional judgment result indicates that the user is interested in the browsed online content, making a secondary emotional judgment using the facial data information to obtain a secondary emotional judgment result; and determining the push ratio of content similar to the online content currently browsed by the user based on the secondary emotional judgment result. The system includes modules corresponding to the steps of the method.

Description

Technical Field

[0001] The present invention proposes an emotion feedback method and system based on content recognition, belonging to the technical field of emotion recognition feedback. Background Art

[0002] With the rapid development of internet technology, browsing online content has become a part of daily life. Understanding users' emotional responses to online content is crucial for improving user experience, optimizing content recommendation algorithms, and achieving more targeted advertising. In recent years, emotion recognition technology, as a key branch of computer vision and natural language processing, has been widely researched and applied. Traditional emotion recognition methods rely on sentiment analysis of textual content. For example, in the social media sector, text-based sentiment analysis is used to identify user emotions and opinions; in the e-commerce sector, user reviews of specific e-commerce products are analyzed to determine user preferences. However, these methods rely solely on textual content, ignoring users' immediate emotional responses during browsing and failing to fully capture their true feelings. To more accurately capture users' emotional feedback while browsing online content, researchers have recently begun exploring biometric-based emotion recognition methods. Eye and facial data, as important biometric features, can reflect users' emotional state in real time. For example, by collecting real-time eye data, it is possible to analyze gaze duration, pupil changes, and other factors, thereby providing a preliminary assessment of the user's interest in the content being browsed. Furthermore, by combining facial data information, such as expression changes and muscle activity, more detailed emotion judgment can be made.

[0003] Several biometric-based emotion recognition methods have been proposed. For example, deep learning techniques can be used to train models to extract emotional features from eye and facial data, enabling emotion recognition. However, these methods still have limitations in their application. For example, how to reduce the workload of data collection and processing while ensuring recognition accuracy, and how to strike a balance between user privacy protection and emotion recognition effectiveness are all pressing issues that need to be addressed. Summary of the Invention

[0004] The present invention provides a method and system for emotional feedback based on content recognition to solve the technical problems existing in the above-mentioned prior art. The technical solutions adopted are as follows:

[0005] A method for providing emotional feedback based on content recognition, comprising:

[0006] When a user browses online content, the user's eye data information and facial data information are collected in real time to obtain the user's eye data information and facial data information when the user browses online content;

[0007] determining a primary emotion of the user browsing the web content based on the eye data information of the user when browsing the web content, and obtaining a primary emotion determination result;

[0008] Determining whether to adjust the frequency of data collection of the user's facial data information according to the primary emotion determination result;

[0009] When the primary emotion determination result indicates that the user is interested in the browsed web content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result;

[0010] The push ratio of content similar to the web content currently browsed by the user is determined based on the secondary emotion determination result.

[0011] Furthermore, when the user browses the network content, the user's eye data information and facial data information are collected in real time, and the eye data information and facial data information of the user when browsing the network content are obtained, including:

[0012] When the user browses the network content, the initial eye data collection frequency corresponding to the eye data information collection and the initial facial data collection frequency corresponding to the facial data information are retrieved from the database;

[0013] Collecting eye data information of the user when browsing online content according to the initial eye data collection frequency to obtain the eye data information of the user when browsing online content;

[0014] The eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blink frequency corresponding to each fixation point;

[0015] Collecting facial data information of the user when browsing online content according to the initial facial data collection frequency to obtain facial data information of the user when browsing online content;

[0016] The facial data information includes coordinate change data corresponding to the eyebrow mark point, coordinate change data corresponding to the mouth mark point, and coordinate change data corresponding to the eye mark point.

[0017] Furthermore, a primary emotion determination result of the user browsing the network content is obtained based on the eye data information of the user when browsing the network content, including:

[0018] Extracting eye data information of the user when browsing online content; wherein the eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blink frequency corresponding to each fixation point;

[0019] Obtaining a content browsing coefficient using a scanning speed of a gaze point transfer process included in eye data information of the user when browsing network content;

[0020] The content browsing coefficient is obtained by the following formula:

[0021]

[0022] in, C Indicates the content browsing coefficient; n Indicates the total number of fixations; S ci Indicates the i The rate of outgoing gaze points; S ri Indicates the i The turn-in rate corresponding to each fixation point; V i Indicates the i The value of the visual complexity of the area corresponding to each fixation point; T si Indicates the i The fixation point corresponds to the repetition rate; V max express n The maximum value of the visual complexity of the area corresponding to the fixation point; S rmax express n The maximum value of the turn-in rate corresponding to the fixation point; S rm express The maximum value of visual complexity corresponds to The rate of transition into the area corresponding to the fixation point ;

[0023] Comparing the content browsing coefficient with a preset browsing coefficient threshold;

[0024] When the content browsing coefficient is lower than a preset browsing coefficient threshold, the primary emotion judgment of the user browsing the network content is made using the gaze point dwell time and the blinking frequency corresponding to each gaze point contained in the eye data information to obtain a primary emotion judgment result.

[0025] Furthermore, the primary emotion judgment of the user browsing the web content is made using the duration of the gaze point and the blinking frequency corresponding to each gaze point included in the eye data information, to obtain a primary emotion judgment result, including:

[0026] When the content browsing coefficient is lower than a preset browsing coefficient threshold, the gaze dwell time corresponding to each gaze point and the blink frequency corresponding to each gaze point included in the eye data information are retrieved;

[0027] Retrieving the content browsing coefficient;

[0028] Obtaining an initial emotion determination coefficient using the gaze dwell time corresponding to each gaze point and the blink frequency corresponding to each gaze point included in the eye data information in combination with a content browsing coefficient;

[0029] The initial emotion determination coefficient is obtained by the following formula:

[0030]

[0031] in, E represents the initial sentiment determination coefficient; n Indicates the total number of fixations; D i Indicates the i The duration of the fixation point corresponding to each fixation point; F i Indicates the i Blink frequency corresponding to each fixation point; e Indicates the minimum constant for pre-trial, used to prevent the denominator from being 0; F max Indicates the maximum blink frequency of the user during browsing;

[0032] Comparing the initial emotion determination coefficient with a preset determination coefficient threshold;

[0033] When the initial emotion determination coefficient exceeds a preset determination coefficient threshold, it is determined that the user has a preliminary interest in the currently browsed web content;

[0034] When the initial emotion determination coefficient does not exceed the preset determination coefficient threshold, it is determined that the user has no initial interest in the currently browsed network content.

[0035] Furthermore, judging whether to adjust the frequency of data collection of the user's facial data information according to the primary emotion determination result includes:

[0036] Retrieve primary emotion determination results;

[0037] When the primary emotion determination result indicates that the user has no initial interest in the currently browsed web content, the frequency of collecting the user's facial data information is not adjusted;

[0038] When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the frequency of data collection of the user's facial data information is adjusted.

[0039] Furthermore, when the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the frequency of data collection of the user's facial data information is adjusted, including:

[0040] When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the coordinate change data corresponding to the eyebrow mark point and the coordinate change data corresponding to the mouth mark point are retrieved;

[0041] Obtaining the coordinate change frequency and position change amplitude corresponding to each eyebrow marker point according to the coordinate change data corresponding to the eyebrow marker point;

[0042] Obtaining the coordinate change frequency and position change amplitude corresponding to each mouth mark point according to the coordinate change data corresponding to the mouth mark point;

[0043] The first action determination coefficient is obtained by using the coordinate change frequency and position change amplitude corresponding to each eyebrow marker point, wherein the first action determination coefficient is obtained by the following formula:

[0044]

[0045] in, R 01 represents the first action determination coefficient; m Indicates the total number of eyebrow markers; f mi Indicates the i The coordinate change frequency corresponding to each eyebrow marker point; a mi Indicates the i The position change amplitude corresponding to each eyebrow marker point; α express m The angle between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; L express m The distance between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point;

[0046] The second action determination coefficient is obtained by using the coordinate change frequency and position change amplitude corresponding to the mouth mark point, wherein the second action determination coefficient is obtained by the following formula:

[0047]

[0048] in, R 02 represents the second action determination coefficient; k Indicates the total number of mouth landmarks; f ki Indicates the i The coordinate change frequency corresponding to each mouth marker point; a ki Indicates thei The position change amplitude corresponding to each mouth marker point; α express m The angle between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; L express m The distance between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; β express k The angle between the mouth marker point with the minimum position change amplitude and the mouth marker point with the maximum position change amplitude corresponding to the mouth marker point; D express k The distance between the mouth marker point with the minimum position change and the mouth marker point with the maximum position change corresponding to each mouth hair marker point;

[0049] Retrieve the initial emotion determination coefficient;

[0050] adjusting the data collection frequency of the facial data information by using the first action determination coefficient and the second action determination coefficient in combination with the initial emotion determination coefficient, and collecting facial data information according to the adjusted data collection frequency;

[0051] The adjusted data collection frequency is obtained by the following formula:

[0052]

[0053] in, F represents the adjusted data collection frequency; F 0 indicates the adjusted data collection frequency; R 01 represents the first action determination coefficient; R 02 Indicates the second action determination coefficient.

[0054] Furthermore, when the primary emotion determination result indicates that the user is interested in the browsed web content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result, including:

[0055] When the primary emotion determination result indicates that the user is interested in the browsed web content, facial data information is retrieved;

[0056] Performing data preprocessing on the facial data information to obtain preprocessed facial data information; wherein the data preprocessing includes noise reduction processing and invalid value removal processing;

[0057] The pre-processed facial data information is input into a deep learning model for facial expression recognition and expression degree recognition to obtain the facial expression type and the emotion degree corresponding to the facial expression type; wherein the deep learning model adopts a convolutional neural network model.

[0058] Furthermore, the structure of the deep learning model is as follows:

[0059] Input layer, used to receive pre-processed facial data information;

[0060] The convolution layer is used to extract spatial features from the processed facial data information received to obtain spatial features;

[0061] The flattening layer is used to flatten the spatial features of the output of the convolutional layer into a one-dimensional array;

[0062] The fully connected layer is used to integrate the one-dimensional arrays corresponding to the extracted spatial features and learn global information;

[0063] The output layer is used to output the expression recognition results and the corresponding emotional level of the expression;

[0064] The expression recognition results include boredom, indifference, happiness, sadness, anger and surprise; and the emotion levels include normal and strong.

[0065] Furthermore, determining the push ratio of content similar to the web content currently browsed by the user based on the secondary emotion determination result includes:

[0066] If the secondary emotion determination result determines that the user's expression when browsing the current web content is either bored or indifferent, and the emotion level is strong, then similar content to the current web content will not be pushed;

[0067] When the secondary emotion judgment result determines that the user's expression when browsing the current web content is bored or indifferent, and the emotional level is average, the push ratio of content similar to the current web content is set to 3%-8%; that is, the push ratio of content similar to the current web content included in the daily push content is 3%-8%;

[0068] When the secondary emotion judgment result determines that the user's expression when browsing the current web content is any of happiness, sadness, anger, and surprise, and the emotional level is average, the push ratio of content similar to the current web content is set to 8%-15%; that is, the push ratio of content similar to the current web content included in the daily push content is 8%-15%;

[0069] If the secondary emotion assessment results determine that the user's expression when browsing the current web content is any of the following: happiness, sadness, anger, or surprise, and the emotion is strong, the push ratio of content similar to the current web content is set to 15%-28%. That is, the push ratio of content similar to the current web content in the daily push content is 15%-28%;

[0070] A content recognition-based emotional feedback system, comprising:

[0071] A real-time collection module is used to collect the user's eye data information and facial data information in real time when the user is browsing the network content, and obtain the user's eye data information and facial data information when the user is browsing the network content;

[0072] A primary emotion determination module is used to determine the primary emotion of the user browsing the network content based on the eye data information of the user when browsing the network content, and obtain a primary emotion determination result;

[0073] An acquisition frequency adjustment module is used to determine whether to adjust the frequency of data acquisition of the user's facial data information according to the primary emotion determination result;

[0074] A secondary emotion determination module is configured to perform a secondary emotion determination using facial data information to obtain a secondary emotion determination result when the primary emotion determination result indicates that the user is interested in the browsed web content;

[0075] The push setting module is used to determine the push ratio of content similar to the network content currently browsed by the user based on the secondary emotion judgment result.

[0076] Beneficial effects of the present invention:

[0077] The present invention proposes an emotion feedback method and system based on content recognition, which collects the user's eye data information and facial data information in real time when the user is browsing online content, and performs a primary emotion judgment on the user based on this information. If the primary emotion judgment result shows that the user is interested in the browsed online content, the facial data information is further used to perform a secondary emotion judgment to obtain more accurate emotion feedback. According to the secondary emotion judgment result, the proportion of content push that is similar to the online content currently browsed by the user can be determined, thereby achieving more accurate content recommendation. The present invention proposes an emotion feedback method and system based on content recognition, which not only overcomes the limitation of traditional emotion recognition methods that rely only on text content, but also realizes the comprehensive capture and accurate recognition of the user's emotional state through real-time collection and processing of biometric data. At the same time, by dynamically adjusting the data collection frequency and optimizing the emotion judgment algorithm, the accuracy and efficiency of emotion recognition are improved, providing strong support for improving user experience and optimizing content recommendation algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A flow chart of the method of the present invention;

[0079] Figure 2 This is a system block diagram of the system of the present invention. DETAILED DESCRIPTION

[0080] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0081] The embodiment of the present invention proposes an emotional feedback method based on content recognition, such as Figure 1 As shown, the emotional feedback method based on content recognition includes:

[0082] S1. When a user browses online content, the user's eye data information and facial data information are collected in real time to obtain the user's eye data information and facial data information when the user browses online content;

[0083] S2. determining a primary emotion of the user browsing the web content based on the eye data information of the user when browsing the web content, and obtaining a primary emotion determination result;

[0084] S3, determining whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result;

[0085] S4. When the primary emotion determination result indicates that the user is interested in the browsed web content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result;

[0086] S5. Determine the push ratio of content similar to the web content currently browsed by the user based on the secondary emotion determination result.

[0087] The working principle of the above technical solution is as follows: When a user browses content on a web platform, the system collects eye and facial data in real time. This data includes information such as gaze direction, pupil size changes, and facial expression changes, which can reflect the user's emotional response and level of interest during browsing. Based on the collected eye data, such as gaze duration and pupil changes, the system determines the user's primary emotion while browsing the web content. For example, prolonged gaze and pupil dilation may indicate interest in the content, while rapid glances and pupil constriction may indicate disinterest or boredom. Based on the results of the primary emotion determination, the system dynamically adjusts the frequency of facial data collection. If the user expresses strong interest in the content, the system increases the data collection frequency to more accurately capture the user's emotional changes. Conversely, if the user expresses disinterest or boredom, the system appropriately reduces the data collection frequency to reduce unnecessary resource consumption. If the primary emotion determination indicates that the user is interested in the content, the system further utilizes the facial data information for secondary emotion determination. This assessment is more detailed and in-depth, analyzing facial expressions, muscle movement, and other factors to more accurately determine the user's emotional state. Based on this secondary emotional assessment, the system determines the proportion of content similar to the content the user is currently viewing. If the user expresses strong interest and a positive emotional response, the system will increase the proportion of similar content recommended to meet their needs. Conversely, if the user expresses disinterest or a negative emotional response, the system will reduce the proportion of similar content recommended to prevent the user from becoming bored.

[0088] The effect of the above technical solution is: by collecting and analyzing the user's eye and facial data information in real time, the system can more accurately capture the user's emotional response and interest level, thereby providing users with more personalized content recommendations and a better service experience. This technical solution can dynamically adjust the content recommendation algorithm based on the user's real-time emotional feedback to make it more in line with the user's interests and needs. This helps to improve the accuracy of content recommendations and user satisfaction. At the same time, by dynamically adjusting the data collection frequency, the system can reduce unnecessary resource consumption and data collection while ensuring recognition accuracy. The application of this technical solution helps to promote human-computer interaction towards a more intelligent and personalized direction. By analyzing the user's emotional response and interest level in real time, the system can provide users with a more intelligent and caring service experience.

[0089] In one embodiment of the present invention, when a user browses online content, eye data information and facial data information of the user are collected in real time, and the eye data information and facial data information of the user when browsing online content are obtained, including:

[0090] S101, when a user is browsing online content, retrieving from a database an initial eye data collection frequency corresponding to eye data information collection and an initial facial data collection frequency corresponding to facial data information;

[0091] S102: collecting eye data information of the user when browsing online content according to the initial eye data collection frequency to obtain the eye data information of the user when browsing online content;

[0092] The eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blink frequency corresponding to each fixation point;

[0093] S103, collecting facial data information of the user when browsing online content according to the initial facial data collection frequency, to obtain facial data information of the user when browsing online content;

[0094] The facial data information includes coordinate change data corresponding to the eyebrow mark point, coordinate change data corresponding to the mouth mark point, and coordinate change data corresponding to the eye mark point.

[0095] The working principle of this technical solution is as follows: When a user begins browsing online content, the system first retrieves the initial eye data collection frequency corresponding to eye data information and the initial facial data collection frequency corresponding to facial data information from a preset database. These initial frequencies are derived from statistical analysis of a large amount of user data and are designed to balance data collection accuracy and system resource consumption.

[0096] Based on the initial eye data collection frequency, the system collects real-time eye data from the user. This data includes the duration of gaze dwell, the speed of gaze shifts, and the blink rate corresponding to each gaze point. This data can reveal key information such as the user's attention allocation, reading speed, and eye fatigue during browsing.

[0097] Specifically,

[0098] Gaze dwell time: reflects the user's attention to a certain content point. The longer the dwell time, the more interested the user is in the content.

[0099] Scan speed: reflects the speed at which users browse content. A faster speed may indicate that users are browsing quickly to find content of interest.

[0100] Blinking frequency: It can indirectly reflect the user's eye fatigue level. A high blinking frequency may mean that the user needs a break.

[0101] At the same time, the system also collects real-time facial data from the user, based on the initial facial data collection frequency. This facial data includes coordinate changes corresponding to the eyebrow markers, mouth markers, and eye markers. This data can capture subtle changes in the user's facial expressions and infer their emotional state.

[0102] Specifically:

[0103] Changes in eyebrow marker coordinates: can reflect the user's emotional changes such as surprise and confusion.

[0104] Changes in the coordinates of the mouth marker: This can capture the user's smile, frown, and other expressions, and then determine the user's emotional state.

[0105] Changes in eye marker coordinates: In addition to analyzing eye fatigue in combination with blink frequency, changes in eye expression, such as staring and wandering, can also be captured to determine the user's level of concentration.

[0106] The above technical solution achieves the following: By collecting and analyzing users' eye and facial data in real time, the system can more accurately understand users' emotional reactions and interests during browsing, thereby providing more personalized content recommendations and services. This helps improve user satisfaction and loyalty. By combining eye and facial data, the system can more accurately determine users' preferences and interests in content, thereby optimizing content recommendation algorithms and improving the accuracy and relevance of recommendations. This helps increase content click-through rates and conversion rates. Although the system requires collecting users' eye and facial data, all collected data is strictly encrypted and processed to ensure user privacy. Furthermore, the system dynamically adjusts the scope and frequency of data collection based on user privacy settings and feedback to maximize user privacy protection. By analyzing users' eye and facial data in real time, the system can more intelligently understand users' intentions and emotional states, thereby providing a more intelligent and user-friendly interactive experience. This helps improve the efficiency and satisfaction of human-computer interaction. In the advertising sector, by capturing and analyzing users' eye and facial data in real time, the system can more accurately determine users' interest in and reactions to ads, thereby optimizing advertising strategies and increasing ad exposure and conversion rates. In addition to its application in online content browsing, this technical solution can also be used in the field of mental health monitoring. By analyzing users' eye and facial data in real time, the system can promptly detect abnormal emotions and mental health issues, providing strong support for mental health intervention. The successful application of this technical solution will promote technological innovation and development in related fields. For example, in fields such as artificial intelligence and computer vision, continuous optimization of algorithms and models can further enhance the ability to collect and analyze eye and facial data, providing technical support for more application scenarios. For online platforms, the application of this technical solution can provide users with a more personalized and intelligent service experience, thereby enhancing their market competitiveness. This will help attract more users to join and use the platform, laying a solid foundation for its long-term development.

[0107] To summarize, this technical solution achieves accurate capture and interpretation of user emotional responses by real-time collection and analysis of user eye and facial data information, bringing significant technical effects in improving user experience, optimizing content recommendation algorithms, enhancing user privacy protection, and promoting intelligent human-computer interaction. Among them, the data collection and analysis process needs to be collected and used with the user's permission.

[0108] In one embodiment of the present invention, determining a primary emotion of a user browsing online content based on eye data information of the user while browsing online content, and obtaining a primary emotion determination result, includes:

[0109] S201, extracting eye data information of the user when browsing online content; wherein the eye data information includes the duration of the fixation point, the scanning speed during the fixation point shift process, and the blink frequency corresponding to each fixation point;

[0110] S202, obtaining a content browsing coefficient by using a scanning speed of a gaze point transfer process included in the eye data information of the user when browsing web content;

[0111] The content browsing coefficient is obtained by the following formula:

[0112]

[0113] in, C Indicates the content browsing coefficient; n Indicates the total number of fixations; S ci Indicates the i The rate of outgoing gaze points; S ri Indicates the i The turn-in rate corresponding to each fixation point; V i Indicates the i The value of the visual complexity of the area corresponding to each fixation point; T si Indicates the i The fixation point corresponds to the repetition rate; V max express n The maximum value of the visual complexity of the area corresponding to the fixation point; S rmax express n The maximum value of the turn-in rate corresponding to the fixation point; S rm express The maximum value of visual complexity corresponds to The rate of transition into the area corresponding to the fixation point ;

[0114] S203, comparing the content browsing coefficient with a preset browsing coefficient threshold;

[0115] S204. When the content browsing coefficient is lower than a preset browsing coefficient threshold, the primary emotion judgment of the user browsing the network content is made using the gaze point dwell time and the blinking frequency corresponding to each gaze point contained in the eye data information to obtain a primary emotion judgment result.

[0116] The working principle of the above technical solution is to use eye tracking technology to capture and record real-time eye data of users while browsing online content. This data includes gaze dwell duration, saccade speed during gaze shifts, and blink frequency corresponding to each gaze point. Based on the extracted eye data, particularly the saccade speed during gaze shifts, a content browsing coefficient is calculated. The content browsing coefficient is a comprehensive indicator that takes into account multiple factors, such as gaze exit rate, gaze entry rate, visual complexity, and re-gaze rate. By weighting and normalizing these factors, a coefficient value is obtained that reflects the efficiency and depth of user content browsing. The calculated content browsing coefficient is then compared with a preset browsing coefficient threshold. The preset browsing coefficient threshold is derived from extensive user data and analysis and is used to determine the user's primary emotional orientation towards content browsing. If the content browsing coefficient falls below the preset browsing coefficient threshold, it is considered that the user is likely uninterested in or confused by the content being browsed. In this case, the eye data, including gaze dwell duration and blink frequency corresponding to each gaze point, is used for further analysis. Through comprehensive processing of this information, we can make primary emotional judgments about users browsing online content and output the judgment results.

[0117] The above technical solution provides a more comprehensive reflection of a user's emotional state by combining multiple dimensions of eye data (such as gaze dwell duration, saccade speed, and blink frequency). Compared to single-dimensional data, this multi-dimensional analysis method can more accurately identify a user's primary emotional tendencies. This technical solution monitors users' browsing behavior and emotional state in real time, providing strong support for personalized recommendation systems. Based on users' emotional tendencies and interests, the system can intelligently adjust recommended content, improving user satisfaction and loyalty. By analyzing users' eye data, it is possible to promptly identify potential problems and confusions that users may encounter during browsing. This helps website or app developers optimize page layouts and adjust content presentation, thereby enhancing user browsing experience and satisfaction. This technical solution provides new ideas and methods for the field of human-computer interaction, helping to promote the intelligent development of human-computer interaction technology. By monitoring and analyzing users' emotional states in real time, the system can more intelligently respond to user needs and instructions, improving the efficiency and naturalness of human-computer interaction.

[0118] This technical solution involves extensive data processing and analysis, requiring the use of advanced algorithms and techniques. This will help enhance data processing and analysis capabilities, driving technological progress and innovation in related fields. In practical applications, this technical solution must ensure the security and privacy of user data. This technical solution has a wide range of applications and scenarios, such as online education, e-commerce, and virtual reality. In these areas, real-time monitoring and analysis of users' emotional states can provide personalized services and support for teaching, shopping, entertainment, and other areas, improving user experience and satisfaction.

[0119] Furthermore, by comprehensively considering user eye data (such as gaze dwell duration, saccade speed, and blink frequency), the above-mentioned technical solution can more comprehensively reflect users' physiological responses when browsing online content, thereby more accurately judging their primary emotions. In particular, the introduction of a content browsing coefficient, which integrates multiple factors such as saccade speed, visual complexity, and re-gaze rate during the gaze shift process, can more precisely characterize user browsing behavior and improve the accuracy of emotion assessment. This solution can capture and analyze user eye data in real time, allowing for timely adjustments to online content presentation and recommendation strategies to meet users' personalized needs and enhance the user experience. Furthermore, this solution can reduce unnecessary computational overhead and improve algorithm efficiency. It can also adapt to different users' browsing habits and emotional expressions. Since each person's eye data and emotional expressions vary, this solution, by comprehensively considering multiple eye data points, can more flexibly adapt to the characteristics of different users.

[0120] In summary, this technical solution extracts and analyzes user eye data while browsing online content, enabling a preliminary assessment of users' emotions. This solution has significant technical effectiveness and application value, enabling users to experience a more personalized and intelligent service experience.

[0121] In one embodiment of the present invention, the primary emotion judgment of a user browsing web content is performed using the duration of a gaze point and the blink frequency corresponding to each gaze point included in the eye data information, and the primary emotion judgment result is obtained, including:

[0122] S2041: When the content browsing coefficient is lower than a preset browsing coefficient threshold, retrieving the gaze dwell time corresponding to each gaze point and the blink frequency corresponding to each gaze point included in the eye data information;

[0123] S2042, retrieve the content browsing coefficient;

[0124] S2043, using the gaze dwell time corresponding to each gaze point and the blink frequency corresponding to each gaze point included in the eye data information in combination with the content browsing coefficient to obtain an initial emotion determination coefficient;

[0125] The initial emotion determination coefficient is obtained by the following formula:

[0126]

[0127] in, E represents the initial sentiment determination coefficient; n Indicates the total number of fixations; D i Indicates the i The duration of the fixation point corresponding to each fixation point; F i Indicates the i Blink frequency corresponding to each fixation point; e Indicates the minimum constant for pre-trial, used to prevent the denominator from being 0; F max Indicates the maximum blink frequency of the user during browsing;

[0128] S2044, comparing the initial emotion determination coefficient with a preset determination coefficient threshold;

[0129] S2045: When the initial emotion determination coefficient exceeds a preset determination coefficient threshold, it is determined that the user has a preliminary interest in the currently browsed web content;

[0130] S2046: When the initial emotion determination coefficient does not exceed the preset determination coefficient threshold, it is determined that the user has no initial interest in the currently browsed network content.

[0131] The working principle of the above technical solution is as follows: when the content browsing coefficient falls below a preset browsing coefficient threshold, the system determines that the user may have some emotional inclination towards the currently viewed content and requires further analysis. At this point, the system retrieves the gaze dwell duration and blink frequency corresponding to each fixation point from the eye data information. This information can reflect the user's attention and emotional state while browsing the content. Although the content browsing coefficient has already been calculated in the previous step, it is retrieved again in this step to be combined with the subsequently calculated initial emotion determination coefficient for a more comprehensive assessment of the user's emotional state. The initial emotion determination coefficient is calculated using a specific formula using the gaze dwell duration and blink frequency corresponding to each fixation point, combined with the content browsing coefficient. This coefficient is a comprehensive indicator that takes into account multiple factors such as the user's attention allocation, emotional response, and browsing efficiency while browsing the content. The calculated initial emotion determination coefficient is then compared with a preset determination coefficient threshold. This threshold is derived from extensive user data and analysis and is used to determine whether the user has initially expressed interest in the currently viewed content. Based on the comparison results, if the initial sentiment determination coefficient exceeds the preset determination coefficient threshold, it is determined that the user has initially shown interest in the currently viewed web content; if it does not exceed the threshold, it is determined that the user has not shown initial interest. This result can provide strong support for subsequent content recommendations and user experience optimization.

[0132] The above technical solution has the following benefits: By combining multiple dimensions, such as gaze dwell duration, blink frequency, and content browsing coefficient, it can more accurately identify users' emotional states while browsing online content. This multi-dimensional analysis method is more comprehensive and reliable than single-dimensional data. Based on the user's emotional assessment results, the system can intelligently adjust content recommendation strategies to provide users with content that better suits their interests and needs. This helps improve user browsing experience and satisfaction, and enhances user stickiness and loyalty. By monitoring and analyzing users' emotional states in real time, the system can promptly detect changes in user interest in content and quickly adjust recommended content. This helps improve the efficiency and accuracy of content recommendations, reducing ineffective recommendations and wasted resources. This technical solution provides new ideas and methods in the field of human-computer interaction, helping to promote the intelligent development of human-computer interaction technology. By monitoring and analyzing users' emotional states in real time, the system can more intelligently respond to user needs and instructions, improving the efficiency and naturalness of human-computer interaction. This technical solution can also provide strong support for psychological research and applications. By analyzing user eye data, we can gain a deeper understanding of human emotional states and cognitive processes, providing new perspectives and methods for psychological research. This technical solution has a wide range of applications and scenarios, including online education, e-commerce, and virtual reality. By monitoring and analyzing users' emotional states in real time, it can provide personalized services and support for teaching, shopping, entertainment, and other fields, improving user experience and satisfaction. The successful application of this technical solution will drive technological innovation and development in related fields. For example, in fields such as artificial intelligence and computer vision, continuous optimization of algorithms and models can further enhance the accuracy and efficiency of emotion recognition, providing technical support for a wider range of application scenarios.

[0133] By integrating two key metrics, gaze dwell duration and blink frequency, with the content browsing coefficient, this solution can more meticulously capture the emotional changes of users as they browse online content. This comprehensive, multi-dimensional approach makes the emotion determination results more accurate and refined. The solution only further analyzes gaze dwell duration and blink frequency when the content browsing coefficient falls below a preset threshold, which helps reduce unnecessary computation and improves determination efficiency. Furthermore, using the preset determination coefficient threshold, the user's interest in the current content can be quickly determined, reducing judgment time. Refined emotion determination results can provide a basis for personalized recommendations and content optimization, thereby enhancing the user experience. For example, if a user demonstrates initial interest in the current content, more relevant content can be recommended; if the user demonstrates no interest, the content presentation can be adjusted or alternative content can be recommended. This solution is adaptable to different users' browsing habits and emotional expressions. Since browsing behavior and blink frequency vary from person to person, this solution, by comprehensively considering multiple metrics, can more flexibly adapt to the characteristics of different users, improving the system's adaptability. This solution offers a new approach and method for the field of affective computing. By combining eye data with parameters such as the content browsing coefficient, it enables quantitative analysis and determination of user emotions. This will help further develop affective computing technology and provide new insights and references for research and application in related fields.

[0134] By introducing a minimum constant e for pre-screening, this solution effectively prevents the denominator from being zero, enhancing the robustness and stability of the algorithm. Furthermore, by considering the maximum blink frequency Fmax that occurs during user browsing, the judgment results are made more reasonable and reliable.

[0135] In summary, the technical benefits of this solution in terms of performance indicators are primarily reflected in refined emotion assessment, improved assessment efficiency, optimized user experience, enhanced system adaptability, promotion of the development of affective computing technology, and robustness and stability. These technical benefits give this solution greater practical value and potential for widespread adoption. Furthermore, by combining eye data such as gaze dwell duration, blink frequency, and content browsing coefficient, this solution enables primary emotional assessment of users' online content. This technical solution has significant technical benefits and application value, providing users with a more personalized and intelligent service experience and promoting technological innovation and development in related fields.

[0136] In one embodiment of the present invention, determining whether to adjust the frequency of data collection of the user's facial data information based on the primary emotion determination result includes:

[0137] S301, retrieve the primary emotion determination result;

[0138] S302: when the primary emotion determination result indicates that the user has no initial interest in the currently browsed web content, then the frequency of collecting the user's facial data information is not adjusted;

[0139] S303: When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the frequency of data collection of the user's facial data information is adjusted.

[0140] The technical solution works as follows: First, the system retrieves the primary emotion determination results obtained through eye data analysis. This result is calculated based on the user's gaze duration and blink frequency on the current online content, reflecting the user's initial interest in the content.

[0141] Next, the system determines the user's interest in the content based on the initial emotion determination results. If the user does not show initial interest, the system considers the current data collection frequency sufficient and no adjustment is required to maintain efficient resource utilization and reduce unnecessary data processing. Conversely, if the user shows initial interest, the system determines that more detailed facial data capture is necessary to more deeply analyze the user's emotions and reactions. Therefore, the system increases the data collection frequency accordingly to obtain more and more accurate facial data.

[0142] The above technical solution achieves the following benefits: By adjusting the data collection frequency based on the user's level of interest, it enables more efficient utilization of system resources. When the user is not interested in the content, reducing the collection frequency reduces data storage and processing requirements, thereby reducing system load and energy consumption. However, when the user is interested, increasing the collection frequency ensures the capture of more useful facial data, providing strong support for subsequent in-depth analysis and applications. This technical solution also enhances the user experience. By analyzing the user's emotional state in real time and adjusting the data collection strategy, the system can more intelligently respond to user needs. For example, when a user becomes interested in a piece of content, the system can immediately increase the data collection frequency to more accurately capture the user's facial reactions and emotional changes. This helps the system gain a deeper understanding of the user, thereby providing more personalized and attentive services. Increasing the data collection frequency ensures the acquisition of more and more accurate facial data information. This information is crucial for subsequent applications such as sentiment analysis and user behavior prediction. By increasing the amount of data and improving data quality, this technical solution can significantly improve the accuracy and reliability of emotion recognition, providing stronger support for related applications. The successful application of this technical solution will promote technological innovation and development in related fields. For example, in fields like artificial intelligence and computer vision, the efficiency and accuracy of emotion recognition can be further improved by continuously optimizing data collection and analysis algorithms. This technical solution can also provide new ideas and methods for applications in other fields, promoting cross-disciplinary technological innovation and collaboration. Furthermore, this technical solution can provide strong support for psychological research and applications. By analyzing users' facial data in real time, a deeper understanding of human emotional states and cognitive processes can be achieved. This helps psychologists more accurately understand human emotions and behavioral patterns, providing new perspectives and methods for applications such as psychological counseling and psychotherapy. This technical solution is highly adaptable. It can flexibly adjust the data collection frequency based on the interests and needs of different users, thereby meeting the needs of different application scenarios. This flexibility makes this technical solution widely applicable in fields such as online education, e-commerce, and virtual reality.

[0143] In summary, this technical solution optimizes resource utilization, improves user experience, enhances data accuracy and reliability, and promotes technological innovation and development by adjusting the frequency of user facial data collection based on primary emotion determination results. These technical effects give this solution broad application prospects and significant practical value.

[0144] In one embodiment of the present invention, when the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the frequency of collecting the user's facial data information is adjusted, including:

[0145] S3031. When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, retrieve the coordinate change data corresponding to the eyebrow mark point and the coordinate change data corresponding to the mouth mark point;

[0146] S3032. Obtaining a coordinate change frequency and a position change amplitude corresponding to each eyebrow marker point according to the coordinate change data corresponding to the eyebrow marker point;

[0147] S3033. Obtain the coordinate change frequency and position change amplitude corresponding to each mouth mark point according to the coordinate change data corresponding to the mouth mark point;

[0148] S3034. Obtain a first action determination coefficient using the coordinate change frequency and position change amplitude corresponding to each eyebrow marker point, wherein the first action determination coefficient is obtained by the following formula:

[0149]

[0150] in, R 01 represents the first action determination coefficient; m Indicates the total number of eyebrow markers; f mi Indicates the i The coordinate change frequency corresponding to each eyebrow marker point; a mi Indicates the i The position change amplitude corresponding to each eyebrow marker point; α express m The angle between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; L express m The distance between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point;

[0151] S3035. Obtain a second action determination coefficient using the coordinate change frequency and position change amplitude corresponding to the mouth marker point, wherein the second action determination coefficient is obtained by the following formula:

[0152]

[0153] in, R 02 represents the second action determination coefficient; k Indicates the total number of mouth landmarks; f ki Indicates the i The coordinate change frequency corresponding to each mouth marker point; aki Indicates the i The position change amplitude corresponding to each mouth marker point; α express m The angle between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; L express m The distance between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; β express k The angle between the mouth marker point with the minimum position change amplitude and the mouth marker point with the maximum position change amplitude corresponding to the mouth marker point; D express k The distance between the mouth marker point with the minimum position change and the mouth marker point with the maximum position change corresponding to each mouth hair marker point;

[0154] S3036, retrieve the initial emotion determination coefficient;

[0155] S3037: Adjust the data collection frequency of the facial data information by using the first action determination coefficient and the second action determination coefficient in combination with the initial emotion determination coefficient, and collect facial data information according to the adjusted data collection frequency;

[0156] The adjusted data collection frequency is obtained by the following formula:

[0157]

[0158] in, F represents the adjusted data collection frequency; F 0 indicates the adjusted data collection frequency; R 01 represents the first action determination coefficient; R 02 Indicates the second action determination coefficient.

[0159] The working principle of the above technical solution is as follows: When the primary emotion determination results indicate that the user has a preliminary interest in the currently viewed online content, the system first retrieves coordinate change data for the eyebrow and mouth markers. This data reflects the subtle movements of the user's facial muscles when expressing emotions and serves as an important basis for subsequent analysis. The system further analyzes the coordinate change data for the eyebrow and mouth markers, extracting the coordinate change frequency and position change amplitude of each marker. These features can quantify the dynamic changes in the user's face when expressing emotions, providing key data for subsequent action determination. Using the extracted eyebrow and mouth movement features, the system calculates the first and second action determination coefficients, respectively. These coefficients comprehensively consider factors such as the coordinate change frequency and position change amplitude of the markers, as well as the relative positions and angles between the markers, to comprehensively reflect the overall dynamic characteristics of the user's face when expressing emotions. The system also retrieves the previously calculated initial emotion determination coefficient, which reflects the user's initial interest in the currently viewed content. Finally, the system combines the first and second action determination coefficients with the initial emotion determination coefficient to calculate the adjusted data collection frequency using a specific formula. This frequency can more accurately capture the subtle facial movements of users when expressing emotions, providing stronger support for subsequent sentiment analysis and applications.

[0160] The above technical solution has the following benefits: By deeply analyzing the user's facial data, particularly the coordinate changes of the eyebrow and mouth landmarks, this solution can more accurately capture subtle facial movements during emotional expression. This helps improve the accuracy of data collection and provides stronger data support for subsequent sentiment analysis and applications. This technical solution can dynamically adjust the data collection frequency based on the user's emotional state and facial movement characteristics. This dynamic adjustment strategy helps optimize data collection efficiency and improve resource utilization. By comprehensively considering the movement characteristics of the eyebrows and mouth, as well as the initial emotion determination coefficient, this technical solution can more comprehensively assess the user's emotional state. This helps improve the accuracy of emotion recognition and provide more reliable results for subsequent sentiment analysis and applications. This technical solution can more accurately capture the user's emotional state and facial movement characteristics, thereby providing users with more personalized and attentive services. For example, in the field of online education, the system can adjust the teaching content and pace based on the student's emotional state to improve learning outcomes and satisfaction. In the field of e-commerce, the system can recommend products and services that better meet the user's needs and interests based on the user's emotional response. This technical solution can also provide strong support for psychological research and applications. By deeply analyzing users' facial data, psychologists can more accurately understand human emotional states and cognitive processes, providing new perspectives and methods for applications such as psychological counseling and psychotherapy. The successful application of this technical solution will drive technological innovation and development in related fields. For example, in fields such as artificial intelligence and computer vision, continuous optimization of data collection and analysis algorithms can further improve the efficiency and accuracy of emotion recognition. This technical solution can also provide new ideas and methods for applications in other fields, promoting cross-disciplinary technological innovation and collaboration. This technical solution is highly adaptable. It can flexibly adjust data collection frequency and strategies based on different users' emotional states and facial movement characteristics to meet the application needs of different scenarios. This flexibility makes this technical solution widely applicable in fields such as online education, e-commerce, and virtual reality.

[0161] Furthermore, by dynamically adjusting the frequency of facial data collection based on the user's initial interest (determined by the primary emotion determination results), this solution increases data collection density when the user expresses greater interest, thereby more accurately capturing the user's emotional responses and subtle movements. This dynamic adjustment helps improve data relevance and accuracy while maintaining data processing efficiency. By introducing the first and second motion determination coefficients, the solution comprehensively considers geometric factors such as the frequency and magnitude of coordinate changes of the eyebrows and mouth, as well as their relative position and angle. This precise calculation method more accurately reflects the user's facial movement characteristics, providing a more reliable foundation for subsequent emotion analysis and data processing. By combining the initial emotion determination coefficient, the first and second motion determination coefficients to adjust the data collection frequency, the solution achieves a coordinated optimization between emotion determination and data collection frequency. This coordinated optimization helps improve the relevance and effectiveness of data collection while maintaining user privacy and data security. By dynamically adjusting the data collection frequency based on the user's interest, the solution reduces unnecessary data collection and processing when user interest is low, thereby saving computing resources and storage space. At the same time, increasing data collection density when users are most interested helps more accurately capture and analyze users' emotional responses, improving the system's overall performance and user experience. This solution introduces adjustable parameters and formulas to calculate action determination coefficients and data collection frequency, enabling the system to be flexibly configured and optimized based on different application scenarios and user needs.

[0162] In summary, this technical solution achieves multiple technical benefits in terms of performance indicators, including dynamic adjustment of data collection frequency, precise calculation of action determination coefficients, coordinated optimization of emotion determination and data collection frequency, performance improvement and resource optimization, and adaptability and scalability. Furthermore, by deeply analyzing the user's facial data and dynamically adjusting the data collection frequency based on the user's emotional state and facial movement characteristics, this technical solution improves data collection accuracy, optimizes efficiency, and enhances emotion recognition accuracy. These technical benefits give this technical solution broad application prospects and significant practical value. Furthermore, this technical solution provides strong support for psychological research and applications, promoting technological innovation and development in related fields.

[0163] In one embodiment of the present invention, when the primary emotion determination result indicates that the user is interested in the browsed web content, a secondary emotion determination is performed using facial data information to obtain a secondary emotion determination result, including:

[0164] S401, when the primary emotion determination result indicates that the user is interested in the browsed web content, retrieving facial data information;

[0165] S402, performing data preprocessing on the facial data information to obtain preprocessed facial data information; wherein the data preprocessing includes noise reduction processing and invalid value removal processing;

[0166] S403. Input the pre-processed facial data information into a deep learning model to perform facial expression recognition and expression degree recognition, and obtain the facial expression type and the emotion degree corresponding to the facial expression type; wherein the deep learning model adopts a convolutional neural network model.

[0167] The structure of the deep learning model is as follows:

[0168] Input layer, used to receive pre-processed facial data information;

[0169] The convolution layer is used to extract spatial features from the processed facial data information received to obtain spatial features;

[0170] The flattening layer is used to flatten the spatial features of the output of the convolutional layer into a one-dimensional array;

[0171] The fully connected layer is used to integrate the one-dimensional arrays corresponding to the extracted spatial features and learn global information;

[0172] The output layer is used to output the expression recognition results and the corresponding emotional level of the expression;

[0173] The expression recognition results include boredom, indifference, happiness, sadness, anger and surprise; and the emotion levels include normal and strong.

[0174] The above technical solution works as follows: First, the system uses primary emotion assessment methods (e.g., analysis based on user behavior, click-through rate, and dwell time) to determine whether the user is interested in the content being viewed. If the user expresses interest, the system immediately retrieves the user's facial data, providing the basis for subsequent emotion assessment.

[0175] After acquiring facial data, the system preprocesses it to eliminate noise and invalid values, improving data accuracy and reliability. This preprocessing step includes noise reduction and invalid value removal, ensuring that subsequent analysis is based on high-quality data.

[0176] The pre-processed facial data is then fed into a deep learning model for emotion determination. This model, which uses a convolutional neural network (CNN) architecture, automatically extracts spatial features from facial images and learns global information through fully connected layers, ultimately outputting the expression recognition result and the corresponding emotional level.

[0177] Input layer: Receives preprocessed facial data information as input to the model.

[0178] Convolutional layer: extracts spatial features from the input facial data information and captures key features in the facial image through convolution operations.

[0179] Flattening layer: Flattens the spatial features output by the convolutional layer into a one-dimensional array for subsequent processing.

[0180] Fully connected layer: Integrates the flattened features, learns global information, and outputs expression recognition results and preliminary judgments on the degree of emotion.

[0181] Output layer: Based on the output of the fully connected layer, the user's facial expression type (such as boredom, indifference, happiness, sadness, anger, surprise) and the corresponding emotional level (such as normal, strong) are finally determined.

[0182] The above technical solution achieves the following: Through automatic feature extraction and global information learning using a deep learning model, this solution can more accurately identify a user's facial expression type and emotional intensity. Compared to traditional rule-based or manual feature extraction methods, deep learning models have greater adaptability and robustness, and can handle complex and changing facial expressions and emotions. This technical solution can capture a user's emotional state in real time and adjust the presentation or recommendation strategy of online content based on the user's interest level and emotional response. For example, when a user expresses strong interest or joy, the system can recommend more similar or related content; whereas, when a user expresses boredom or dissatisfaction, the system can adjust the content or provide alternative entertainment services. This personalized service can significantly improve user experience and satisfaction. This technical solution can provide strong support for sentiment analysis applications. By deeply analyzing a user's facial expressions and emotional intensity, the system can understand the user's emotional tendencies and preferences, providing valuable data and insights for sentiment analysis, user profiling, market research, and other fields. The successful application of this technical solution demonstrates the enormous potential of deep learning models in the field of emotion assessment. With the continuous advancement of technology and the in-depth expansion of its applications, deep learning models will play a vital role in more fields and promote the overall development of artificial intelligence technology. By capturing and analyzing users' facial expressions and emotional levels in real time, this technical solution can provide more intelligent and humane services for human-computer interaction. For example, in virtual reality or augmented reality applications, the system can adjust the behavior or expression of the virtual character according to the user's emotional state, enhancing the user's immersion and interactivity. The introduction of data preprocessing steps, such as noise reduction and invalid value removal, can significantly improve the efficiency and accuracy of subsequent emotional judgment. By eliminating noise and invalid values, the system can focus more on useful facial data information and improve the accuracy and stability of emotional judgment. This technical solution can also provide support for the fields of emotional education and mental health. By analyzing the user's facial expressions and emotional level, educators or psychologists can understand the user's emotional state and emotional changes, and provide more accurate and personalized guidance for emotional education and mental health counseling.

[0183] In summary, this technical solution achieves accurate assessment of a user's facial expression type and emotional intensity through automatic feature extraction and global information learning using a deep learning model. This technical solution not only improves the accuracy of emotion assessment but also enhances the user experience, supports the application of sentiment analysis, promotes the development of artificial intelligence technology, promotes intelligent human-computer interaction, and improves the efficiency and accuracy of data processing. Furthermore, it provides strong support for the fields of emotional education and mental health, possessing broad application prospects and significant practical value.

[0184] In one embodiment of the present invention, determining the push ratio of content similar to the web content currently browsed by the user based on the secondary emotion determination result includes:

[0185] S501: When the secondary emotion determination result determines that the user's expression when browsing the current web content is either bored or indifferent, and the emotion level is strong, similar content to the current web content is not pushed;

[0186] S502: When the secondary emotion determination result determines that the user's expression when browsing the current web content is either bored or indifferent, and the emotional level is average, the push ratio of content similar to the current web content is set to 3%-8%; that is, the push ratio of content similar to the current web content included in the daily push content is 3%-8%;

[0187] S503. When the secondary emotion determination result determines that the user's expression when browsing the current web content is any one of happiness, sadness, anger, and surprise, and the emotional level is average, the push ratio of content similar to the current web content is set to 8%-15%; that is, the push ratio of content similar to the current web content included in the daily push content is 8%-15%;

[0188] S504. When the secondary emotion determination result determines that the user's expression when browsing the current web content is any one of happiness, sadness, anger, and surprise, and the degree of emotion is strong, the push ratio of content similar to the current web content is set to 15%-28%. That is, the push ratio of content similar to the current web content included in the daily push content is 15%-28%;

[0189] The working principle of the above technical solution is as follows: When a user expresses strong boredom or indifference, the system assumes that the user is not interested in the current content and therefore decides not to push similar content to prevent the user from feeling annoyed or developing negative emotions. When a user expresses moderate boredom or indifference, the system assumes that while the user has little interest in the current content, they may still be receptive to pushes of similar content. Therefore, the system sets a lower push ratio (3%-8%) to test the user's response while avoiding excessive pushes. When a user expresses moderate happiness, sadness, anger, or surprise, the system assumes that the user has some interest in or emotional resonance with the current content. Therefore, the system sets a moderate push ratio (8%-15%) to maintain user interest while providing more relevant content options. When a user expresses strong happiness, sadness, anger, or surprise, the system assumes that the user is very interested in the current content or has a strong emotional reaction. Therefore, the system sets a higher push ratio (15%-28%) to provide more similar content to satisfy the user's emotional needs and desire for exploration.

[0190] The above technical solution achieves the following: By precisely analyzing users' emotional states, the system can more accurately determine their interests and preferences for different content. This personalized push strategy not only reduces unnecessary interruptions but also increases the likelihood that users receive content they find interesting, thereby improving the overall user experience. When users perceive that the content pushed by the system closely aligns with their interests and emotional states, they are more likely to continue using the service and develop a higher level of engagement. This engagement not only improves user satisfaction but also promotes the dissemination and sharing of content, creating a virtuous cycle. By dynamically adjusting the push ratio, the system can more efficiently allocate resources, ensuring that content is delivered precisely to interested users. This optimized distribution strategy not only increases content exposure but also reduces the cost of ineffective push notifications, improving overall content distribution efficiency. When users receive content that resonates with their current emotional state, they are more likely to resonate emotionally and be willing to share it with others. This emotional resonance and social interaction not only strengthens users' emotional connections but also promotes the dissemination and impact of content. Personalized push strategies make users feel valued and cared for, thereby increasing their satisfaction and loyalty. When users believe the system accurately understands their needs and preferences, they are more likely to become long-term loyal users and willing to provide more feedback and support. By collecting and analyzing user sentiment data, the system can provide valuable feedback and insights to content creators. This feedback and insights not only helps creators understand user interests and preferences, but also guides them in creating content that better meets user needs, thereby improving the quality and appeal of content. The successful application of this technical solution demonstrates the enormous potential of AI technology in the fields of sentiment analysis and personalized recommendations. With the continuous advancement of technology and the in-depth expansion of its applications, AI will play a vital role in even more areas, driving innovation and development across the industry.

[0191] In summary, this technical solution, by accurately analyzing users' emotional states and dynamically adjusting push strategies, not only improves user experience and retention, but also optimizes content distribution efficiency, fosters emotional resonance and social interaction, increases user satisfaction and loyalty, and provides valuable data support for content creators. These technical results jointly promote the further development of artificial intelligence technology, bringing more value and opportunities to users and content creators.

[0192] The embodiment of the present invention proposes an emotional feedback system based on content recognition, such as Figure 2 As shown, the emotional feedback system based on content recognition includes:

[0193] A real-time collection module is used to collect the user's eye data information and facial data information in real time when the user is browsing the network content, and obtain the user's eye data information and facial data information when the user is browsing the network content;

[0194] A primary emotion determination module is used to determine the primary emotion of the user browsing the network content based on the eye data information of the user when browsing the network content, and obtain a primary emotion determination result;

[0195] An acquisition frequency adjustment module is used to determine whether to adjust the frequency of data acquisition of the user's facial data information according to the primary emotion determination result;

[0196] A secondary emotion determination module is configured to perform a secondary emotion determination using facial data information to obtain a secondary emotion determination result when the primary emotion determination result indicates that the user is interested in the browsed web content;

[0197] The push setting module is used to determine the push ratio of content similar to the network content currently browsed by the user based on the secondary emotion judgment result.

[0198] The working principle of the above technical solution is as follows: When a user browses content on a web platform, the system collects eye and facial data in real time. This data includes information such as gaze direction, pupil size changes, and facial expression changes, which can reflect the user's emotional response and level of interest during browsing. Based on the collected eye data, such as gaze duration and pupil changes, the system determines the user's primary emotion while browsing the web content. For example, prolonged gaze and pupil dilation may indicate interest in the content, while rapid glances and pupil constriction may indicate disinterest or boredom. Based on the results of the primary emotion determination, the system dynamically adjusts the frequency of facial data collection. If the user expresses strong interest in the content, the system increases the data collection frequency to more accurately capture the user's emotional changes. Conversely, if the user expresses disinterest or boredom, the system appropriately reduces the data collection frequency to reduce unnecessary resource consumption. If the primary emotion determination indicates that the user is interested in the content, the system further utilizes the facial data information for secondary emotion determination. This assessment is more detailed and in-depth, analyzing facial expressions, muscle movement, and other factors to more accurately determine the user's emotional state. Based on this secondary emotional assessment, the system determines the proportion of content similar to the content the user is currently viewing. If the user expresses strong interest and a positive emotional response, the system will increase the proportion of similar content recommended to meet their needs. Conversely, if the user expresses disinterest or a negative emotional response, the system will reduce the proportion of similar content recommended to prevent the user from becoming bored.

[0199] The effect of the above technical solution is: by collecting and analyzing the user's eye and facial data information in real time, the system can more accurately capture the user's emotional response and interest level, thereby providing users with more personalized content recommendations and a better service experience. This technical solution can dynamically adjust the content recommendation algorithm based on the user's real-time emotional feedback to make it more in line with the user's interests and needs. This helps to improve the accuracy of content recommendations and user satisfaction. At the same time, by dynamically adjusting the data collection frequency, the system can reduce unnecessary resource consumption and data collection while ensuring recognition accuracy. The application of this technical solution helps to promote human-computer interaction towards a more intelligent and personalized direction. By analyzing the user's emotional response and interest level in real time, the system can provide users with a more intelligent and caring service experience.

[0200] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for emotional feedback based on content recognition, characterized in that: The emotional feedback method based on content recognition includes: S1: When a user browses online content, the user's eye data information and facial data information are collected in real time to obtain the user's eye data information and facial data information when the user browses online content; S2: determining the primary emotion of the user browsing the web content based on the eye data information of the user when browsing the web content, and obtaining a primary emotion determination result; S3: Determining whether to adjust the frequency of data collection of the user's facial data information according to the primary emotion determination result; S4: When the primary emotion determination result indicates that the user is interested in the browsed web content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result; S5: Determine the push ratio of content similar to the web content currently browsed by the user based on the secondary emotion determination result; Wherein step S2 comprises: Extracting eye data information of the user when browsing online content; wherein the eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blink frequency corresponding to each fixation point; Obtaining a content browsing coefficient using a scanning speed of a gaze point transfer process included in eye data information of the user when browsing network content; Comparing the content browsing coefficient with a preset browsing coefficient threshold; When the content browsing coefficient is lower than a preset browsing coefficient threshold, the gaze dwell time corresponding to each gaze point and the blink frequency corresponding to each gaze point included in the eye data information are retrieved; Retrieving the content browsing coefficient; using the gaze point dwell time corresponding to each gaze point and the blink frequency corresponding to each gaze point included in the eye data information in combination with the content browsing coefficient to obtain an initial emotion determination coefficient; Comparing the initial emotion determination coefficient with a preset determination coefficient threshold; When the initial emotion determination coefficient exceeds a preset determination coefficient threshold, it is determined that the user has a preliminary interest in the currently browsed web content; When the initial emotion determination coefficient does not exceed the preset determination coefficient threshold, it is determined that the user has no initial interest in the currently browsed network content.

2. The emotional feedback method based on content recognition according to claim 1, characterized in that When a user browses online content, the user's eye data information and facial data information are collected in real time, and the eye data information and facial data information of the user when browsing online content are obtained, including: When the user browses the network content, the initial eye data collection frequency corresponding to the eye data information collection and the initial facial data collection frequency corresponding to the facial data information are retrieved from the database; Collecting eye data information of the user when browsing online content according to the initial eye data collection frequency to obtain the eye data information of the user when browsing online content; The eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blink frequency corresponding to each fixation point; Collecting facial data information of the user when browsing online content according to the initial facial data collection frequency to obtain facial data information of the user when browsing online content; The facial data information includes coordinate change data corresponding to the eyebrow mark point, coordinate change data corresponding to the mouth mark point, and coordinate change data corresponding to the eye mark point.

3. The emotional feedback method based on content recognition according to claim 1, characterized in that in, The content browsing coefficient is obtained by the following formula: in, C Indicates the content browsing coefficient; n Indicates the total number of fixations; S ci Indicates the i The rate of outgoing gaze points; S ri Indicates the i The turn-in rate corresponding to each fixation point; V i Indicates the i The value of the visual complexity of the area corresponding to each fixation point; T si Indicates the i The fixation point corresponds to the repetition rate; V max express n The maximum value of the visual complexity of the area corresponding to the fixation point; S rmax express n The maximum value of the turn-in rate corresponding to the fixation point; S rm express Fixations corresponding to the maximum value of visual complexity The transfer rate of the area corresponding to the point .

4. The emotional feedback method based on content recognition according to claim 1, characterized in that in, The initial emotion determination coefficient is obtained by the following formula: in, E represents the initial sentiment determination coefficient; n Indicates the total number of fixations; D i Indicates the i The duration of the fixation point corresponding to each fixation point; F i Indicates the i Blink frequency corresponding to each fixation point; e Indicates the minimum constant for pre-trial, used to prevent the denominator from being 0; F max Indicates the maximum blink frequency of a user during browsing.

5. The emotional feedback method based on content recognition according to claim 1, characterized in that: Determining whether to adjust the frequency of data collection of the user's facial data information according to the primary emotion determination result includes: Retrieve primary emotion determination results; When the primary emotion determination result indicates that the user has no initial interest in the currently browsed web content, the frequency of collecting the user's facial data information is not adjusted; When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the frequency of data collection of the user's facial data information is adjusted.

6. The emotional feedback method based on content recognition according to claim 5, characterized in that: When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the frequency of collecting the user's facial data information is adjusted, including: When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the coordinate change data corresponding to the eyebrow mark point and the coordinate change data corresponding to the mouth mark point are retrieved; Obtaining the coordinate change frequency and position change amplitude corresponding to each eyebrow marker point according to the coordinate change data corresponding to the eyebrow marker point; Obtaining the coordinate change frequency and position change amplitude corresponding to each mouth mark point according to the coordinate change data corresponding to the mouth mark point; The first action determination coefficient is obtained by using the coordinate change frequency and position change amplitude corresponding to each eyebrow marker point, wherein the first action determination coefficient is obtained by the following formula: in, R 01 represents the first action determination coefficient; m Indicates the total number of eyebrow markers; f mi Indicates the i The coordinate change frequency corresponding to each eyebrow marker point; Indicates the i The position change amplitude corresponding to each eyebrow marker point; α express m The angle between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; L express m The distance between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; The second action determination coefficient is obtained by using the coordinate change frequency and position change amplitude corresponding to the mouth mark point, wherein the second action determination coefficient is obtained by the following formula: in, R 02 represents the second action determination coefficient; k Indicates the total number of mouth landmarks; f ki Indicates the i The coordinate change frequency corresponding to each mouth marker point; a ki Indicates the i The position change amplitude corresponding to each mouth marker point; α express m The angle between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; L express m The distance between the eyebrow marker point with the minimum position change amplitude and the eyebrow marker point with the maximum position change amplitude corresponding to each eyebrow marker point; β express k The angle between the mouth marker point with the minimum position change amplitude and the mouth marker point with the maximum position change amplitude corresponding to the mouth marker point; D express k The distance between the mouth marker point with the minimum position change and the mouth marker point with the maximum position change corresponding to each mouth hair marker point; Retrieve the initial emotion determination coefficient; adjusting the data collection frequency of the facial data information by using the first action determination coefficient and the second action determination coefficient in combination with the initial emotion determination coefficient, and collecting facial data information according to the adjusted data collection frequency; The adjusted data collection frequency is obtained by the following formula: in, F represents the adjusted data collection frequency; F 0 indicates the adjusted data collection frequency; R 01 represents the first action determination coefficient; R 02 Indicates the second action determination coefficient.

7. The emotional feedback method based on content recognition according to claim 1, characterized in that: When the primary emotion determination result indicates that the user is interested in the browsed web content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result, including: When the primary emotion determination result indicates that the user is interested in the browsed web content, facial data information is retrieved; Performing data preprocessing on the facial data information to obtain preprocessed facial data information; wherein the data preprocessing includes noise reduction processing and invalid value removal processing; The pre-processed facial data information is input into a deep learning model for facial expression recognition and expression degree recognition to obtain the facial expression type and the emotion degree corresponding to the facial expression type; wherein the deep learning model adopts a convolutional neural network model.

8. The emotional feedback method based on content recognition according to claim 7, characterized in that: The structure of the deep learning model is as follows: Input layer, used to receive pre-processed facial data information; The convolution layer is used to extract spatial features from the processed facial data information received to obtain spatial features; The flattening layer is used to flatten the spatial features of the output of the convolutional layer into a one-dimensional array; The fully connected layer is used to integrate the one-dimensional arrays corresponding to the extracted spatial features and learn global information; The output layer is used to output the expression recognition results and the corresponding emotional level of the expression; The expression recognition results include boredom, indifference, happiness, sadness, anger and surprise; and the emotion levels include normal and strong.

9. The emotional feedback method based on content recognition according to claim 1, characterized in that: Determining the push ratio of content similar to the web content currently browsed by the user based on the secondary emotion determination result includes: If the secondary emotion determination result determines that the user's expression when browsing the current web content is either bored or indifferent, and the emotion level is strong, then similar content to the current web content will not be pushed; When the secondary emotion judgment result determines that the user's expression when browsing the current web content is bored or indifferent, and the emotional level is average, the push ratio of similar content to the current web content is set to 3%-8%; When the secondary emotion judgment result determines that the user's expression when browsing the current web content is any of happiness, sadness, anger, and surprise, and the emotional level is average, the push ratio of similar content to the current web content is set to 8%-15%; When the secondary emotion judgment result determines that the user's expression when browsing the current web content is any one of happiness, sadness, anger and surprise, and the degree of emotion is strong, the push ratio of similar content to the current web content is set to 15%-28%.

10. An emotional feedback system based on content recognition, characterized in that: The emotional feedback system based on content recognition includes: A real-time collection module is used to collect the user's eye data information and facial data information in real time when the user is browsing the network content, and obtain the user's eye data information and facial data information when the user is browsing the network content; A primary emotion determination module is used to determine the primary emotion of the user browsing the network content based on the eye data information of the user when browsing the network content, and obtain a primary emotion determination result; An acquisition frequency adjustment module is used to determine whether to adjust the frequency of data acquisition of the user's facial data information according to the primary emotion determination result; A secondary emotion determination module is configured to perform a secondary emotion determination using facial data information to obtain a secondary emotion determination result when the primary emotion determination result indicates that the user is interested in the browsed web content; A push setting module, configured to determine a push ratio of content similar to the web content currently browsed by the user based on the secondary emotion determination result; The primary emotion judgment module includes: Extracting eye data information of the user when browsing online content; wherein the eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blink frequency corresponding to each fixation point; Obtaining a content browsing coefficient using a scanning speed of a gaze point transfer process included in eye data information of the user when browsing network content; Comparing the content browsing coefficient with a preset browsing coefficient threshold; When the content browsing coefficient is lower than a preset browsing coefficient threshold, the gaze dwell time corresponding to each gaze point and the blink frequency corresponding to each gaze point included in the eye data information are retrieved; Retrieving the content browsing coefficient; using the gaze point dwell time corresponding to each gaze point and the blink frequency corresponding to each gaze point included in the eye data information in combination with the content browsing coefficient to obtain an initial emotion determination coefficient; Comparing the initial emotion determination coefficient with a preset determination coefficient threshold; When the initial emotion determination coefficient exceeds a preset determination coefficient threshold, it is determined that the user has a preliminary interest in the currently browsed web content; When the initial emotion determination coefficient does not exceed the preset determination coefficient threshold, it is determined that the user has no initial interest in the currently browsed network content.

Images