Emotion feedback method and system based on content recognition

Abstract

The invention provides an emotion feedback method and system based on content recognition. The emotion feedback method based on content recognition comprises the steps that when a user browses network content, eye data information and face data information of the user are collected in real time, and the eye data information and the face data information when the user browses the network content are obtained; according to the eye data information when the user browses the network content, primary emotion judgment is carried out on the network content browsed by the user, and a primary emotion judgment result is obtained; judging whether to adjust the data acquisition frequency of the facial data information of the user according to the primary emotion judgment result; when the primary emotion judgment result shows that the user is interested in the browsed network content, performing secondary emotion judgment by utilizing the facial data information to obtain a secondary emotion judgment result; and according to a secondary emotion judgment result, determining a pushing proportion of contents similar to the network contents currently browsed by the user. The system comprises modules corresponding to the steps of the method.

Description

Technical Field

[0001] The invention proposes an emotion feedback method and system based on content recognition, belonging to the technical field of emotion recognition feedback. Background Art

[0002] With the rapid development of Internet technology, users browsing content on the Internet has become a part of daily life. Understanding users' emotional reactions when browsing online content is of great significance for improving user experience, optimizing content recommendation algorithms, and achieving more accurate advertising. In recent years, emotion recognition technology, as an important branch of computer vision and natural language processing, has been widely studied and applied. Traditional emotion recognition methods mostly rely on sentiment analysis of text content. For example, in the field of social media, text-based sentiment analysis is used to identify user emotions and opinions; in the field of e-commerce, users' likes and dislikes are analyzed by analyzing users' comments on a certain e-commerce product. However, these methods are only based on text content, ignoring users' immediate emotional reactions during browsing, and cannot fully capture users' true feelings. In order to more accurately obtain users' emotional feedback when browsing online content, researchers have begun to explore emotion recognition methods based on biometrics in recent years. Eye data information and facial data information, as important biometrics, can reflect users' emotional state in real time. For example, by collecting users' eye data information in real time, users' gaze time, pupil changes, etc. can be analyzed to preliminarily judge the user's interest in browsing content. Furthermore, by combining facial data information, such as expression changes, muscle activity, etc., more detailed emotion judgment can be made.

[0003] In the existing technology, some emotion recognition methods based on biometrics have been proposed. For example, by training the model through deep learning technology, it can extract emotional features from eye and facial data, and then realize emotion recognition. However, these methods still have some limitations in the application process. For example, how to reduce the workload of data collection and processing while ensuring recognition accuracy, and how to find a balance between user privacy protection and emotion recognition effect are all problems that need to be solved urgently. Summary of the invention

[0004] The present invention provides an emotional feedback method and system based on content recognition to solve the technical problems existing in the above-mentioned prior art. The technical solutions adopted are as follows: A method for emotional feedback based on content recognition, the method comprising: When the user browses the network content, the user's eye data information and facial data information are collected in real time to obtain the user's eye data information and facial data information when the user browses the network content; Determine the primary emotion of the user browsing the network content according to the eye data information of the user when browsing the network content, and obtain the primary emotion determination result; Determining whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result; When the primary emotion determination result indicates that the user is interested in the browsed web content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result; The push ratio of content similar to the network content currently browsed by the user is determined according to the secondary emotion determination result.

[0005] Furthermore, when the user browses the network content, the user's eye data information and facial data information are collected in real time, and the eye data information and facial data information of the user when browsing the network content are obtained, including: When the user browses the network content, the initial eye data collection frequency corresponding to the eye data information collection and the initial facial data collection frequency corresponding to the facial data information are retrieved from the database; Collecting eye data information of the user when browsing online content according to the initial eye data collection frequency to obtain the eye data information of the user when browsing online content; The eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blinking frequency corresponding to each fixation point; Collecting facial data information of the user when browsing online content according to the initial facial data collection frequency, and obtaining facial data information of the user when browsing online content; The facial data information includes coordinate change data corresponding to the eyebrow mark point, coordinate change data corresponding to the mouth mark point and coordinate change data corresponding to the eye mark point.

[0006] Further, the primary emotion judgment of the user browsing the network content is determined based on the eye data information of the user when browsing the network content, and the primary emotion judgment result is obtained, including: Extracting eye data information of the user when browsing online content; wherein the eye data information includes the duration of fixation, the scanning speed during the fixation transfer process, and the blinking frequency corresponding to each fixation point; Obtaining a content browsing coefficient using the scanning speed of the gaze point transfer process contained in the eye data information when the user browses the network content; The content browsing coefficient is obtained by the following formula: Where C represents the content browsing coefficient; n represents the total number of fixations; Sci represents the outgoing rate corresponding to the i-th fixation point; S ri represents the transition rate corresponding to the i-th fixation point; V i represents the value of the visual complexity of the area corresponding to the i-th fixation point; T si represents the repetitive fixation rate corresponding to the ith fixation point; V max Indicates the maximum value of the visual complexity of the area corresponding to n fixation points; S rmax represents the maximum value of the transfer rate corresponding to n fixation points; S rm The transfer rate of the area corresponding to the fixation point corresponding to the maximum value of visual complexity; Comparing the content browsing coefficient with a preset browsing coefficient threshold; When the content browsing coefficient is lower than a preset browsing coefficient threshold, the primary emotion judgment of the user browsing the network content is made using the gaze point dwell time and the blinking frequency corresponding to each gaze point contained in the eye data information to obtain a primary emotion judgment result.

[0007] Further, the primary emotion judgment of the user browsing the web content is performed using the duration of the gaze point and the blinking frequency corresponding to each gaze point included in the eye data information to obtain the primary emotion judgment result, including: When the content browsing coefficient is lower than a preset browsing coefficient threshold, the gaze point dwelling time corresponding to each gaze point and the blinking frequency corresponding to each gaze point included in the eye data information are retrieved; Retrieving the content browsing coefficient; The initial emotion determination coefficient is obtained by using the gaze point dwell time corresponding to each gaze point and the blinking frequency corresponding to each gaze point contained in the eye data information in combination with the content browsing coefficient; The initial emotion determination coefficient is obtained by the following formula: Where E represents the initial emotion determination coefficient; n represents the total number of fixations; D i represents the duration of the fixation point corresponding to the i-th fixation point; F i represents the blink frequency corresponding to the i-th fixation point; e represents the minimum constant of the pre-trial, which is used to prevent the denominator from being 0; F max Indicates the maximum blink frequency of the user during browsing; Comparing the initial emotion determination coefficient with a preset determination coefficient threshold; When the initial emotion determination coefficient exceeds a preset determination coefficient threshold, it is determined that the user has a preliminary interest in the currently browsed web content; When the initial emotion determination coefficient does not exceed the preset determination coefficient threshold, it is determined that the user has no initial interest in the currently browsed network content.

[0008] Further, judging whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result includes: Retrieve primary emotion determination results; When the primary emotion determination result indicates that the user has no initial interest in the currently browsed web content, the data collection frequency of the user's facial data information is not adjusted; When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the data collection frequency of the user's facial data information is adjusted.

[0009] Furthermore, when the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed network content, the data collection frequency of the user's facial data information is adjusted, including: When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the coordinate change data corresponding to the eyebrow mark point and the coordinate change data corresponding to the mouth mark point are retrieved; Obtaining the coordinate change frequency and position change amplitude corresponding to each eyebrow marking point according to the coordinate change data corresponding to the eyebrow marking point; Obtaining the coordinate change frequency and position change amplitude corresponding to each mouth mark point according to the coordinate change data corresponding to the mouth mark point; The first action determination coefficient is obtained by using the coordinate change frequency and position change amplitude corresponding to each eyebrow mark point, wherein the first action determination coefficient is obtained by the following formula: Among them, R 01 represents the first action determination coefficient; m represents the total number of eyebrow marking points; f mi represents the coordinate change frequency corresponding to the i-th eyebrow marker point; a mi represents the position change amplitude corresponding to the i-th eyebrow marker; α represents the angle between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; L represents the distance between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; The second action determination coefficient is obtained by using the coordinate change frequency and position change amplitude corresponding to the mouth mark point, wherein the second action determination coefficient is obtained by the following formula: Among them, R02 represents the second action determination coefficient; k represents the total number of mouth marking points; f ki Indicates the coordinate change frequency corresponding to the i-th mouth mark point; a ki Represents the position change amplitude corresponding to the i-th mouth marker; α represents the angle between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; L represents the distance between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; β represents the angle between the mouth marker with the minimum position change amplitude corresponding to the k mouth markers and the mouth marker with the maximum position change amplitude; D represents the distance between the mouth marker with the minimum position change amplitude corresponding to the k mouth hair markers and the mouth marker with the maximum position change amplitude; Retrieve the initial emotion determination coefficient; Using the first action determination coefficient and the second action determination coefficient in combination with the initial emotion determination coefficient, the data collection frequency of the facial data information is adjusted, and facial data information is collected according to the adjusted data collection frequency; The adjusted data collection frequency is obtained by the following formula: Where F represents the adjusted data collection frequency; F0 represents the adjusted data collection frequency; R 01 Represents the first action determination coefficient; R 02 Indicates the second action determination coefficient.

[0010] Further, when the primary emotion determination result indicates that the user is interested in the browsed network content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result, including: When the primary emotion determination result indicates that the user is interested in the browsed web content, facial data information is retrieved; Performing data preprocessing on the facial data information to obtain preprocessed facial data information; wherein the data preprocessing includes noise reduction processing and invalid value removal processing; The preprocessed facial data information is input into a deep learning model to perform facial expression recognition and expression degree recognition, and obtain the facial expression type and the emotion degree corresponding to the facial expression type; wherein the deep learning model adopts a convolutional neural network model.

[0011] Furthermore, the structure of the deep learning model is as follows: Input layer, used to receive pre-processed facial data information; The convolution layer is used to extract spatial features from the received processed facial data information to obtain spatial features; The flattening layer is used to flatten the spatial features of the output of the convolutional layer into a one-dimensional array; The fully connected layer is used to integrate the one-dimensional array corresponding to the extracted spatial features and learn global information; The output layer is used to output the expression recognition results and the corresponding emotional level of the expression; The expression recognition results include boredom, indifference, happiness, sadness, anger and surprise; and the emotion levels include normal and strong.

[0012] Further, determining the push ratio of content similar to the network content currently browsed by the user according to the secondary emotion determination result includes: When the secondary emotion determination result determines that the user's expression when browsing the current web content is bored or indifferent, and the emotion level is strong, similar content to the current web content will not be pushed; When the secondary emotion judgment result determines that the user's expression when browsing the current network content is bored or indifferent, and the emotional level is average, the push ratio of content similar to the current network content is set to 3%-8%; that is, the push ratio of content similar to the current network content contained in the content pushed every day is 3%-8%; When the secondary emotion judgment result determines that the user's expression when browsing the current web content is any one of happiness, sadness, anger and surprise, and the emotional level is average, the push ratio of content similar to the current web content is set to 8%-15%; that is, the push ratio of content similar to the current web content contained in the content pushed every day is 8%-15%; When the secondary emotion judgment result determines that the user's expression when browsing the current web content is any one of happiness, sadness, anger and surprise, and the emotion is strong, the push ratio of content similar to the current web content is set to 15%-28%. That is, the push ratio of content similar to the current web content contained in the content pushed every day is 15%-28%;

[0013] A content recognition-based emotional feedback system, the content recognition-based emotional feedback system comprising: A real-time acquisition module is used to acquire the user's eye data information and facial data information in real time when the user is browsing the network content, and obtain the user's eye data information and facial data information when the user is browsing the network content; A primary emotion determination module, used to determine the primary emotion of the user browsing the network content according to the eye data information of the user when browsing the network content, and obtain a primary emotion determination result; A collection frequency adjustment module, used to determine whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result; A secondary emotion determination module is used to perform secondary emotion determination using facial data information to obtain a secondary emotion determination result when the primary emotion determination result indicates that the user is interested in the browsed network content; The push setting module is used to determine the push ratio of content similar to the network content currently browsed by the user according to the secondary emotion judgment result.

[0014] Beneficial effects of the present invention: The present invention proposes an emotion feedback method and system based on content recognition, which collects the user's eye data information and facial data information in real time when the user is browsing online content, and performs primary emotion judgment on the user based on this information. If the primary emotion judgment result shows that the user is interested in the browsed online content, the facial data information is further used to perform secondary emotion judgment to obtain more accurate emotion feedback. According to the secondary emotion judgment result, the proportion of content push similar to the online content currently browsed by the user can be determined, thereby achieving more accurate content recommendation. The present invention proposes an emotion feedback method and system based on content recognition, which not only overcomes the limitation of traditional emotion recognition methods that only rely on text content, but also realizes comprehensive capture and accurate recognition of the user's emotional state through real-time collection and processing of biometric data. At the same time, by dynamically adjusting the data collection frequency and optimizing the emotion judgment algorithm, the accuracy and efficiency of emotion recognition are improved, providing strong support for improving user experience and optimizing content recommendation algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the method of the present invention; Figure 2 The system block diagram of the system of the present invention. DETAILED DESCRIPTION

[0016] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0017] The embodiment of the present invention proposes an emotional feedback method based on content recognition, such as Figure 1 As shown, the emotional feedback method based on content recognition includes: S1. When a user browses online content, the user's eye data information and facial data information are collected in real time to obtain the user's eye data information and facial data information when the user browses online content; S2, determining the primary emotion of the user browsing the network content according to the eye data information of the user when browsing the network content, and obtaining a primary emotion determination result; S3, judging whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result; S4. When the primary emotion determination result indicates that the user is interested in the browsed network content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result; S5. Determine the push ratio of content similar to the web content currently browsed by the user according to the secondary emotion determination result.

[0018] The working principle of the above technical solution is: when the user browses the content on the network platform, the system will collect the user's eye data information and facial data information in real time. These data include the user's gaze direction, pupil size changes, facial expression changes, etc., which can reflect the user's emotional response and interest level during the browsing process. The system will judge the primary emotion of the user browsing the network content based on the collected eye data information, such as gaze time, pupil changes, etc. For example, long-term gaze and pupil dilation may indicate that the user is interested in the current content, while rapid scanning and pupil constriction may indicate that the user is not interested or bored. According to the primary emotion judgment result, the system will dynamically adjust the data collection frequency of the user's facial data information. If the user shows a strong interest in the content, the system will increase the data collection frequency to more accurately capture the user's emotional changes; conversely, if the user shows disinterest or boredom, the system will appropriately reduce the data collection frequency to reduce unnecessary resource consumption. When the primary emotion judgment result shows that the user is interested in the browsed network content, the system will further use the facial data information for secondary emotion judgment. This time the judgment will be more detailed and in-depth, by analyzing the user's facial expressions, muscle movements, etc., to more accurately judge the user's emotional state. Based on the results of the secondary emotional judgment, the system will determine the push ratio of content similar to the web content currently browsed by the user. If the user shows strong interest and positive emotional response, the system will increase the push ratio of similar content to meet the user's further needs; conversely, if the user shows no interest or negative emotional response, the system will reduce the push ratio of similar content to avoid the user from feeling bored.

[0019] The effect of the above technical solution is: by collecting and analyzing the user's eye and facial data information in real time, the system can more accurately capture the user's emotional response and interest level, thereby providing users with more personalized content recommendations and better service experience. This technical solution can dynamically adjust the content recommendation algorithm based on the user's real-time emotional feedback to make it more in line with the user's interests and needs. This helps to improve the accuracy of content recommendations and user satisfaction. At the same time, by dynamically adjusting the data collection frequency, the system can reduce unnecessary resource consumption and data collection while ensuring recognition accuracy. The application of this technical solution helps to promote human-computer interaction in a more intelligent and personalized direction. By analyzing the user's emotional response and interest level in real time, the system can provide users with a more intelligent and considerate service experience.

[0020] In one embodiment of the present invention, when a user browses a network content, eye data information and facial data information of the user are collected in real time, and the eye data information and facial data information of the user when browsing the network content are obtained, including: S101, when a user browses a network content, an initial eye data collection frequency corresponding to the eye data information collection and an initial facial data collection frequency corresponding to the facial data information are retrieved from a database; S102, collecting eye data information of the user when browsing online content according to the initial eye data collection frequency, to obtain the eye data information of the user when browsing online content; The eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blinking frequency corresponding to each fixation point; S103, collecting facial data information of the user when browsing online content according to the initial facial data collection frequency, and obtaining facial data information of the user when browsing online content; The facial data information includes coordinate change data corresponding to the eyebrow mark point, coordinate change data corresponding to the mouth mark point and coordinate change data corresponding to the eye mark point.

[0021] The working principle of the above technical solution is as follows: when a user starts browsing online content, the system first retrieves the initial eye data collection frequency corresponding to the eye data information collection and the initial facial data collection frequency corresponding to the facial data information from the preset database. These initial frequencies are obtained based on a large amount of user data statistical analysis, aiming to balance the accuracy of data collection and system resource consumption.

[0022] According to the initial eye data collection frequency, the system will collect the user's eye data information in real time. These eye data information include the duration of the gaze point, the scanning speed during the gaze point transfer process, and the blinking frequency corresponding to each gaze point. These data can reflect key information such as the user's attention allocation, reading speed, and eye fatigue during the browsing process.

[0023] Specifically, Gaze dwell time: reflects the user's attention to a certain content point. The longer the dwell time, the more interested the user is in the content.

[0024] Scan speed: reflects the speed at which users browse content. The faster the speed, the more likely it is that users are browsing quickly to find content of interest.

[0025] Blinking frequency: It can indirectly reflect the user's eye fatigue level. Too high a blinking frequency may mean that the user needs a break.

[0026] At the same time, the system will also collect the user's facial data information in real time according to the initial facial data collection frequency. These facial data information include the coordinate change data corresponding to the eyebrow mark point, the coordinate change data corresponding to the mouth mark point, and the coordinate change data corresponding to the eye mark point. These data can capture the subtle changes in the user's facial expression and infer the user's emotional state.

[0027] Specific: Changes in eyebrow marker coordinates: can reflect the user's surprise, confusion and other emotional changes.

[0028] Changes in the coordinates of the mouth marker point: can capture the user's smile, frown and other expressions, and then judge the user's emotional state.

[0029] Changes in eye marker coordinates: In addition to analyzing eye fatigue in combination with blinking frequency, it can also capture changes in eye expression, such as staring and wandering, to determine the user's concentration level.

[0030] The effect of the above technical solution is: by collecting and analyzing the user's eye and facial data information in real time, the system can more accurately understand the user's emotional response and points of interest during the browsing process, thereby providing more personalized content recommendations and services. This helps to improve user satisfaction and loyalty. Combined with the user's eye and facial data information, the system can more accurately judge the user's preferences and points of interest for content, and then optimize the content recommendation algorithm to improve the accuracy and relevance of the recommendation. This helps to improve the click-through rate and conversion rate of the content. Although the system needs to collect the user's eye and facial data information, all collected data will be strictly encrypted and processed to ensure the privacy and security of the user. At the same time, the system will dynamically adjust the scope and frequency of data collection according to the user's privacy settings and feedback to maximize the protection of the user's privacy rights and interests. By analyzing the user's eye and facial data information in real time, the system can more intelligently understand the user's intentions and emotional state, thereby providing a more intelligent and humanized interactive experience. This helps to improve the efficiency and satisfaction of human-computer interaction. In the field of advertising, by capturing and analyzing the user's eye and facial data information in real time, the system can more accurately judge the user's interest and reaction to the advertisement, thereby optimizing the advertising delivery strategy and improving the exposure and conversion rate of the advertisement. In addition to its application in online content browsing, this technical solution can also be used in the field of mental health monitoring. By analyzing the user's eye and facial data information in real time, the system can promptly detect the user's emotional abnormalities and mental health problems, and provide strong support for mental health intervention. The successful application of this technical solution will promote technological innovation and development in related fields. For example, in the fields of artificial intelligence, computer vision, etc., by continuously optimizing algorithms and models, the collection and analysis capabilities of eye and facial data information can be further improved, providing technical support for more application scenarios. For network platforms, by applying this technical solution, users can be provided with a more personalized and intelligent service experience, thereby enhancing market competitiveness. This will help attract more users to settle in and use the platform, laying a solid foundation for the long-term development of the platform.

[0031] To sum up, this technical solution achieves accurate capture and interpretation of user emotional responses by real-time collection and analysis of user eye and facial data information, bringing significant technical effects in improving user experience, optimizing content recommendation algorithms, enhancing user privacy protection, and promoting intelligent human-computer interaction.

[0032] In one embodiment of the present invention, the primary emotion determination result of the user browsing the network content is obtained based on the eye data information of the user when browsing the network content, including: S201, extracting eye data information of the user when browsing online content; wherein the eye data information includes the duration of fixation, the scanning speed during the fixation transfer process, and the blinking frequency corresponding to each fixation point; S202, obtaining a content browsing coefficient by using the scanning speed of the gaze point transfer process contained in the eye data information when the user browses the network content; The content browsing coefficient is obtained by the following formula: Where C represents the content browsing coefficient; n represents the total number of fixations; S ci represents the outgoing rate corresponding to the i-th fixation point; S ri represents the transition rate corresponding to the i-th fixation point; V i represents the value of the visual complexity of the area corresponding to the i-th fixation point; T si represents the repetitive fixation rate corresponding to the ith fixation point; V max Indicates the maximum value of the visual complexity of the area corresponding to n fixation points; S rmax represents the maximum value of the transfer rate corresponding to n fixation points; S rm The transfer rate of the area corresponding to the fixation point corresponding to the maximum value of visual complexity; S203, comparing the content browsing coefficient with a preset browsing coefficient threshold; S204. When the content browsing coefficient is lower than a preset browsing coefficient threshold, the primary emotion judgment of the user browsing the network content is made using the gaze point dwell time and the blinking frequency corresponding to each gaze point contained in the eye data information to obtain a primary emotion judgment result.

[0033] The working principle of the above technical solution is: using eye tracking technology to capture and record the eye data information of users in the process of browsing network content in real time. These data information include the duration of fixation, the scanning speed of the fixation transfer process and the blinking frequency corresponding to each fixation point. According to the extracted eye data information, especially the scanning speed of the fixation transfer process, the content browsing coefficient is calculated. The content browsing coefficient is a comprehensive indicator, which takes into account multiple factors such as the outgoing rate, the incoming rate, the visual complexity and the repeated fixation rate of the fixation point. Through the weighting and normalization of these factors, a coefficient value that can reflect the efficiency and depth of the user's browsing content is obtained. The calculated content browsing coefficient is compared with a preset browsing coefficient threshold. The preset browsing coefficient threshold is obtained based on a large amount of user data and analysis, and is used to judge the primary emotional tendency of the user to browse the content. When the content browsing coefficient is lower than the preset browsing coefficient threshold, it is considered that the user may not be interested in or confused about the content currently browsed. At this time, the duration of fixation and the blinking frequency corresponding to each fixation point in the eye data information are used for further analysis. Through comprehensive processing of this information, primary emotional judgment of users browsing online content can be achieved and the judgment results can be output.

[0034] The effect of the above technical solution is: by combining multiple dimensions of eye data information (such as the duration of fixation, scanning speed and blinking frequency), the user's emotional state can be more comprehensively reflected. Compared with single-dimensional data, this multi-dimensional analysis method can more accurately identify the user's primary emotional tendency. The technical solution can monitor the user's browsing behavior and emotional state in real time, providing strong support for the personalized recommendation system. According to the user's emotional tendency and interest preference, the system can intelligently adjust the recommended content to improve user satisfaction and loyalty. By analyzing the user's eye data information, problems and confusions that the user may encounter during the browsing process can be discovered in time. This helps website or application developers optimize page layout and adjust content presentation methods, thereby improving user browsing experience and satisfaction. The technical solution provides new ideas and methods for the field of human-computer interaction, which helps to promote the intelligent development of human-computer interaction technology. By real-time monitoring and analysis of the user's emotional state, the system can respond to the user's needs and instructions more intelligently, improving the efficiency and naturalness of human-computer interaction.

[0035] This technical solution involves a lot of data processing and analysis work, and requires the use of advanced algorithms and technical means. This helps to improve data processing and analysis capabilities and promote technological progress and innovative development in related fields. In practical applications, this technical solution needs to ensure the security and privacy protection of user data. This technical solution has a wide range of application areas and scenarios, such as online education, e-commerce, virtual reality, etc. In these fields, by real-time monitoring and analysis of the user's emotional state, personalized services and support can be provided for teaching, shopping, entertainment, etc., improving user experience and satisfaction.

[0036] At the same time, the above technical solution can more comprehensively reflect the physiological reaction of users when browsing online content by comprehensively considering the user's eye data information (fixation point dwelling time, scanning speed, blinking frequency, etc.), so as to more accurately judge the user's primary emotion. In particular, the content browsing coefficient is introduced, which combines multiple factors such as scanning speed, visual complexity and repeated fixation rate in the fixation point transfer process, and can more finely characterize the user's browsing behavior and improve the accuracy of emotional judgment. The solution can capture and analyze the user's eye data in real time, so as to timely adjust the presentation method or recommendation strategy of network content to meet the personalized needs of users and improve user experience. At the same time, the solution can reduce unnecessary calculations to a certain extent and improve the operation efficiency of the algorithm. The solution can adapt to the browsing habits and emotional expressions of different users. Since each person's eye data and emotional expression methods are different, the solution can more flexibly adapt to the characteristics of different users by comprehensively considering multiple eye data information points.

[0037] In summary, this technical solution achieves primary emotional judgment of users browsing online content by extracting and analyzing the eye data information of users when browsing online content. This technical solution has significant technical effects and application value, and can provide users with a more personalized and intelligent service experience.

[0038] In one embodiment of the present invention, the primary emotion judgment of the user browsing the web content is obtained by using the duration of the gaze point and the blinking frequency corresponding to each gaze point included in the eye data information, including: S2041, when the content browsing coefficient is lower than a preset browsing coefficient threshold, retrieving the gaze point dwelling time corresponding to each gaze point and the blinking frequency corresponding to each gaze point included in the eye data information; S2042, retrieving the content browsing coefficient; S2043, using the gaze point dwell time corresponding to each gaze point and the blinking frequency corresponding to each gaze point included in the eye data information in combination with the content browsing coefficient to obtain an initial emotion determination coefficient; The initial emotion determination coefficient is obtained by the following formula: Where E represents the initial emotion determination coefficient; n represents the total number of fixations; D i represents the duration of the fixation point corresponding to the i-th fixation point; F i represents the blink frequency corresponding to the i-th fixation point; e represents the minimum constant of the pre-trial, which is used to prevent the denominator from being 0; F max Indicates the maximum blink frequency of the user during browsing; S2044, comparing the initial emotion determination coefficient with a preset determination coefficient threshold; S2045: When the initial emotion determination coefficient exceeds a preset determination coefficient threshold, it is determined that the user has a preliminary interest in the currently browsed network content; S2046: When the initial emotion determination coefficient does not exceed the preset determination coefficient threshold, it is determined that the user has no initial interest in the currently browsed network content.

[0039] The working principle of the above technical solution is: when the content browsing coefficient is lower than the preset browsing coefficient threshold, the system believes that the user may have some emotional tendency towards the currently browsed content and needs further analysis. At this time, the system will retrieve the gaze point dwell time and blinking frequency corresponding to each gaze point in the eye data information. This information can reflect the user's attention and emotional state when browsing the content. Although the content browsing coefficient has been calculated in the previous step, it is retrieved again in this step to combine and analyze with the initial emotional judgment coefficient calculated later to more comprehensively evaluate the user's emotional state. The initial emotional judgment coefficient is calculated by a specific formula using the gaze point dwell time and blinking frequency corresponding to each gaze point, combined with the content browsing coefficient. This coefficient is a comprehensive indicator that takes into account multiple factors such as the user's attention allocation, emotional response, and browsing efficiency when browsing the content. The calculated initial emotional judgment coefficient is compared with the preset judgment coefficient threshold. This threshold is derived based on a large amount of user data and analysis, and is used to determine whether the user has a preliminary interest in the currently browsed content. According to the comparison results, if the initial emotion determination coefficient exceeds the preset determination coefficient threshold, it is determined that the user has a preliminary interest in the currently browsed network content; if it does not exceed the threshold, it is determined that the user has no preliminary interest. This result can provide strong support for subsequent content recommendations, user experience optimization, etc.

[0040] The effect of the above technical solution is: by combining multiple dimensions such as the duration of gaze point stay, blinking frequency and content browsing coefficient, the technical solution can more accurately identify the emotional state of users when browsing online content. This multi-dimensional analysis method is more comprehensive and reliable than single-dimensional data. According to the user's emotional judgment results, the system can intelligently adjust the content recommendation strategy to provide users with content that is more in line with their interests and needs. This helps to improve the user's browsing experience and satisfaction, and enhance the user's stickiness and loyalty. By real-time monitoring and analysis of the user's emotional state, the system can promptly discover changes in the user's interest in the content, so as to quickly adjust the recommended content. This helps to improve the efficiency and accuracy of content recommendation and reduce invalid recommendations and waste of resources. The technical solution provides new ideas and methods in the field of human-computer interaction, and helps to promote the intelligent development of human-computer interaction technology. By real-time monitoring and analysis of the user's emotional state, the system can respond to the user's needs and instructions more intelligently, and improve the efficiency and naturalness of human-computer interaction. The technical solution can also provide strong support for psychological research and application. By analyzing the user's eye data information, we can deeply understand the human's emotional state and cognitive process, and provide new perspectives and methods for psychological research. This technical solution has a wide range of application fields and scenarios, such as online education, e-commerce, virtual reality, etc. In these fields, by real-time monitoring and analysis of the user's emotional state, personalized services and support can be provided for teaching, shopping, entertainment, etc., improving user experience and satisfaction. The successful application of this technical solution will promote technological innovation and development in related fields. For example, in the fields of artificial intelligence and computer vision, by continuously optimizing algorithms and models, the accuracy and efficiency of emotion recognition can be further improved, providing technical support for more application scenarios.

[0041] At the same time, by introducing the two key indicators of fixation duration and blinking frequency, and combining them with the content browsing coefficient, the scheme can more carefully capture the emotional changes of users when browsing online content. This comprehensive judgment method of multiple dimensions makes the emotional judgment results more accurate and refined. The scheme further analyzes the fixation duration and blinking frequency only when the content browsing coefficient is lower than the preset threshold, which helps to reduce unnecessary calculations and improve judgment efficiency. At the same time, through the preset judgment coefficient threshold, the user's interest in the current content can be quickly judged to reduce the judgment time. The refined emotional judgment results can provide a basis for personalized recommendations and content optimization, thereby improving the user experience. For example, when it is determined that the user has a preliminary interest in the current content, more relevant content can be recommended; when it is determined that the user has no interest, the content presentation method can be adjusted or other content can be recommended. The scheme can adapt to the browsing habits and emotional expressions of different users. Since each person's browsing behavior and blinking frequency are different, the scheme can more flexibly adapt to the characteristics of different users by comprehensively considering multiple indicators, thereby improving the adaptability of the system. This solution provides a new idea and method for the field of emotional computing. By combining eye data information and content browsing coefficients and other parameters, it realizes the quantitative analysis and judgment of user emotions. This will help promote the further development of emotional computing technology and provide new inspiration and reference for research and application in related fields.

[0042] By introducing the minimum constant e of the pre-trial, the scheme effectively prevents the situation where the denominator is 0, and enhances the robustness and stability of the algorithm. At the same time, by considering the maximum blinking frequency Fmax that occurs during the user's browsing process, the judgment result is more reasonable and reliable.

[0043] In summary, the technical effects of this technical solution in terms of performance indicators are mainly reflected in the refinement of emotion judgment, improvement of judgment efficiency, optimization of user experience, enhancement of system adaptability, promotion of the development of emotion computing technology, as well as robustness and stability. These technical effects make this solution have higher practical value and promotion potential in practical applications. At the same time, this technical solution realizes the primary emotion judgment of users browsing network content by combining the fixation point dwell time, blinking frequency and content browsing coefficient in the eye data information. This technical solution has significant technical effects and application value, can provide users with a more personalized and intelligent service experience, and promote technological innovation and development in related fields.

[0044] In one embodiment of the present invention, judging whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result includes: S301, retrieve the primary emotion determination result; S302: when the primary emotion determination result indicates that the user has no initial interest in the currently browsed web content, the data collection frequency of the user's facial data information is not adjusted; S303. When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the data collection frequency of the user's facial data information is adjusted.

[0045] The working principle of the above technical solution is as follows: First, the system needs to retrieve the primary emotion judgment result obtained through the analysis of eye data information. This result is calculated based on the user's gaze dwell time on the current web content, blinking frequency, etc., which can reflect the user's initial interest in the content.

[0046] Next, the system determines the user's interest in the current content based on the primary emotion judgment results. If the user does not have a preliminary interest, the system believes that the current data collection frequency is sufficient and no adjustment is required to maintain efficient resource utilization and reduce unnecessary data processing. On the contrary, if the user has a preliminary interest, the system believes that it is necessary to capture the user's facial data information in more detail in order to more deeply analyze the user's emotions and reactions. Therefore, the system will increase the data collection frequency accordingly to obtain more and more accurate user facial data.

[0047] The effect of the above technical solution is that by adjusting the data collection frequency according to the user's interest level, the technical solution can more effectively utilize system resources. When the user is not interested in the content, reducing the collection frequency can reduce the demand for data storage and processing, thereby reducing the system load and energy consumption. When the user is interested, increasing the collection frequency can ensure that more useful facial data information is captured, providing strong support for subsequent in-depth analysis and application. The technical solution can also improve the user experience. By analyzing the user's emotional state in real time and adjusting the data collection strategy, the system can respond to the user's needs more intelligently. For example, when the user is interested in a certain content, the system can immediately increase the data collection frequency to more accurately capture the user's facial reaction and emotional changes. This helps the system to understand the user more deeply, thereby providing more personalized and considerate services. Increasing the data collection frequency can ensure that more and more accurate facial data information is obtained. This information is crucial for subsequent applications such as emotion analysis and user behavior prediction. By increasing the amount of data and improving the data quality, the technical solution can significantly improve the accuracy and reliability of emotion recognition and provide stronger support for related applications. The successful application of this technical solution will promote technological innovation and development in related fields. For example, in the fields of artificial intelligence and computer vision, the efficiency and accuracy of emotion recognition can be further improved by continuously optimizing data collection and analysis algorithms. At the same time, this technical solution can also provide new ideas and methods for applications in other fields, and promote cross-field technological innovation and cooperation. In addition, this technical solution can also provide strong support for psychological research and applications. By analyzing the user's facial data information in real time, we can gain an in-depth understanding of human emotional states and cognitive processes. This helps psychologists understand human emotions and behavior patterns more accurately, and provides new perspectives and methods for applications such as psychological counseling and psychotherapy. This technical solution is highly adaptable. It can flexibly adjust the frequency of data collection according to the interests and needs of different users, thereby meeting the application needs in different scenarios. This flexibility enables this technical solution to be widely used in multiple fields such as online education, e-commerce, and virtual reality.

[0048] In summary, this technical solution achieves optimal resource utilization, improved user experience, enhanced data accuracy and reliability, and promoted technological innovation and development by adjusting the data collection frequency of user facial data information according to the primary emotion judgment results. These technical effects make this technical solution have broad application prospects and important practical value.

[0049] In one embodiment of the present invention, when the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed network content, the frequency of data collection of the user's facial data information is adjusted, including: S3031, when the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the coordinate change data corresponding to the eyebrow mark point and the coordinate change data corresponding to the mouth mark point are retrieved; S3032, acquiring the coordinate change frequency and position change amplitude corresponding to each eyebrow marking point according to the coordinate change data corresponding to the eyebrow marking point; S3033, obtaining the coordinate change frequency and position change amplitude corresponding to each mouth mark point according to the coordinate change data corresponding to the mouth mark point; S3034. Obtain a first action determination coefficient using the coordinate change frequency and position change amplitude corresponding to each eyebrow mark point, wherein the first action determination coefficient is obtained by the following formula: Among them, R 01 represents the first action determination coefficient; m represents the total number of eyebrow marking points; f mi represents the coordinate change frequency corresponding to the i-th eyebrow marker point; a mi represents the position change amplitude corresponding to the i-th eyebrow marker; α represents the angle between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; L represents the distance between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; S3035. Obtain a second action determination coefficient using the coordinate change frequency and position change amplitude corresponding to the mouth mark point, wherein the second action determination coefficient is obtained by the following formula: Among them, R 02 represents the second action determination coefficient; k represents the total number of mouth marking points; f ki Indicates the coordinate change frequency corresponding to the i-th mouth mark point; a ki Represents the position change amplitude corresponding to the i-th mouth marker; α represents the angle between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; L represents the distance between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; β represents the angle between the mouth marker with the minimum position change amplitude corresponding to the k mouth markers and the mouth marker with the maximum position change amplitude; D represents the distance between the mouth marker with the minimum position change amplitude corresponding to the k mouth hair markers and the mouth marker with the maximum position change amplitude; S3036, retrieve the initial emotion determination coefficient; S3037, adjusting the data collection frequency of the facial data information by using the first action determination coefficient and the second action determination coefficient in combination with the initial emotion determination coefficient, and collecting facial data information according to the adjusted data collection frequency; The adjusted data collection frequency is obtained by the following formula: Where F represents the adjusted data collection frequency; F0 represents the adjusted data collection frequency; R 01 Represents the first action determination coefficient; R 02 Indicates the second action determination coefficient.

[0050] The working principle of the above technical solution is as follows: when the primary emotion judgment result shows that the user has a preliminary interest in the currently browsed network content, the system first retrieves the coordinate change data of the eyebrow and mouth mark points. These data reflect the subtle movements of the user's facial muscles when the user's emotions change, and are an important basis for subsequent analysis. The system further analyzes the coordinate change data of the eyebrow and mouth mark points, and extracts the coordinate change frequency and position change amplitude of each mark point. These features can quantify the dynamic changes of the user's face when expressing emotions, and provide key data for subsequent action judgment. Using the extracted eyebrow and mouth action features, the system calculates the first action judgment coefficient and the second action judgment coefficient respectively. These two coefficients comprehensively consider factors such as the coordinate change frequency, position change amplitude of the mark points, and the relative position and angle between the mark points, and can fully reflect the overall dynamic characteristics of the user's face when expressing emotions. The system also retrieves the initial emotion judgment coefficient calculated previously, which reflects the user's initial interest in the currently browsed content. Finally, the system combines the first action judgment coefficient, the second action judgment coefficient, and the initial emotion judgment coefficient to calculate the adjusted data collection frequency through a specific formula. This frequency can more accurately capture the subtle facial movements of users when expressing emotions, providing stronger support for subsequent sentiment analysis and applications.

[0051] The effect of the above technical solution is: by deeply analyzing the user's facial data information, especially the changes in the coordinates of the eyebrow and mouth markers, the technical solution can more accurately capture the subtle facial movements of the user when expressing emotions. This helps to improve the accuracy of data collection and provide more powerful data support for subsequent emotional analysis and applications. The technical solution can dynamically adjust the data collection frequency according to the user's emotional state and facial movement characteristics. This dynamic adjustment strategy helps to optimize data collection efficiency and improve resource utilization. By comprehensively considering the movement characteristics of eyebrows and mouth and the initial emotional judgment coefficient, the technical solution can more comprehensively evaluate the user's emotional state. This helps to improve the accuracy of emotion recognition and provide more reliable results for subsequent emotional analysis and applications. The technical solution can more accurately capture the user's emotional state and facial movement characteristics, thereby providing users with more personalized and considerate services. For example, in the field of online education, the system can adjust the teaching content and rhythm according to the students' emotional state to improve learning effects and satisfaction. In the field of e-commerce, the system can recommend goods and services that are more in line with their needs and interests based on the user's emotional response. The technical solution can also provide strong support for psychological research and applications. By deeply analyzing the user's facial data information, psychologists can more accurately understand the human emotional state and cognitive process, and provide new perspectives and methods for applications such as psychological counseling and psychotherapy. The successful application of this technical solution will promote technological innovation and development in related fields. For example, in the fields of artificial intelligence and computer vision, the efficiency and accuracy of emotion recognition can be further improved by continuously optimizing data collection and analysis algorithms. At the same time, this technical solution can also provide new ideas and methods for applications in other fields, and promote cross-domain technological innovation and cooperation. This technical solution has strong adaptability. It can flexibly adjust the data collection frequency and strategy according to the emotional state and facial movement characteristics of different users, so as to meet the application needs in different scenarios. This flexibility enables this technical solution to be widely used in many fields such as online education, e-commerce, virtual reality, etc.

[0052] At the same time, by dynamically adjusting the data collection frequency of facial data information according to the user's initial interest (determined by the primary emotion judgment result), the scheme can increase the density of data collection when the user shows higher interest, thereby more accurately capturing the user's emotional response and subtle movement changes. This dynamic adjustment helps to improve the relevance and accuracy of the data while maintaining data processing efficiency. By introducing the first action judgment coefficient and the second action judgment coefficient, the scheme can comprehensively consider the coordinate change frequency and position change amplitude of the eyebrows and mouth, as well as geometric factors such as the relative position and angle between them. This precise calculation method can more accurately reflect the user's facial movement characteristics and provide a more reliable basis for subsequent emotion analysis and data processing. By combining the initial emotion judgment coefficient, the first action judgment coefficient and the second action judgment coefficient to adjust the data collection frequency, the scheme achieves the coordinated optimization between emotion judgment and data collection frequency. This coordinated optimization helps to improve the pertinence and effectiveness of data collection while maintaining user privacy and data security. Since the data collection frequency can be dynamically adjusted according to the user's interest, the scheme can reduce unnecessary data collection and processing when the user's interest is low, thereby saving computing resources and storage space. At the same time, increasing the data collection density when the user's interest is high helps to more accurately capture and analyze the user's emotional response, improving the overall performance of the system and user experience. This solution introduces adjustable parameters and formulas to calculate the action determination coefficient and data collection frequency, allowing the system to be flexibly configured and optimized according to different application scenarios and user needs.

[0053] In summary, the technical solution has achieved many technical effects in terms of performance indicators, including dynamic adjustment of data collection frequency, accurate calculation of action determination coefficient, coordinated optimization of emotion determination and data collection frequency, performance improvement and resource optimization, as well as adaptability and scalability. At the same time, the technical solution has achieved improved accuracy of data collection, optimized efficiency, and enhanced accuracy of emotion recognition by deeply analyzing the user's facial data information and dynamically adjusting the data collection frequency according to the user's emotional state and facial action characteristics. These technical effects make the technical solution have broad application prospects and important practical value. At the same time, the technical solution also provides strong support for psychology research and applications, and promotes technological innovation and development in related fields.

[0054] In one embodiment of the present invention, when the primary emotion determination result indicates that the user is interested in the browsed network content, a secondary emotion determination is performed using facial data information to obtain a secondary emotion determination result, including: S401, when the primary emotion determination result indicates that the user is interested in the browsed network content, retrieving facial data information; S402, performing data preprocessing on the facial data information to obtain preprocessed facial data information; wherein the data preprocessing includes noise reduction processing and invalid value removal processing; S403, inputting the pre-processed facial data information into a deep learning model to perform facial expression recognition and expression degree recognition, and obtaining facial expression types and emotion degrees corresponding to the facial expression types; wherein the deep learning model adopts a convolutional neural network model.

[0055] The structure of the deep learning model is as follows: Input layer, used to receive pre-processed facial data information; The convolution layer is used to extract spatial features from the received processed facial data information to obtain spatial features; The flattening layer is used to flatten the spatial features of the output of the convolutional layer into a one-dimensional array; The fully connected layer is used to integrate the one-dimensional array corresponding to the extracted spatial features and learn global information; The output layer is used to output the expression recognition results and the corresponding emotional level of the expression; The expression recognition results include boredom, indifference, happiness, sadness, anger and surprise; and the emotion levels include normal and strong.

[0056] The working principle of the above technical solution is as follows: First, the system uses primary emotion judgment methods (such as analysis based on user behavior, click-through rate, dwell time and other data) to determine whether the user is interested in the current web content. When the user shows interest, the system immediately retrieves the user's facial data information to provide basic data for subsequent emotion judgment.

[0057] After acquiring facial data information, the system preprocesses it to eliminate noise and invalid values ​​to improve the accuracy and reliability of the data. The data preprocessing steps include noise reduction and invalid value removal to ensure that subsequent analysis can be based on high-quality data.

[0058] The pre-processed facial data information is input into the deep learning model for emotion judgment. The deep learning model uses a convolutional neural network (CNN) structure, which can automatically extract the spatial features of facial images, learn global information through a fully connected layer, and finally output the expression recognition results and the corresponding emotional level of the expression.

[0059] Input layer: receives preprocessed facial data information as input of the model.

[0060] Convolution layer: extracts spatial features from the input facial data information and captures the key features in the facial image through convolution operations.

[0061] Flattening layer: Flattens the spatial features output by the convolutional layer into a one-dimensional array for subsequent processing.

[0062] Fully connected layer: Integrate the flattened features, learn global information, and output expression recognition results and preliminary judgments on the degree of emotion.

[0063] Output layer: Based on the output of the fully connected layer, the user’s facial expression type (such as boredom, indifference, happiness, sadness, anger, surprise) and the degree of emotion corresponding to the expression (such as normal, strong) are finally determined.

[0064] The effect of the above technical solution is: through the automatic feature extraction and global information learning of the deep learning model, the technical solution can more accurately identify the user's facial expression type and emotional level. Compared with the traditional rule-based or manual feature extraction method, the deep learning model has stronger adaptability and robustness, and can cope with complex and changeable facial expressions and emotional changes. The technical solution can capture the user's emotional state in real time and adjust the presentation method or recommendation strategy of network content according to the user's interest level and emotional response. For example, when the user shows strong interest or pleasant emotions, the system can recommend more similar or related content; when the user shows boredom or dissatisfaction, the system can adjust the content or provide other types of entertainment services. This personalized service can significantly improve user experience and satisfaction. The technical solution can provide strong support for sentiment analysis applications. By deeply analyzing the user's facial expressions and emotional levels, the system can understand the user's emotional tendencies and preferences, and provide valuable data and insights for sentiment analysis, user portrait construction, market research and other fields. The successful application of this technical solution demonstrates the great potential of deep learning models in the field of emotion judgment. With the continuous advancement of technology and the in-depth expansion of applications, deep learning models will play an important role in more fields and promote the overall development of artificial intelligence technology. By capturing and analyzing the user's facial expressions and emotional levels in real time, this technical solution can provide more intelligent and humanized services for human-computer interaction. For example, in virtual reality or augmented reality applications, the system can adjust the behavior or expression of the virtual character according to the user's emotional state to enhance the user's immersion and interactivity. The introduction of data preprocessing steps, such as noise reduction and invalid value removal, can significantly improve the efficiency and accuracy of subsequent emotional judgment. By eliminating noise and invalid values, the system can focus more on useful facial data information and improve the accuracy and stability of emotional judgment. This technical solution can also provide support in the fields of emotional education and mental health. By analyzing the user's facial expressions and emotional levels, educators or psychologists can understand the user's emotional state and emotional changes, and provide more accurate and personalized guidance for emotional education and mental health counseling.

[0065] In summary, this technical solution achieves accurate judgment of the user's facial expression type and emotional level through automatic feature extraction and global information learning of the deep learning model. This technical solution not only improves the accuracy of emotional judgment, but also enhances user experience, supports the application of emotional analysis, promotes the development of artificial intelligence technology, promotes the intelligence of human-computer interaction, and improves the efficiency and accuracy of data processing. At the same time, it also provides strong support for the fields of emotional education and mental health, and has broad application prospects and important practical value.

[0066] In one embodiment of the present invention, determining the push ratio of content similar to the network content currently browsed by the user according to the secondary emotion determination result includes: S501, when the secondary emotion determination result determines that the user's expression when browsing the current network content is bored or indifferent, and the emotion level is strong, similar content to the current network content is not pushed; S502, when the secondary emotion determination result determines that the user's expression when browsing the current network content is bored or indifferent, and the emotion level is average, the push ratio of the content similar to the current network content is set to 3%-8%; that is, the push ratio of the content similar to the current network content contained in the content pushed every day is 3%-8%; S503, when the secondary emotion determination result determines that the user's expression when browsing the current network content is any one of happiness, sadness, anger and surprise, and the emotion level is average, the push ratio of the content similar to the current network content is set to 8%-15%; that is, the push ratio of the content similar to the current network content contained in the content pushed every day is 8%-15%; S504. When the secondary emotion determination result determines that the user's expression when browsing the current web content is any one of happiness, sadness, anger and surprise, and the emotion level is strong, the push ratio of content similar to the current web content is set to 15%-28%. That is, the push ratio of content similar to the current web content contained in the content pushed every day is 15%-28%;

[0067] The working principle of the above technical solution is as follows: when the user shows strong boredom or indifference, the system believes that the user is not interested in the current content, so it decides not to push similar content to avoid the user from feeling bored or having negative emotions. When the user shows a general degree of boredom or indifference, the system believes that although the user is not very interested in the current content, he may still accept the push of some similar content. Therefore, the system sets a lower push ratio (3%-8%) to test the user's reaction while avoiding excessive push. When the user shows a general degree of happiness, sadness, anger or surprise, the system believes that the user has a certain interest or emotional resonance in the current content. Therefore, the system sets a moderate push ratio (8%-15%), aiming to maintain the user's interest while giving them more relevant content choices. When the user shows strong happiness, sadness, anger or surprise, the system believes that the user is very interested in the current content or has a strong emotional response. Therefore, the system sets a higher push ratio (15%-28%) to provide more similar content to meet the user's emotional needs and desire for exploration.

[0068] The effect of the above technical solution is: by accurately analyzing the user's emotional state, the system can more accurately judge the user's interest and preference for different content. This personalized push strategy not only reduces unnecessary interruptions, but also increases the probability of users receiving content of interest, thereby improving the user's overall experience. When users feel that the content pushed by the system is highly consistent with their interests and emotional state, they are more likely to continue to use the service and generate higher stickiness. This stickiness not only helps to improve user satisfaction, but also helps to promote the dissemination and sharing of content, forming a virtuous circle. By dynamically adjusting the push ratio, the system can allocate resources more effectively to ensure that the content can be accurately delivered to interested users. This optimized distribution strategy not only improves the exposure rate of content, but also reduces the cost of invalid pushes and improves the overall content distribution efficiency. When users receive content similar to their current emotional state, they are more likely to have emotional resonance and are willing to share this content with others. This emotional resonance and social interaction can not only enhance the user's emotional connection, but also promote the dissemination of content and the expansion of influence. The personalized push strategy makes users feel valued and concerned, thereby improving their satisfaction and loyalty. When users believe that the system can accurately understand their needs and preferences, they are more likely to become long-term loyal users and are willing to provide more feedback and support to the system. By collecting and analyzing users' emotional data, the system can provide valuable feedback and insights to content creators. These feedback and insights not only help creators understand users' interests and preferences, but also guide them to create content that better meets user needs, thereby improving the quality and attractiveness of the content. The successful application of this technical solution demonstrates the great potential of artificial intelligence technology in the field of sentiment analysis and personalized recommendation. With the continuous advancement of technology and the in-depth expansion of applications, artificial intelligence will play an important role in more fields and promote innovation and development of the entire industry.

[0069] In summary, this technical solution not only improves user experience and stickiness, but also optimizes content distribution efficiency, promotes emotional resonance and social interaction, improves user satisfaction and loyalty, and provides valuable data support for content creators by accurately analyzing the user's emotional state and dynamically adjusting the push strategy. These technical effects jointly promote the further development of artificial intelligence technology and bring more value and opportunities to users and content creators.

[0070] The embodiment of the present invention proposes an emotional feedback system based on content recognition, such as Figure 2 As shown, the emotional feedback system based on content recognition includes: A real-time acquisition module is used to acquire the user's eye data information and facial data information in real time when the user is browsing the network content, and obtain the user's eye data information and facial data information when the user is browsing the network content; A primary emotion determination module, used to determine the primary emotion of the user browsing the network content according to the eye data information of the user when browsing the network content, and obtain a primary emotion determination result; A collection frequency adjustment module, used to determine whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result; A secondary emotion determination module is used to perform secondary emotion determination using facial data information to obtain a secondary emotion determination result when the primary emotion determination result indicates that the user is interested in the browsed network content; The push setting module is used to determine the push ratio of content similar to the network content currently browsed by the user according to the secondary emotion judgment result.

[0071] The working principle of the above technical solution is: when the user browses the content on the network platform, the system will collect the user's eye data information and facial data information in real time. These data include the user's gaze direction, pupil size changes, facial expression changes, etc., which can reflect the user's emotional response and interest level during the browsing process. The system will judge the primary emotion of the user browsing the network content based on the collected eye data information, such as gaze time, pupil changes, etc. For example, long-term gaze and pupil dilation may indicate that the user is interested in the current content, while rapid scanning and pupil constriction may indicate that the user is not interested or bored. According to the primary emotion judgment result, the system will dynamically adjust the data collection frequency of the user's facial data information. If the user shows a strong interest in the content, the system will increase the data collection frequency to more accurately capture the user's emotional changes; conversely, if the user shows disinterest or boredom, the system will appropriately reduce the data collection frequency to reduce unnecessary resource consumption. When the primary emotion judgment result shows that the user is interested in the browsed network content, the system will further use the facial data information for secondary emotion judgment. This time the judgment will be more detailed and in-depth, by analyzing the user's facial expressions, muscle movements, etc., to more accurately judge the user's emotional state. Based on the results of the secondary emotional judgment, the system will determine the push ratio of content similar to the web content currently browsed by the user. If the user shows strong interest and positive emotional response, the system will increase the push ratio of similar content to meet the user's further needs; conversely, if the user shows no interest or negative emotional response, the system will reduce the push ratio of similar content to avoid the user from feeling bored.

[0072] The effect of the above technical solution is: by collecting and analyzing the user's eye and facial data information in real time, the system can more accurately capture the user's emotional response and interest level, thereby providing users with more personalized content recommendations and better service experience. This technical solution can dynamically adjust the content recommendation algorithm based on the user's real-time emotional feedback to make it more in line with the user's interests and needs. This helps to improve the accuracy of content recommendations and user satisfaction. At the same time, by dynamically adjusting the data collection frequency, the system can reduce unnecessary resource consumption and data collection while ensuring recognition accuracy. The application of this technical solution helps to promote human-computer interaction in a more intelligent and personalized direction. By analyzing the user's emotional response and interest level in real time, the system can provide users with a more intelligent and considerate service experience.

[0073] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for emotional feedback based on content recognition, characterized in that: The emotional feedback method based on content recognition includes: When the user browses the network content, the user's eye data information and facial data information are collected in real time to obtain the user's eye data information and facial data information when the user browses the network content; Determine the primary emotion of the user browsing the network content according to the eye data information of the user when browsing the network content, and obtain the primary emotion determination result; Determining whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result; When the primary emotion determination result indicates that the user is interested in the browsed web content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result; The push ratio of content similar to the network content currently browsed by the user is determined according to the secondary emotion determination result.

2. The emotional feedback method based on content recognition according to claim 1, characterized in that: When a user browses online content, the user's eye data information and facial data information are collected in real time, and the eye data information and facial data information of the user when browsing online content are obtained, including: When the user browses the network content, the initial eye data collection frequency corresponding to the eye data information collection and the initial facial data collection frequency corresponding to the facial data information are retrieved from the database; Collecting eye data information of the user when browsing online content according to the initial eye data collection frequency to obtain the eye data information of the user when browsing online content; The eye data information includes the duration of the fixation point, the scanning speed during the fixation point transfer process, and the blinking frequency corresponding to each fixation point; Collecting facial data information of the user when browsing online content according to the initial facial data collection frequency, and obtaining facial data information of the user when browsing online content; The facial data information includes coordinate change data corresponding to the eyebrow mark point, coordinate change data corresponding to the mouth mark point and coordinate change data corresponding to the eye mark point.

3. The emotional feedback method based on content recognition according to claim 1, characterized in that: The primary emotion determination result of the user browsing the network content is obtained based on the eye data information of the user when browsing the network content, including: Extracting eye data information of the user when browsing online content; wherein the eye data information includes the duration of fixation, the scanning speed during the fixation transfer process, and the blinking frequency corresponding to each fixation point; Obtaining a content browsing coefficient using the scanning speed of the gaze point transfer process contained in the eye data information when the user browses the network content; The content browsing coefficient is obtained by the following formula: Where C represents the content browsing coefficient; n represents the total number of fixations; S ci represents the outgoing rate corresponding to the i-th fixation point; S ri V represents the transition rate corresponding to the i-th fixation point; i represents the value of the visual complexity of the area corresponding to the i-th fixation point; T si represents the repetitive fixation rate corresponding to the ith fixation point; V max Indicates the maximum value of the visual complexity of the area corresponding to n fixation points; S rmax represents the maximum value of the transfer rate corresponding to n fixation points; S rm The transfer rate of the area corresponding to the fixation point corresponding to the maximum value of visual complexity; Comparing the content browsing coefficient with a preset browsing coefficient threshold; When the content browsing coefficient is lower than a preset browsing coefficient threshold, the primary emotion judgment of the user browsing the network content is made using the gaze point dwell time and the blinking frequency corresponding to each gaze point contained in the eye data information to obtain a primary emotion judgment result.

4. The emotional feedback method based on content recognition according to claim 3 is characterized in that: The primary emotion judgment of the user browsing the web content is obtained by using the duration of the gaze point and the blinking frequency corresponding to each gaze point included in the eye data information, including: When the content browsing coefficient is lower than a preset browsing coefficient threshold, the gaze point dwelling time corresponding to each gaze point and the blinking frequency corresponding to each gaze point included in the eye data information are retrieved; Retrieving the content browsing coefficient; The initial emotion determination coefficient is obtained by using the gaze point dwell time corresponding to each gaze point and the blinking frequency corresponding to each gaze point contained in the eye data information in combination with the content browsing coefficient; The initial emotion determination coefficient is obtained by the following formula: Where E represents the initial emotion determination coefficient; n represents the total number of fixations; D i represents the duration of the fixation point corresponding to the i-th fixation point; F i represents the blink frequency corresponding to the i-th fixation point; e represents the minimum constant of the pre-trial, which is used to prevent the denominator from being 0; F max Indicates the maximum blink frequency of the user during browsing; Comparing the initial emotion determination coefficient with a preset determination coefficient threshold; When the initial emotion determination coefficient exceeds a preset determination coefficient threshold, it is determined that the user has a preliminary interest in the currently browsed web content; When the initial emotion determination coefficient does not exceed the preset determination coefficient threshold, it is determined that the user has no initial interest in the currently browsed network content.

5. The emotional feedback method based on content recognition according to claim 1, characterized in that: Determining whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result includes: Retrieve primary emotion determination results; When the primary emotion determination result indicates that the user has no initial interest in the currently browsed web content, the data collection frequency of the user's facial data information is not adjusted; When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the data collection frequency of the user's facial data information is adjusted.

6. The emotional feedback method based on content recognition according to claim 5, characterized in that: When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the data collection frequency of the user's facial data information is adjusted, including: When the primary emotion determination result indicates that the user has a preliminary interest in the currently browsed web content, the coordinate change data corresponding to the eyebrow mark point and the coordinate change data corresponding to the mouth mark point are retrieved; Obtaining the coordinate change frequency and position change amplitude corresponding to each eyebrow marking point according to the coordinate change data corresponding to the eyebrow marking point; Obtaining the coordinate change frequency and position change amplitude corresponding to each mouth mark point according to the coordinate change data corresponding to the mouth mark point; The first action determination coefficient is obtained by using the coordinate change frequency and position change amplitude corresponding to each eyebrow mark point, wherein the first action determination coefficient is obtained by the following formula: Among them, R 01 represents the first action determination coefficient; m represents the total number of eyebrow marking points; f mi represents the coordinate change frequency corresponding to the i-th eyebrow marker point; a mi represents the position change amplitude corresponding to the i-th eyebrow marker; α represents the angle between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; L represents the distance between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; The second action determination coefficient is obtained by using the coordinate change frequency and position change amplitude corresponding to the mouth mark point, wherein the second action determination coefficient is obtained by the following formula: Among them, R 02 represents the second action determination coefficient; k represents the total number of mouth marking points; f ki Indicates the coordinate change frequency corresponding to the i-th mouth mark point; a ki Represents the position change amplitude corresponding to the i-th mouth marker; α represents the angle between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; L represents the distance between the eyebrow marker with the minimum position change amplitude corresponding to the m eyebrow markers and the eyebrow marker with the maximum position change amplitude; β represents the angle between the mouth marker with the minimum position change amplitude corresponding to the k mouth markers and the mouth marker with the maximum position change amplitude; D represents the distance between the mouth marker with the minimum position change amplitude corresponding to the k mouth hair markers and the mouth marker with the maximum position change amplitude; Retrieve the initial emotion determination coefficient; Using the first action determination coefficient and the second action determination coefficient in combination with the initial emotion determination coefficient, the data collection frequency of the facial data information is adjusted, and facial data information is collected according to the adjusted data collection frequency; The adjusted data collection frequency is obtained by the following formula: Where F represents the adjusted data collection frequency; F0 represents the adjusted data collection frequency; R 01 Represents the first action determination coefficient; R 02 Indicates the second action determination coefficient.

7. The emotional feedback method based on content recognition according to claim 1, characterized in that: When the primary emotion determination result indicates that the user is interested in the browsed network content, a secondary emotion determination is performed using the facial data information to obtain a secondary emotion determination result, including: When the primary emotion determination result indicates that the user is interested in the browsed web content, facial data information is retrieved; Performing data preprocessing on the facial data information to obtain preprocessed facial data information; wherein the data preprocessing includes noise reduction processing and invalid value removal processing; The preprocessed facial data information is input into a deep learning model to perform facial expression recognition and expression degree recognition, and obtain the facial expression type and the emotion degree corresponding to the facial expression type; wherein the deep learning model adopts a convolutional neural network model.

8. The emotional feedback method based on content recognition according to claim 7, characterized in that: The structure of the deep learning model is as follows: Input layer, used to receive pre-processed facial data information; The convolution layer is used to extract spatial features from the received processed facial data information to obtain spatial features; The flattening layer is used to flatten the spatial features of the output of the convolutional layer into a one-dimensional array; The fully connected layer is used to integrate the one-dimensional array corresponding to the extracted spatial features and learn global information; The output layer is used to output the expression recognition results and the corresponding emotional level of the expression; The expression recognition results include boredom, indifference, happiness, sadness, anger and surprise; and the emotion levels include normal and strong.

9. The emotional feedback method based on content recognition according to claim 1, characterized in that: Determining the push ratio of content similar to the network content currently browsed by the user according to the secondary emotion determination result includes: When the secondary emotion determination result determines that the user's expression when browsing the current web content is bored or indifferent, and the emotion level is strong, similar content to the current web content will not be pushed; When the secondary emotion judgment result determines that the user's expression when browsing the current web content is bored or indifferent, and the emotional level is average, the push ratio of similar content to the current web content is set to 3%-8%; When the secondary emotion judgment result determines that the user's expression when browsing the current web content is any one of happiness, sadness, anger and surprise, and the emotional level is average, the push ratio of similar content to the current web content is set to 8%-15%; When the secondary emotion judgment result determines that the user's expression when browsing the current network content is any one of happiness, sadness, anger and surprise, and the degree of emotion is strong, the push ratio of similar content to the current network content is set to 15%-28%.

10. An emotional feedback system based on content recognition, characterized in that: The emotional feedback system based on content recognition includes: A real-time acquisition module is used to acquire the user's eye data information and facial data information in real time when the user is browsing the network content, and obtain the user's eye data information and facial data information when the user is browsing the network content; A primary emotion determination module, used to determine the primary emotion of the user browsing the network content according to the eye data information of the user when browsing the network content, and obtain a primary emotion determination result; A collection frequency adjustment module, used to determine whether to adjust the data collection frequency of the user's facial data information according to the primary emotion determination result; A secondary emotion determination module is used to perform secondary emotion determination using facial data information to obtain a secondary emotion determination result when the primary emotion determination result indicates that the user is interested in the browsed network content; The push setting module is used to determine the push ratio of content similar to the network content currently browsed by the user according to the secondary emotion judgment result.

Images