Dynamic advertisement content intelligent delivery method based on user emotion recognition

Abstract

The invention relates to a dynamic advertisement content intelligent delivery method based on user emotion recognition. According to the method, face image data when a user watches an advertisement are collected in real time through a camera device, the data are input into a computer vision sentiment analysis model, sight focus data and a user experience score are generated, and the user experience score comprises the interestingness, the concentration degree and the sentiment tendency; based on the score, selecting the advertisement content with the highest matching degree with the current user state from a preset advertisement content library, and dynamically adjusting the visual layout and the key information element position of the advertisement content according to the sight focus data identified by the sentiment analysis model; in addition, according to the method, the presentation mode of the advertisement is adjusted according to the real-time change of the user experience score, so that the individuation and interaction experience of the advertisement are enhanced; the method can improve the precise delivery of the advertisement content and the user attention, and is suitable for application scenes such as digital advertisement screens, e-commerce platforms, intelligent retail terminals and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of advertisement delivery, and in particular to a method for intelligent delivery of dynamic advertisement content based on user emotion recognition. Background Art

[0002] In the prior art, advertising delivery mainly relies on personalized recommendations based on user historical behavior, interest tags or demographic characteristics, such as analyzing the preferences of users through browsing history, purchase data or social media interactions to match the corresponding advertising content. In addition, some advertising delivery systems use static display or fixed playback mode, and the content and presentation method remain unchanged regardless of whether the user is interested in the advertisement. In recent years, the development of computer vision technology has enabled advertising systems to use cameras to capture users' facial expressions and eye tracking, thereby analyzing users' emotional states to optimize the push strategy of advertising content.

[0003] The existing advertising delivery system still has certain limitations in terms of accuracy and real-time adjustment capabilities. On the one hand, the recommendation method based on historical behavior cannot timely reflect the user's immediate emotions and attention during the ad viewing process, resulting in some delivery content failing to match the user's current interests. On the other hand, the existing advertising delivery technology that uses computer vision for emotion recognition is usually only used for ad selection, and fails to dynamically adjust key elements such as the visual layout of the ad and the way the information is presented to further enhance the user experience. In addition, the adjustment of ad content often lacks real-time response capabilities and is difficult to adapt to the rapid changes in user emotions, thus affecting the effectiveness of the ad.

[0004] In order to solve the above problems, the present invention provides a method for intelligent delivery of dynamic advertising content based on user emotion recognition. Summary of the invention

[0005] The present application provides a method for intelligent delivery of dynamic advertising content based on user emotion recognition to improve the accurate delivery of advertising content and user attention.

[0006] The present application provides a method for intelligent delivery of dynamic advertising content based on user emotion recognition, comprising: The facial image data of the user when watching the advertisement is collected in real time through a camera device, including facial expressions, eye movement trajectories, gaze lingering positions and lingering time; Inputting the facial image data into a computer vision sentiment analysis model to generate gaze focus data and user experience scores including interest, concentration, and emotional tendency; According to the user experience score, selecting the advertising content that best matches the current user status from a preset advertising content library; Dynamically adjust the visual layout and key information element positions of the selected advertising content based on the sight focus data identified by the computer vision sentiment analysis model; Based on the real-time changes in user experience scores, dynamically adjust the presentation of advertising content, including content type, image-text ratio, and information density.

[0007] The beneficial effects of the technical solution provided by this application include: (1) Based on the real-time emotion recognition results of users, the present invention selects the advertising content that best matches the current user status from the preset advertising content library, thereby improving the relevance of the advertisement, enhancing the user's viewing interest, avoiding invalid or inefficient advertising, and improving the advertising conversion rate. (2) By analyzing the user's visual focus data and emotional tendencies, the visual layout and key information element positions of the advertising content are dynamically adjusted to make the advertising content more in line with the user's visual habits and attention focus areas, thereby improving the readability and attractiveness of the advertisement and enhancing the user's immersion and participation. (3) The present invention can dynamically adjust the presentation of advertisements, including content type, image-text ratio, and information density, according to the real-time changes in the user experience score, so that the advertisement can adapt to changes in user emotions and attention, ensuring that the advertising information is conveyed in the best way, thereby improving the advertising communication effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a flowchart of a method for intelligent delivery of dynamic advertising content based on user emotion recognition provided in the first embodiment of the present application. DETAILED DESCRIPTION

[0009] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0010] The first embodiment of the present application provides a method for intelligent delivery of dynamic advertising content based on user emotion recognition. Figure 1 , which is a schematic diagram of the first embodiment of the present application. Figure 1 The first embodiment of the present application provides a method for intelligent delivery of dynamic advertising content based on user emotion recognition, which is described in detail.

[0011] Step S101: collecting facial image data of a user while watching an advertisement in real time through a camera device, including facial expressions, eye movement trajectories, gaze lingering positions and lingering time.

[0012] In step S101, the facial image data of the user watching the advertisement is collected in real time by a camera device to provide a basis for emotion recognition and dynamic adjustment of the advertisement. The camera device can be a high-resolution camera, an infrared camera, or a camera device with depth perception capability to ensure that the user's facial expression, eye movement trajectory, and the position and duration of the sight can be accurately captured. The installation position of the camera device can be optimized according to the application scenario. For example, in front of an indoor digital advertising screen, the camera device can be embedded above or below the display screen to obtain the best user facial perspective; on a self-service terminal or smart retail device, the camera device can be installed on the top of the screen to achieve frontal capture of the user's face.

[0013] The collection of facial image data uses high frame rate video capture technology to reduce motion blur and improve the recognition accuracy of key facial features. The collected data includes but is not limited to facial expression features, such as eyebrow curvature, mouth corner curve, eyelid opening and closing status, etc. These features can be used to judge the user's emotional state. The eye movement trajectory is determined by detecting the continuous change of the pupil center position. It is usually identified by computer vision technologies such as optical flow method and Hough transform, and the trajectory data is smoothed by Kalman filter or LSTM (long short-term memory network) to reduce the collection error. The position of the line of sight is determined based on the mapping relationship between the line of sight direction and the screen content area. The camera device can combine the screen coordinate system and three-dimensional depth information to calculate the specific position of the user's line of sight through regression analysis or geometric transformation. In addition, the measurement of the dwell time is based on the continuity of the line of sight focus. If the user's line of sight stays in a certain advertising area for more than a set threshold, the area can be recorded as a high-attention area of ​​the user.

[0014] In order to improve the robustness of facial data collection, step S101 can also be combined with multi-frame image fusion technology to improve the accuracy of facial feature detection. In low-light environments, infrared cameras or HDR (high dynamic range) image enhancement technology can be used to ensure the visibility of facial expressions and sight features. For possible facial occlusion problems, such as users wearing glasses, hats or masks, deep learning-based occlusion compensation algorithms can be used, such as using a pre-trained convolutional neural network (CNN) to predict facial features of the occluded part.

[0015] In addition, in order to ensure the real-time nature of the data, the camera device needs to have high-speed data transmission capabilities, and can use USB3.0, PCIe or MIPI transmission protocols to reduce acquisition delays. On the processing side, FPGA (field programmable gate array) or GPU can be used for parallel accelerated computing to improve the processing efficiency of facial image data and ensure the real-time nature of user emotion recognition. For privacy protection, the collected facial data can be processed by edge computing, that is, feature extraction is completed on the local device to avoid transmitting complete image data, so as to reduce the risk of user privacy leakage.

[0016] Through the above steps, the user's facial expressions, eye movement trajectories, gaze position and duration can be obtained efficiently and accurately, providing reliable data support for subsequent sentiment analysis and dynamic adjustment of advertising content.

[0017] Furthermore, the real-time collection of facial image data of the user when watching the advertisement by means of a camera device includes: After detecting that the user's face enters the advertisement viewing area, the exposure parameters of the camera device are dynamically adjusted based on active light adaptation imaging technology to adapt to different ambient lighting conditions; Combined with an adaptive frame rate control algorithm, the sampling frame rate is increased when the user's eye movement frequency is high to accurately capture the instantaneous attention shift, and the sampling frequency is reduced when the user's gaze remains stable to reduce data redundancy and optimize computing efficiency.

[0018] After detecting that the user's face has entered the ad viewing area, the system first uses a face detection algorithm to determine the user's position, and dynamically adjusts the exposure parameters of the camera device through active light adaptation imaging technology to adapt to different ambient lighting conditions. The adjustment of exposure parameters is based on a real-time light intensity analysis module, which detects the current light level through the ambient light sensor of the camera device and calculates the optimal exposure value so that the facial image can maintain high contrast and clarity in both strong light and low light environments. For example, in a strong light environment outdoors, the system will reduce the exposure time and optimize the contrast enhancement algorithm to reduce the loss of details caused by overexposure; in a low light environment, the system will increase the gain parameter or enable infrared fill light to ensure that facial features are still clearly visible.

[0019] In order to ensure high-precision tracking of the user's eye movement trajectory and the position where the gaze stays, the system combines an adaptive frame rate control algorithm to dynamically adjust the frame rate of image acquisition according to the user's eye movement frequency. When the user's gaze moves quickly or the eye movement trajectory changes dramatically, the system will automatically increase the sampling frame rate to capture the changes in attention in a short period of time. For example, when the user's eyes jump quickly between multiple areas of the screen, the system will increase the data collection density to accurately record the process of gaze transfer. When the user's gaze stays in a fixed area for a long time and the eye movement changes little, the system will reduce the sampling frame rate to reduce data redundancy and reduce the computational burden, while ensuring that the recorded dwell time data is still accurate. This adaptive frame rate control mechanism combined with historical eye movement pattern analysis enables the system to collect data in the best way at different stages of the user's viewing of the advertisement, which not only ensures the accuracy of tracking, but also improves the utilization of computing resources.

[0020] Through the above method, the system can stably obtain facial expressions, eye movement trajectories and gaze retention information under different lighting environments and changes in user viewing behavior, providing high-quality data support for subsequent sentiment analysis and advertising content optimization, and improving the accuracy of personalized advertising matching and the effect of user experience optimization.

[0021] Step S102: Input the facial image data into a computer vision sentiment analysis model to generate gaze focus data and a user experience score including interest, concentration, and emotional tendency.

[0022] In step S102, the facial image data is input into a computer vision emotion analysis model to analyze the user's emotional state and generate eye focus data and user experience scores. The process first pre-processes the collected facial image data, including image denoising, illumination compensation, facial key point detection and normalization processing, to improve the stability and accuracy of subsequent analysis. Facial key point detection can use a variety of deep learning models, such as detection based on convolutional neural networks (CNN) or self-supervised learning models to identify the positions of facial feature points such as eyebrows, eyes, and mouth corners, thereby extracting features related to user expressions.

[0023] In the facial expression analysis process, the computer vision sentiment analysis model uses the trained expression classification network to classify the user's expression state, and combines the expression change sequence to predict the emotional trend. For example, if the user's mouth corners are raised and eyes are wide open, it can be determined that the user's emotions tend to be positive, while if frowning and drooping mouth corners are detected, it may indicate dissatisfaction or low interest. In addition, for the user's eye movement trajectory and gaze retention data, the sentiment analysis model tracks the user's gaze direction and combines the screen area division information to calculate the user's gaze focus and determine the distribution of their attention on the advertising content. This process can predict the user's gaze point through corneal reflection detection (PCCR) or deep learning regression methods, and use dynamic time warping (DTW) or Kalman filtering methods to smooth the gaze trajectory to improve tracking accuracy.

[0024] The generation of user experience scores is a comprehensive value calculated based on multi-dimensional emotional features, including key indicators such as interest, concentration and emotional tendency. Interest can be calculated by the user's gaze dwell time, the frequency of facial expression changes and the blinking rate. Longer gazes accompanied by positive expressions such as smiles may indicate higher interest, while frequent blinking or rapid gaze aversion may indicate lower interest. The calculation of concentration usually involves the user's eye movement stability and changes in the area of ​​focus. For example, if a user stares at a specific advertising element for a long time and the eye movement trajectory deviates less, it indicates that the user's concentration is high, while if the gaze changes frequently, it indicates that the concentration is low. Emotional tendency combines expression recognition and semantic sentiment analysis technology, based on expression classification results and time series analysis, to determine whether the user is in a positive, negative or neutral emotional state, and uses time series models such as long short-term memory networks (LSTM) to predict the trend of emotional changes.

[0025] In the above analysis process, pre-trained deep learning models can be used, such as expression recognition networks based on ResNet and VGG, and emotional trend analysis models based on Transformer structures, to improve the accuracy and generalization of calculations. In addition, the calculation of user experience scores can adopt a weighted scoring algorithm, which assigns different weights according to the user's facial expressions, sight data, and attention characteristics to calculate a comprehensive score, thereby more accurately reflecting the user's advertising experience status. In order to improve real-time performance, computer vision sentiment analysis models can be deployed on edge computing devices so that data processing is performed locally to reduce the latency of cloud processing and enhance user privacy protection capabilities. Ultimately, the generated user experience score will serve as the basis for subsequent advertising content selection and dynamic adjustment to optimize the advertising effect.

[0026] Furthermore, the computer vision sentiment analysis model includes a visual feature extraction unit, a sentiment state reasoning unit, and a user experience score calculation unit; The visual feature extraction unit is used to extract multi-dimensional features from the input facial image data, including facial expression features, eye movement trajectory data and sight stop information; the visual feature extraction unit detects facial key points through a convolutional neural network to obtain the user's eyebrows, eyes, and mouth corners. In combination with the optical flow method, the eye movement trajectory and sight movement direction of the user are calculated; the user's sight point is spatially mapped by using a deep learning model combined with the viewing angle parameters of the camera device, thereby generating accurate sight focus data and calibrating the user's gaze area in the advertising picture; The emotional state inference unit is used to synthesize the output data of the visual feature extraction unit to infer the user's real-time emotional state, including interest, concentration and emotional tendency; the emotional state inference unit adopts a time series modeling method based on a long short-term memory network to identify the dynamic changes of the user's emotional state, and combines the self-attention mechanism to optimize the weight allocation of different visual features to improve the accuracy of emotional analysis; by jointly analyzing the change pattern of facial expressions, the stability of eye movement trajectories, the duration and area of ​​gaze retention, the user's interest in the current advertising content is calculated, the user's concentration level is evaluated, and the user's overall emotional tendency is identified; The user experience score calculation unit is used to calculate the final user experience score based on the output result of the emotional state reasoning unit, and optimize the adaptability of the advertising content in combination with the line of sight focus data; the user experience score calculation unit adopts a weighted multi-factor scoring model to comprehensively consider the interest, concentration, emotional tendency and the stability of the user's line of sight focus to generate a user experience score, and makes personalized adjustments in combination with historical viewing behavior and individual preferences.

[0027] In this embodiment, the computer vision sentiment analysis model consists of a visual feature extraction unit, a sentiment state reasoning unit, and a user experience score calculation unit. These units cooperate with each other to extract and analyze the user's sentiment state from the input facial image data, and finally generate key data for dynamic optimization of advertising. The input and output of each unit are interrelated to ensure the consistency and effectiveness of the entire analysis process.

[0028] The main function of the visual feature extraction unit is to extract the key visual features of the user from the input facial image data, including facial expression features, eye movement trajectory data, and gaze retention information. The unit first uses a convolutional neural network to detect the key points of the user's face and identify the tiny movements of the eyebrows, eyes, corners of the mouth, etc. to extract the expression feature vector. Then, the user's eye movement trajectory is calculated in combination with the optical flow method to analyze the direction of his or her gaze movement, the change of the gaze area, and the gaze jump pattern. In addition, the unit uses a deep learning model combined with the viewing angle parameters of the camera device to spatially map the user's gaze point, generate accurate gaze focus data, and calibrate the user's specific gaze area in the advertising screen. The output of this unit includes expression feature vectors, eye movement trajectory data, and gaze focus data, which will be passed as input to the emotional state inference unit for more in-depth emotional analysis.

[0029] After receiving the output of the visual feature extraction unit, the emotional state inference unit infers the user's real-time emotional state based on these data, including interest, concentration and emotional tendency. The unit adopts a time series modeling method based on long short-term memory network, combined with the user's expression feature vector, to analyze the user's expression change trend at different time points to determine the dynamic evolution of emotions. At the same time, the unit uses the self-attention mechanism to optimize the weight distribution of different visual features, so that the eye movement trajectory data is combined with the expression features to calculate the user's interest. The calculation of interest is based on the user's gaze dwell time, eye movement trajectory stability and the intensity of attention to a specific advertising area. If the user stays on an advertising element for a long time and the gaze jumps less, the interest is high. The calculation of concentration depends on the user's eye movement fixation and the amplitude of the gaze deviation in a short period of time. If the user's gaze is too scattered or the gaze point is frequently changed, the concentration is low. The recognition of emotional tendency is based on the time series analysis of facial expressions. The system determines whether the user's current emotion is positive, neutral or negative by identifying the user's micro-expression features, such as a slight upward corner of the mouth or a tightened eyebrow. The output of this unit includes interest, concentration and emotional tendency, which will be input into the user experience score calculation unit to further evaluate the user's overall experience of the current advertising content.

[0030] The user experience score calculation unit receives the output data of the emotional state reasoning unit and calculates the user's final experience score in combination with the sight focus data to optimize the adaptability of the advertising content. The unit adopts a weighted multi-factor scoring model to generate a user experience score by integrating interest, concentration and emotional tendency. During the scoring calculation process, interest is given a higher weight to measure the user's actual attention to the advertising content, concentration is used to evaluate whether the user can continue to pay attention, and emotional tendency is used to judge the user's acceptance of the advertising content. In addition, the unit uses the sight focus data to analyze the user's gaze area distribution, determine the advertising elements that the user mainly pays attention to, and adjust the importance weight of the advertising content when calculating the score. In order to improve the personalized matching of the score, the unit combines the user's historical viewing behavior data to dynamically optimize the scoring results so that it can reflect the user's long-term interest trend. Finally, the output of the unit includes the user experience score, which will serve as the core basis for the adjustment of advertising delivery to ensure that the advertising content can accurately match the user's real-time emotional state, improve the advertising delivery effect and the user's viewing experience.

[0031] Through the collaborative processing of the visual feature extraction unit, the emotional state inference unit and the user experience score calculation unit, the computer vision sentiment analysis model can accurately analyze the user's emotional state from facial image data, and optimize the presentation of advertising content based on line of sight and emotional feedback, making advertising delivery more intelligent and personalized.

[0032] The following is a schematic reference implementation code of the computer vision sentiment analysis model, which includes a visual feature extraction unit, a sentiment state reasoning unit, and a user experience score calculation unit. The code uses the common deep learning libraries TensorFlow and OpenCV for calculation.

[0033] import cv2 import numpy as np import tensorflow as tf from tensorflow.keras.models import Model from tensorflow.keras.layers import LSTM, Dense, Input, Attention import dlib # Visual feature extraction unit class VisualFeatureExtractor: def __init__(self, face_model_path): """Initialize facial key point detection model""" self.detector = dlib.get_frontal_face_detector() self.predictor = dlib.shape_predictor(face_model_path) def extract_facial_features(self, image): """ Extract facial expression features :param image: input image (BGR format) :return: key point coordinates """ gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) faces = self.detector(gray) features = [] for face in faces: landmarks = self.predictor(gray, face) for i in range(68): x, y = landmarks.part(i).x, landmarks.part(i).y features.append((x, y)) return np.array(features) def compute_optical_flow(self, prev_frame, curr_frame): """ Calculate eye movement trajectory and gaze direction (using optical flow method) :param prev_frame: previous frame :param curr_frame: current frame :return: eye movement trajectory vector """ prev_gray = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY) curr_gray = cv2.cvtColor(curr_frame, cv2.COLOR_BGR2GRAY) # Shi-Tomasi corner detection features = cv2.goodFeaturesToTrack(prev_gray, maxCorners=100,qualityLevel=0.3, minDistance=7) if features is not None: next_pts, status, _ = cv2.calcOpticalFlowPyrLK(prev_gray,curr_gray, features, None) flow_vectors = next_pts - features return flow_vectors return None def compute_gaze_focus(self, landmarks): """ Combine facial key points and camera perspective to calculate the gaze focus :param landmarks: facial key point coordinates :return: The focus coordinates (screen position) """ eye_region = landmarks[36:48] # Eye region gaze_x = np.mean(eye_region[:, 0]) gaze_y = np.mean(eye_region[:, 1]) return gaze_x, gaze_y # Emotional state inference unit class EmotionStateInference: def __init__(self, lstm_units=64): """Initialize Long Short-Term Memory (LSTM) network for emotional state modeling""" self.model = self.build_lstm_model(lstm_units) def build_lstm_model(self, lstm_units): """ Build an LSTM model to process time series sentiment data :param lstm_units: number of LSTM hidden layer units :return: LSTM model """ input_data = Input(shape=(10, 3)) # 10 time steps, each with 3 input features (interest, concentration, emotional tendency) lstm = LSTM(lstm_units, return_sequences=True)(input_data) attention = Attention()([lstm, lstm]) dense = Dense(3, activation='softmax')(attention) # Output 3 emotional states model = Model(inputs=input_data, outputs=dense) model.compile(optimizer='adam', loss='categorical_crossentropy') return model def infer_emotion_state(self, facial_features, eye_movement, gaze_focus): """ Calculate user interest, concentration and emotional tendencies :param facial_features: facial expression features :param eye_movement: eye movement data :param gaze_focus: gaze focus data :return: predicted emotional state """ interest_score = np.linalg.norm(gaze_focus - np.mean(facial_features, axis=0)) # Focus on distance attention_score = 1.0 / (1.0 + np.linalg.norm(eye_movement)) #eye movement stability emotion_score = np.tanh(np.sum(facial_features)) # Emotional tendency (based on facial features) input_data = np.array([[interest_score, attention_score,emotion_score]] 10).reshape(1, 10, 3) return self.model.predict(input_data)[0] # User experience score calculation unit class UserExperienceScoring: def __init__(self): """Initialize user experience score calculation""" self.weights = {"interest": 0.5,"attention": 0.3,"emotion": 0.2} # Rating weight def compute_experience_score(self, emotion_state): """ Calculate the final user experience score :param emotion_state: interest, concentration, and emotional tendency provided by the emotion state inference unit :return: User experience score (between 0-1) """ interest_score, attention_score, emotion_score = emotion_state experience_score = (self.weights["interest"] interest_score + self.weights["attention"] attention_score + self.weights["emotion"] emotion_score) return experience_score # Computer Vision Sentiment Analysis Model class EmotionRecognitionModel: def __init__(self, face_model_path): """Initialize the computer vision sentiment analysis model, including three units""" self.visual_extractor = VisualFeatureExtractor(face_model_path) self.emotion_inference = EmotionStateInference() self.experience_scorer = UserExperienceScoring() def analyze_emotion(self, prev_frame, curr_frame): """ Run the entire analysis process to calculate the user's emotional state and experience score :param prev_frame: previous video frame :param curr_frame: current video frame :return: User experience rating """ facial_features = self.visual_extractor.extract_facial_features(curr_frame) eye_movement = self.visual_extractor.compute_optical_flow(prev_frame, curr_frame) gaze_focus = self.visual_extractor.compute_gaze_focus(facial_features) emotion_state = self.emotion_inference.infer_emotion_state(facial_features, eye_movement, gaze_focus) experience_score = self.experience_scorer.compute_experience_score(emotion_state) return experience_score Training a computer vision sentiment analysis model requires building a multimodal dataset containing facial expressions, eye movement trajectories, and gaze focus data. First, video data of users watching advertisements is collected, and facial key point detection algorithms are used to extract expression features. Eye movement trajectories are calculated using the optical flow method, and gaze focus is mapped in combination with the viewing angle parameters of the camera device. These data are used as input to train a convolutional neural network to extract visual features, and a long short-term memory network is used to model expression changes and eye movement patterns in time series. In order to improve accuracy, a self-attention mechanism is introduced to optimize the weight distribution of visual features so that the model can focus on the most important information. During the training process, a weighted multi-factor scoring model is used to calculate the user experience score, a cross-entropy loss function is used to optimize the emotion classification task, and the error of the user experience score prediction is adjusted based on the mean square error. Through multiple rounds of iterative training and adjusting the model parameters in combination with individual preferences, the user's emotional state is finally accurately predicted, so that the advertising content can be optimized according to real-time emotional feedback.

[0034] Furthermore, the user experience score calculation unit uses the following formula 1 to calculate the user experience score: ; in, Score the end-user experience; Respectively represent the user in The interest, concentration and emotional tendency of each time step ranges from ; For users in The degree of deviation of interest, concentration and emotional tendency at a time step from the average level in the past specified time period; For users in The deviation of interest, concentration and emotional tendency at a time step from the average level in the past specified time period; is the time window size used to calculate the user's historical sentiment trend; is a tiny constant that prevents the denominator from going to zero; is the sentiment weight, which is used to dynamically adjust the influence of each time step in the overall score calculation, and is calculated using the following formula 2: ; in, They are the average interest, concentration and emotional tendency of the user in watching ads in the past; is the personalized adjustment coefficient; is the time window size used to calculate the user's historical sentiment trend.

[0035] In this embodiment, in the process of calculating the user experience score, a weighted scoring model that comprehensively considers the user's emotional changes, individual attention characteristics, and historical emotional trends is adopted to ensure that the advertising content can accurately match the user's real-time emotional state, thereby optimizing the effect of advertising delivery. The user experience score is calculated using formula (1), the core idea of ​​which is to combine the user's emotional characteristics at different time steps and assign dynamic weights to short-term emotional fluctuations to ensure the personalization and adaptability of the score.

[0036] The core variables of formula (1) include the user’s interest, concentration, and emotional tendency at each time step, which are derived from the output of the emotional state inference unit. Represents the user's attraction to the current advertisement, which is usually determined by the user's gaze dwelling time, facial expression changes and eye movement stability. Its value range is between 0 and 1, where 1 represents extremely high interest and 0 represents no interest at all.

[0037] Focus Reflects the user's concentration on the advertising content. This indicator is calculated based on the stability of the eye movement trajectory, the frequency of line of sight jumps, and the user's blinking rate. A higher degree of concentration means that the user may be deeply understanding the advertising content.

[0038] Emotional Tendency It is used to measure the user's overall emotional state. It is usually calculated through facial expression recognition and time series analysis. The value range is also between 0 and 1, with 1 representing positive emotions and 0 representing negative emotions.

[0039] In formula (1), Is the user in The emotional weight of a time step is used to dynamically adjust the contribution of the time step in the overall score calculation. The emotional weight is calculated using formula (2) and is normalized based on the degree of deviation of the user's current emotional state from its historical average level. Respectively represent the user in the past The average interest, concentration, and emotional tendency within a time step are used to measure the user's long-term emotional trend, allowing short-term emotional fluctuations to be compared with historical emotional states. By calculating the deviation of the user's current time step from his or her historical emotional trend and adjusting the weight distribution in combination with an exponential decay function, it ensures that abnormal emotional fluctuations (such as a sudden surge in interest or loss of attention) can be reasonably reflected in the final score calculation.

[0040] In formula (1), the user's sentiment variance It is used to measure the user's emotional fluctuation at the current time step. It calculates the degree of deviation of interest, concentration and emotional tendency from the historical emotional trend. This indicates that the user's emotions fluctuate greatly at this time step, which may mean that a sudden change in mood or the advertisement has a significant impact on the user.

[0041] variance It reflects the user's The degree of deviation of the emotional state (interest, concentration, emotional tendency) in the user from the overall emotional trend of the user. It is used to measure the fluctuation of user emotions, so that when the user's emotions change drastically, the scoring calculation pays more attention to these moments, thereby enhancing the personalized adaptation ability.

[0042] The calculation method is as follows: ; in Respectively represent the user in interest, concentration and emotional tendency at each time step; They are the user's time in the past period of time (such as the past The average interest, concentration, and emotional tendency of watching ads in time steps are calculated as follows: ; The variance of the emotional state at a time step represents the user's The degree of deviation of the sentiment value from its historical average. When the user's sentiment state at a certain time step deviates significantly from its historical trend (such as a sudden increase in interest or a sharp fluctuation in mood), the variance value of this time step will be larger, which will give it a higher weight in the final score calculation and increase the sensitivity to the user's current emotional state. If the user's emotions are relatively stable and the variance is small, the score calculation will rely more on long-term trends, making personalized matching more stable.

[0043] In the score calculation, the ratio term of variance is introduced: ; in represents the total emotional fluctuation in the past N time steps, and is a small constant that prevents the denominator from approaching zero. The recommended value is The purpose of this ratio is to ensure that when emotions fluctuate greatly, the score calculation can pay more attention to short-term emotional changes, thereby adjusting the adaptability of the advertising content.

[0044] Personalized adjustment factor In formula (2), it is used to control the relative influence of interest, concentration and emotional tendency in weight calculation. The recommended value is , , Higher This makes the change in interest level have a greater impact on weight distribution, which is suitable for scenarios where advertising content is driven by user interest, such as entertainment or personalized recommendation ads. This makes the change in concentration have a stronger impact on weight distribution, which is suitable for scenarios where users need to actively understand the content of the advertisement, such as financial or technology product advertisements. It makes the impact of emotional tendencies more obvious and is suitable for emotion-driven advertising, such as public service advertisements or brand story advertisements.

[0045] The time window size N represents the number of time steps used to calculate the user's historical sentiment trend. The recommended value is generally between 10 and 30, depending on the duration of the ad and the average viewing time of the user. A smaller N makes the score calculation more dependent on short-term sentiment changes, while a larger N makes the system pay more attention to long-term sentiment trends to ensure the stability and personalization of the score.

[0046] Finally, the user experience score By taking a weighted sum of the sentiment data at all time steps and adjusting it in combination with sentiment fluctuations, we ensure that the score can fully reflect the user's real experience of the ad. This calculation method enables the system to comprehensively consider long-term trends and short-term sentiment fluctuations when evaluating the user's acceptance of ad content, and to adapt to the behavior patterns of different users through personalized adjustment parameters, thereby improving the accuracy of ad delivery and optimizing the user experience.

[0047] Step S103: According to the user experience score, select the advertisement content that best matches the current user status from a preset advertisement content library.

[0048] In step S103, based on the user experience score, the advertising content with the highest matching degree with the current user status is selected from the preset advertising content library to ensure the accuracy and personalization of advertising delivery. First, the advertising content library needs to be built in advance and contain multiple types of advertising content. Each advertising content should have structured label information, such as advertising type, target audience, brand style, product category, color theme, context adaptability, etc. In addition, each advertising content can also contain historical delivery data, user feedback data, and emotional matching weight to optimize the matching process.

[0049] In order to improve the accuracy of ad matching, the system first needs to parse the user experience score, which is composed of parameters such as interest, concentration, and emotional tendency. Interest reflects the user's potential interest level in the current ad content, concentration indicates the user's concentration when watching the ad, and emotional tendency indicates whether the user's emotional state is positive, neutral, or negative. Based on these indicators, a multidimensional vector can be defined to represent the current user's emotional state, and combined with weights to calculate a comprehensive emotional score for ad matching.

[0050] The core of advertising content matching is to calculate the matching degree between the current user's emotional state and each advertising content in the advertising content library. The matching algorithm can be optimized using rule-based decision-making methods, machine learning models, or deep neural networks. For example, in the rule-based method, preset rules can be defined, such as "when the user's emotional tendency is positive and the interest is high, entertainment ads are pushed first" or "when the user's concentration is low, choose advertising content with strong visual impact". In the machine learning method, collaborative filtering or content-based recommendation algorithms can be used to compare the user's current emotional characteristics with historical data to select the advertisement that best matches the user's state. In the deep learning method, a pre-trained emotion-advertising matching model can be used to input the user's emotional state vector and output the best matching advertising content.

[0051] Once the advertising content with the highest matching degree is calculated, the system will push the advertising content based on the matching results. At the same time, in order to improve the real-time matching, the advertising selection process can adopt a cache mechanism or index optimization to speed up the search and matching speed. If the advertising content with the highest matching degree is the same as the current advertising content, there is no need to switch, otherwise the system can trigger the update of the advertising content and provide an optimization basis for subsequent steps. In addition, in order to improve the long-term matching accuracy, the system can store user viewing records, matching results and user interaction feedback data, and use this data to continuously optimize the advertising selection strategy to enhance the user experience and the commercial value of advertising.

[0052] Furthermore, selecting the advertisement content with the highest matching degree with the current user status from a preset advertisement content library according to the user experience score includes: Based on the interest, concentration and emotional tendency in the user experience score, the advertising content in the advertising content library is preliminarily screened to eliminate the types of advertisements that do not match the user's current emotional state, and to give priority to retaining the advertising content for which historical viewing data indicates that the user has positive feedback; For the initially screened advertising content, analyze the user's eye focus data to determine the characteristics of the advertising elements that the user is more likely to focus on during the current advertising viewing process, including image areas, text information or dynamic content, and calculate the matching degree of candidate advertisements in the advertising content library based on the distribution of the user's interest points to improve the accuracy of advertising matching; When the matching degrees of multiple advertising contents are close, the recommendation ranking of the advertising contents is adjusted based on the user's historical viewing behavior, ad click records and preference data of similar users.

[0053] When selecting the advertising content that best matches the current user status based on the user experience score, the system first needs to screen the ads in the advertising content library to ensure that the recommended advertising content is consistent with the user's current emotional state, interest level, and concentration level. The interest, concentration, and emotional tendency in the user experience score jointly determine the screening criteria. Among them, interest is used to judge the user's willingness to accept advertising content, concentration is used to measure the user's current tolerance for information acquisition, and emotional tendency reflects the user's potential preference for different types of advertisements. Based on these factors, the system screens the advertising content library and eliminates those types of advertisements that do not match the user's current emotional state. For example, when the user's emotional tendency tends to be negative, the system will reduce the push of high-stimulation or high-information-density advertisements, while when the user's interest is high, it will give priority to detailed and information-rich advertisements. In addition, the system also refers to the user's historical viewing records and gives priority to retaining data for those users who have watched for a long time, have more interactive behaviors, or have positive feedback in the past, in order to improve the accuracy of advertising matching.

[0054] After completing the initial screening, the system further analyzes the user's eye focus data to determine which advertising elements the user pays more attention to during the viewing process, including image areas, text information or dynamic content. The system determines the user's interest in different advertising elements through the user's gaze duration, eye movement trajectory and gaze stop point, and calculates the match between the candidate ads and the user's current interest points. For example, if the user's gaze tends to stay in the image area, the system will give priority to recommending ads with stronger visual impact and a higher image ratio; if the user's gaze time on text information is longer, the system will select advertising content that expresses the information more intuitively and in detail; for users who pay attention to dynamic content, animation or short video ads are recommended to increase the attractiveness of the advertising content. In this way, the system can ensure that the selected advertising content is highly matched with the user's interest points, thereby improving the viewing experience and communication effect of the advertisement.

[0055] When the matching degree of candidate advertisement content is close, the system optimizes the ranking of advertisement recommendations by combining the user's historical viewing behavior, ad click records, and preference data of users of the same type. Historical viewing behavior reflects the user's long-term interest trends and information acquisition habits. For example, if the user has shown a high interest in a certain category of advertisements in multiple past viewings, the priority of advertisements in this category will be increased accordingly. Ad click records are used to determine the actual interaction of users with advertisement content. For example, if the click-through rate of a certain type of advertisement is high, the recommendation weight of this type of advertisement will be increased. The preference data of users of the same type can be used to infer the user's potential points of interest. If the system detects that users with similar behavior patterns to the current user have a high preference for a certain type of advertisement, the recommendation priority of this type of advertisement can be appropriately increased. Through this sorting optimization method based on multi-dimensional data, the system can select the advertisement that best meets the individual needs of users from multiple advertisements with similar matching degrees, thereby further improving the accuracy of advertisement matching and delivery effect.

[0056] Step S104: dynamically adjusting the visual layout and key information element positions of the selected advertising content based on the sight focus data identified by the sentiment analysis model.

[0057] In step S104, the system dynamically adjusts the visual layout and key information element positions of the selected advertising content based on the eye focus data identified by the sentiment analysis model to optimize the user's viewing experience and improve the effectiveness of the advertisement. First, the user's eye focus data needs to be parsed, which includes but is not limited to the user's gaze area, eye focus dwell time, eye movement trajectory change trend, and position mapping relative to the screen content. The system uses this data to determine the user's focus area in the advertising content, that is, the area where the user's gaze is concentrated and stays for a long time is usually a high-attention area, while the area where the gaze moves quickly or stays for a short time may indicate low attention or information redundancy.

[0058] After determining the area of ​​user attention, the system optimizes and adjusts the layout of the advertising content. Specifically, the system first analyzes the key information elements of the current advertisement, such as brand logo, product name, core selling point, price information, promotional content, and image or video body, and combines the user's line of sight focus data to rearrange the positions of these elements so that key information can more easily enter the user's visual center. For example, if the user's line of sight is mainly concentrated on the left area of ​​the advertising screen, and the core selling point of the advertisement is on the right, the core selling point information can be moved to the left to ensure that the user can quickly receive key information. Similarly, if the user's line of sight stays for a short time or is more scattered, the system can focus the high-value information of the advertisement on the area where the line of sight stays for a longer time, thereby improving the effective communication rate of the advertisement.

[0059] In terms of specific adjustment strategies, an adaptive layout optimization method based on regional division can be used to divide the advertising content into multiple dynamic adjustment areas according to the grid structure of the advertising screen. The system can adjust the position, size or transparency of information blocks in these areas based on line of sight data to enhance visual appeal. In addition, the system can combine an adaptive optimization algorithm based on reinforcement learning to learn the line of sight distribution patterns of different user groups, so as to perform pre-optimization in subsequent advertising displays, reduce adjustment delays, and improve the coherence of advertising displays.

[0060] In the process of adjusting visual elements, the system can also dynamically change the arrangement of the text and images of the advertisement. For example, for advertisements with long text descriptions, if it is detected that the user's concentration is low, the amount of text can be reduced or the font size of keywords can be increased to make the key information more prominent. If the advertisement contains multiple products or services, the system can prioritize the most relevant content in the user's high-attention area based on the user's interest. In addition, if the advertisement is in video form, the system can optimize the display of key frames based on the user's line of sight, so that important information appears at the time when the user's attention is highest, in order to improve the effective communication rate of advertising information.

[0061] When making layout adjustments, visual balance and aesthetics also need to be considered to avoid abrupt changes in advertising elements that affect user experience. The system can use a computer vision-based image aesthetics evaluation model to ensure that the adjusted advertising layout complies with visual design principles, such as symmetry, contrast, and clarity of information hierarchy. For interactive ads, the system can also further optimize the layout based on user gestures or click behaviors. For example, when a user shows high attention to a certain area, the relevant content in that area can be enlarged or buttons for further interaction can be provided to enhance user engagement.

[0062] The entire layout adjustment process can be performed in real time during the ad display process, or it can be optimized in advance based on historical user data to reduce computing overhead and the impact of content mutations on user experience. Ultimately, by dynamically adjusting the visual layout of the ad and the position of key information elements, the ad content can more accurately adapt to the user's attention distribution, improving the ad's communication effect and user experience satisfaction.

[0063] Furthermore, the dynamically adjusting the visual layout and key information element positions of the selected advertising content based on the sight focus data identified by the computer vision sentiment analysis model includes: Based on the user's sight focus data, determine the user's main attention area and low attention area in the advertising content, wherein when the user's sight is focused on a specific area for a long time, the specific area is determined as a high attention area, and the part where the user's sight is less focused is identified as a low attention area; Optimize the display of key information elements in high-attention areas. If the ad content contains multiple information levels, prioritize the core brand logo, price information or main ad copy to the high-attention area to improve information delivery efficiency. Weaken or reduce non-core content in low-attention areas to reduce visual interference. During the advertising playback process, dynamic advertising elements are adaptively adjusted in combination with the real-time changes in the user's line of sight. When it is detected that the user's line of sight stays in the key information area for a short time, the visual prominence of the area is appropriately enhanced, including enlarging the font, increasing the contrast, or adding animation effects to attract the user's attention. When the user pays attention to the specified content for a long time, excessive dynamic changes are avoided.

[0064] When adjusting the visual layout of the advertising content and the position of key information elements based on the sight focus data identified by the computer vision sentiment analysis model, the system first needs to determine the user's main focus area and low focus area in the advertising screen. The camera device collects the user's sight data when watching the advertisement in real time, and conducts a comprehensive analysis based on the sight dwell time, eye movement trajectory and focus stability to determine which advertising areas the user stays for a long time and which areas the eyes only briefly scan. Areas where the gaze is focused for a long time are judged as high-attention areas, while areas where the gaze stays less or jumps frequently are identified as low-attention areas. This area division can be dynamically adjusted according to the type of advertisement and the user's visual behavior pattern to ensure that the system can accurately identify the user's actual focus.

[0065] After completing the analysis of the user's attention area, the system optimizes the display of key information elements according to the high-attention area to ensure that the most important information appears in the area where the user's vision is prioritized. If the advertising content contains multiple levels of information structure, such as brand logo, product introduction, price information, discount information or advertising copy, the system will prioritize adjusting the core brand logo, price information or content with strong marketing intentions to the user's high-attention area to improve the efficiency of information transmission and enable users to obtain key information faster. On the contrary, for low-attention areas, the system will appropriately weaken the visual weight of non-core content, such as reducing font size, lowering contrast or reducing dynamic elements to reduce visual interference, optimize the hierarchical relationship of advertising information, make the overall advertising structure more in line with the user's natural line of sight flow path, and improve information absorption efficiency.

[0066] During the ad playback process, the system makes adaptive adjustments to dynamic ad elements based on the real-time changes in the user's line of sight to ensure that the ad content can continue to attract the user's attention while avoiding information overload or interference. When it is detected that the user's line of sight stays in the key information area for a short time, the system will appropriately enhance the visual prominence of the area, such as enlarging the font, increasing the color contrast, or adding slight animation effects to strengthen the user's attention to the information. If the user stays in a key information area for a long time, the system will reduce the dynamic changes in the area to avoid user fatigue or distraction due to excessive visual stimulation. This adaptive adjustment strategy can be optimized according to the user's viewing pattern, so that the ad content strikes a balance between attracting attention and information stability, thereby improving the effectiveness of the ad and the user experience.

[0067] Step S105: dynamically adjusting the presentation mode of the advertisement content, including the content type, image-text ratio, and information density, according to the real-time changes in the user experience score.

[0068] In step S105, the system dynamically adjusts the presentation of the advertising content according to the real-time changes in the user experience score to ensure that the advertising content is highly matched with the user's emotional state and attention level, thereby improving the advertising communication effect and the user's viewing experience. This adjustment process involves multiple aspects, including the optimization of content type, image-text ratio, and information density to adapt to the user's current mood and concentration.

[0069] First, in terms of adjusting the content type, the system will select the most suitable advertising presentation mode for the user's current state according to the changes in the user experience score. When the user's interest score is high and the concentration score is at a high level, the system can choose to push more detailed advertising content, such as video ads or long text ads with in-depth product introductions, to maximize the delivery effect of advertising information by taking advantage of the user's high attention. When the system detects that the user's interest is low or the concentration decreases, it can adopt more refined advertising forms, such as short video ads, dynamic picture ads, or short text content, to reduce the user's visual fatigue caused by excessive information and improve the acceptance of information. In addition, if the user's emotional tendency tends to be negative, the system can adjust the tone of the advertising content and choose more relaxed, soft-colored or positive emotionally guided advertising materials to reduce the user's resistance and improve the affinity of the advertisement.

[0070] In terms of adjusting the image-text ratio, the system will dynamically optimize the ratio of images to text in the advertisement based on the user's visual focus data and content preferences. If the user pays more attention to the image part of the advertisement and spends less time reading the text content, the system can increase the proportion of images and reduce the density of text information to make the advertisement content more intuitive and easy to understand, in line with the user's viewing habits. Conversely, if the user's eyes stay in the text area for a long time, the system can increase the level of detail of the text information and provide more product introductions or promotional details to fully meet the user's reading needs. The adjustment of the image-text ratio is not limited to static advertisements, but can also be applied to video advertisements, such as adjusting the size of subtitles, display position, or highlighting key information to suit the preferences of different users. In addition, the system can also combine the user's historical behavior data to determine their preference trends for images and texts, and optimize the visual layout of the advertisement content accordingly to improve the efficiency of information communication.

[0071] In terms of adjusting the information density, the system will adjust the complexity of the content of the advertisement according to the user's emotional state and concentration changes. When the user is in a state of high concentration and shows a sustained interest in the content of the advertisement, the system can increase the level of detail of the information, such as adding more product specifications, use cases or user reviews to the advertisement page to meet the user's need for in-depth understanding. When the user's concentration decreases or the mood fluctuates greatly, the system will simplify the advertisement content, reduce the interference of irrelevant information, highlight the core points of the advertisement, and make the information more concise and easy to read. For example, the system can automatically shorten the advertisement playback time, reduce the number of text lines on the page, or use keyword highlighting to enable users to obtain the core information of the advertisement in the shortest time. In addition, the system can also combine the user's eye movement data to identify the information density range that the user is most likely to accept, and adaptively optimize the advertisement according to this range to ensure the efficiency of information communication.

[0072] During the entire dynamic adjustment process, the system not only relies on real-time user experience scores, but also combines historical user data, industry standards, and advertising effectiveness evaluation models to continuously optimize advertising adjustment strategies. The system can use a content recommendation algorithm based on machine learning to train the optimal advertising presentation mode through historical data, and continuously fine-tune it in real-time delivery to improve the personalized adaptation of advertising. In addition, in order to ensure the stability of user experience, the system can adopt a gradual adjustment method rather than a sudden adjustment to reduce visual discomfort caused by rapid changes in advertising content, thereby ensuring a natural transition of advertising information. Ultimately, through dynamic optimization of content type, image-text ratio, and information density, the matching degree of advertising content can be effectively improved, and user participation and advertising communication effects can be enhanced.

[0073] Furthermore, dynamically adjusting the presentation mode of the advertisement content according to the real-time change of the user experience score includes: Based on the interest, concentration and emotional tendency in the user experience score, the user's current ability to accept information and preference characteristics are judged, and the advertising content adjustment strategy is set according to different score ranges. When the interest and concentration are high, the advertising content with rich information is given priority, and when the interest or concentration is low, the non-core information is reduced to avoid the user's resistance to the advertisement; Optimize the image-text ratio for the adjusted ad content. When the user's concentration is high and interest is low, increase the proportion of image elements in the ad to enhance the visual impact. When the user's interest is high, increase the clarity and content of the text information to meet the user's in-depth reading needs. During the advertising process, the changing trend of the user experience score is continuously monitored, and the information density is dynamically adjusted. When the user's emotional tendency is positive and the concentration is stable, the details of the advertising content are increased. When the user's concentration decreases or the emotion tends to be negative, the information density is reduced to improve the acceptability of the advertisement and the efficiency of information transmission.

[0074] When dynamically adjusting the presentation of advertising content based on real-time changes in user experience scores, the system first comprehensively judges the user's current ability to accept information and preference characteristics for advertising content based on the user's interest, concentration, and emotional tendencies. The user experience score is used to measure the user's acceptance of the current advertisement. The interest reflects the user's attraction to the advertising content, the concentration indicates the user's concentration, and the emotional tendency determines the user's psychological state when watching the advertisement. Based on different combinations of these scores, the system formulates an adjustment strategy for advertising content. For example, when the user's interest and concentration are both at a high level, the system gives priority to advertising content with rich information and complete structure to meet the user's need for in-depth understanding. When the user's interest or concentration is low, the system reduces non-core information in the advertisement to simplify the content presentation and avoid user resistance or skipping of advertisements due to information overload.

[0075] After completing the initial adjustment of the ad content, the system further optimizes the image-text ratio to enhance the visual appeal of the ad and improve the effectiveness of information delivery. The user's concentration and interest determine the relative proportion of images and text in the ad content. When the user's concentration is high but the interest is low, the system increases the proportion of image elements in the ad and reduces the long text content to attract the user's attention with visual impact while ensuring that the information is expressed intuitively and concisely. When the user's interest is high, the system enhances the clarity of the text information in the ad, improves the contrast and readability of the text, and provides richer product introductions or additional information to meet the user's in-depth reading needs. This adjustment method ensures that the expression of the ad content is in line with the user's actual attention pattern and improves the effectiveness of information delivery.

[0076] During the ad playback process, the system continuously monitors the changing trend of the user experience score and dynamically adjusts the content information density of the ad accordingly. When the user's emotional tendency is positive and the concentration remains stable, the system gradually increases the details of the ad content, such as adding more product descriptions, discount information or interactive elements to the screen to enhance the user's sense of participation and improve the conversion rate of the ad. When the user's concentration decreases or the emotion tends to be negative, the system reduces the information density, reduces excessive text content and visual elements, to improve the acceptability of the ad and reduce the user's cognitive burden. This dynamic information density adjustment mechanism enables the ad to adapt to the user's emotional state and attention level, thereby optimizing the user experience and improving the communication effect and interaction rate of the ad.

[0077] Furthermore, the method for intelligently delivering dynamic advertising content based on user emotion recognition further includes: Setting a visual heat detection area around the interface displaying the advertisement content, wherein the visual heat detection area does not display the actual advertisement content but can capture the user's marginal visual attention; Predicting the user's visual fatigue level and attention distraction tendency based on the user's gaze behavior in the visual heat detection area; When it is detected that the user's visual fatigue exceeds a preset threshold, the visual complexity and refresh frequency of the advertising content are adjusted to alleviate the user's visual fatigue and optimize the advertising experience.

[0078] In this embodiment, by setting a visual heat detection area around the interface displaying the advertising content, the system can detect the user's marginal visual attention, thereby providing an in-depth analysis of the user's overall visual state when watching the advertisement. The visual heat detection area refers to a certain range around the advertising interface. This area does not directly display the advertising content, but is used to track the user's eye movement trajectory, line of sight deviation pattern and gaze within the marginal visual field. The detection area can use invisible sensing technology to enable the user to passively provide visual data without knowing it, ensuring the naturalness and reliability of the detection data. Through a high-precision eye tracking algorithm, the system can capture information such as the user's gaze dwell time, line of sight movement frequency, saccade behavior and line of sight deviation direction in the area.

[0079] Based on the collected user's gaze behavior in the visual heat detection area, the system predicts the user's visual fatigue level and attention distraction tendency. Visual fatigue can be judged by a variety of parameters. For example, the user's frequent gaze in the visual heat detection area may indicate that the user's interest in the main advertising content has decreased, while a longer fixed gaze may indicate that the user's visual fatigue level is high. The system can also combine the blinking frequency, pupil dilation, and nystagmus characteristics to comprehensively evaluate the user's visual comfort and determine whether there is fatigue caused by long-term high-stimulation visual input. In addition, the attention distraction tendency can be analyzed by the frequency of the user's eyes leaving the advertising content, the increase in random eye movement trajectories, and the decrease in the duration of gaze on the main advertising elements. By building a real-time visual state model, the system can predict that the user is about to enter the stage of attention decline or visual fatigue, and dynamically adjust the advertising presentation strategy accordingly to maintain the user's viewing experience.

[0080] When the system detects that the user's visual fatigue exceeds the preset threshold, it will automatically adjust the visual complexity and refresh rate of the advertising content to relieve the user's visual fatigue and optimize the viewing experience of the advertisement. The adjustment of visual complexity can be done by reducing the image detail density of the advertisement, reducing color saturation, reducing contrast, or simplifying visual elements to make the advertising interface softer and reduce the visual burden on the user. In addition, for dynamic advertisements, the system can reduce the speed of animation switching and reduce the visual stimulation of frequent flashing, thereby alleviating visual fatigue caused by rapid changes. The adjustment of refresh frequency is mainly aimed at the update rhythm of advertisements. When the user's visual fatigue increases, the system can reduce the frequency of advertising content switching, extend the display time of the advertisement, or use a gradual transition method to update the content to reduce abrupt visual changes and avoid interference to users.

[0081] In order to further optimize the user experience, the system can also combine personalized visual fatigue threshold adjustment strategies to dynamically adjust the control parameters of visual load according to the user's viewing history and preferences. For example, for users who watch advertisements for a long time, the system can set a lower visual fatigue trigger threshold to enter the low stimulation mode earlier, and for users who watch advertisements with high concentration for a short time, the system can appropriately increase the visual complexity to ensure the effective communication of the advertising content. Ultimately, this method can achieve accurate prediction of user visual fatigue and attention distraction through intelligent monitoring of the visual heat detection area, and improve the long-term acceptability of advertisements and the quality of user experience by intelligently regulating the visual expression of advertising content.

[0082] Furthermore, the method for intelligently delivering dynamic advertising content based on user emotion recognition further includes: Real-time analysis of the correlation between the user's facial micro-expressions and the rate of eye movement, identifying the user's subconscious interest and resistance points to the advertising content; Map the recognition results with the semantic partitions of the advertising content to generate a user interest heat map; Based on the user interest heat map, the advertisement content is reconstructed in real time, the semantic elements that the user is concerned about are strengthened, and the content elements that cause resistance are weakened or replaced, thereby realizing the imperceptible personalized optimization of the advertisement content.

[0083] In this embodiment, the correlation between the user's facial micro-expressions and the rate of sight transfer is analyzed in real time to identify the user's subconscious interest points and resistance points to the advertising content. Facial micro-expressions are short-term, unconscious changes in expression, which can usually reflect the user's true emotional state. For example, a short rise in the corners of the mouth may represent slight interest or pleasure, while a rapid drooping of the eyebrows may indicate dissatisfaction or boredom. In order to capture these micro-expression changes, the system uses a high-frame rate camera and a deep learning expression recognition model to extract feature data of expression changes by analyzing the user's facial muscle movement pattern. At the same time, combined with the user's sight transfer rate, that is, the speed and dwell time of the user's sight between different areas of the advertising content, the system can further analyze the user's active focus points and possible resistance areas on the advertising content. When the user's sight stays on a certain advertising element for a long time, accompanied by a slight positive micro-expression change, it can be judged that the area is a potential point of interest for the user. If the user frequently and quickly moves his sight in a certain area, or is accompanied by negative micro-expression changes, it may mean that the user has resistance to the content.

[0084] After obtaining the user's points of interest and resistance, the system needs to map these recognition results with the semantic partitions of the advertising content to generate a user interest heat map. Advertising content usually contains multiple information elements, such as brand logos, product images, text descriptions, price information, discount tags, etc., and these elements occupy different areas in the advertising layout. In order to achieve semantic partition mapping, the system will structure the advertising content, divide the different elements of the advertisement into regions, and classify the content semantics of each region based on natural language processing (NLP) or computer vision technology, such as classifying product descriptions as informational content, brand logos as brand recognition elements, and price discounts as promotional stimulus elements. Then, the system matches the user's points of interest and resistance with different semantic regions of the advertising content to generate a user interest heat map, which can intuitively show the distribution of users' attention to each part of the advertisement. The interest heat map can be presented in a weighted color coding manner. The area where the user's gaze time is long and the emotion is positive is marked as a high interest area, while the area where the user quickly jumps and is accompanied by negative emotions is marked as a low interest or resistance area.

[0085] After generating the user interest heat map, the system reconstructs the advertising content in real time based on the data to optimize the advertising display effect. The reconstruction methods of advertising content include strengthening the semantic elements that users pay attention to, weakening the content elements that cause resistance, and even replacing the content of the resistance area in some cases to achieve imperceptible personalized optimization. The semantic elements that strengthen the user's attention can be adjusted dynamically, such as enlarging the font size of the user's high-attention area, improving its contrast or adding animation effects to enhance the information transmission efficiency of the area. If the user has a strong interest in a product image, the system can perform high-definition enhancement on the image area, or extend its display time, so that the user can better understand the information. For content elements that cause user resistance, the system can reduce their visual weight, such as reducing color saturation, brightness, dynamic effects, or using fade-out processing to reduce interference to users without affecting the overall advertising layout. In extreme cases, if a content element is determined to be a high-resistance area, the system can dynamically replace it with alternative materials in the advertising content library. For example, if a user expresses discomfort with a promotional label with overly bright colors, the system can replace it with a version with softer tones to increase user acceptance.

[0086] The entire optimization process is carried out without the user's knowledge, so it will not affect the overall coherence of the advertising content and the viewing experience. Because this method combines the user's real emotional feedback with the intelligent adjustment mechanism of the advertising content, the advertisement can be dynamically adapted according to the real-time emotional state of the individual user, thereby improving the user's advertising acceptance, reducing advertising interference, and ultimately improving the conversion effect of the advertisement. This non-perceptual personalized optimization method is more intelligent and efficient than the traditional static advertising delivery method, enabling the advertising content to adapt to the user's viewing preferences and emotional needs to the greatest extent while ensuring the complete delivery of brand information.

[0087] Furthermore, the method for intelligently delivering dynamic advertising content based on user emotion recognition further includes: Monitor users’ emotional fluctuations and concentration changes during advertising; When the emotion fluctuation data shows that the user's emotion tends to be negative or the concentration change data shows that the decrease in the user's concentration exceeds a preset threshold, the buffer adjustment mode of the advertising content is automatically triggered; In the buffer adjustment mode, based on the emotion guidance strategy, the color parameters, content rhythm speed and background music characteristics of the advertisement are adjusted to switch the advertisement content from a high-stimulation information flow mode to a low-stimulation soothing mode; Continuously monitor the user's emotional state, and when it is detected that the user's emotional indicators return to the preset normal range, gradually restore the dynamic delivery strategy to the original advertising content.

[0088] In this embodiment, by monitoring the user's emotional fluctuations and concentration change data during the advertising process, intelligent adjustment of advertising content is achieved, so that advertising can adapt to the user's real-time psychological state and optimize the viewing experience. The user's emotional fluctuation data is calculated based on facial expressions, eye movement trajectories, gaze dwell time, and other physiological indicators (such as blinking frequency, facial muscle tension). The system uses a high-frame rate camera and a deep learning model to analyze the user's facial emotional changes in real time, and extracts features that reflect emotional tendencies, such as drooping corners of the mouth, frowning, or distracted eyes. These features may indicate that the user may have boredom, anxiety, or negative emotions. The concentration change data is calculated based on the user's eye movement stability, the frequency of sight deviation, and the concentration of the sight dwell area. When the user's sight frequently moves away from the advertising area, the dwell time is shortened, or the eye movement pattern becomes irregular, it indicates that his attention is declining, and he may have become impatient or lost interest in the current advertising content.

[0089] When it is detected that the user's emotions tend to be negative or the degree of concentration decreases by more than the preset threshold, the system will automatically trigger the buffering adjustment mode of the advertising content to reduce the visual or emotional burden of the advertisement on the user, and prevent the user from skipping the advertisement or having negative cognition due to discomfort. The core of this buffering adjustment mode is to dynamically adjust the visual and auditory elements of the advertisement based on the emotional guidance strategy, so that the advertisement switches from a high-stimulation fast-paced mode to a softer, low-stimulation soothing mode. Specifically, the adjustment of the color parameters can make the picture softer by reducing the color saturation of the advertisement screen, reducing the visual impact of high-contrast areas, and adding warm-toned elements to reduce visual fatigue and reduce the user's emotional fluctuations. The adjustment of the content rhythm speed may include reducing the animation switching rate in the advertisement, reducing the fast-flickering visual effects, and extending the information display time, so that users can receive the advertising content more comfortably. The adjustment of the background music characteristics includes reducing the volume, reducing high-frequency sound effects, and using more soothing or low-paced audio to reduce auditory stimulation, thereby creating a more peaceful advertising atmosphere so that users will not resist due to excessive information stimulation.

[0090] In the buffering adjustment mode, the system will continuously monitor the user's emotional state to determine whether it has returned to the normal range. When it is detected that the user's emotions tend to stabilize, the expression changes return to neutral or positive, the eye movement trajectory becomes more concentrated, and the concentration returns to a high level, the system will gradually return to the dynamic delivery strategy of the original advertising content. The recovery process adopts a gradual adjustment method to avoid sudden visual or auditory changes affecting the user experience. For example, the color parameters can be slowly adjusted back to the original contrast, the animation rhythm can be gradually accelerated, and the volume and rhythm of the background music can also be adjusted according to a specific gradient curve to ensure that the user's emotions adapt to the changes in the advertising rhythm. In order to improve the overall user experience, the system can also automatically adjust the trigger threshold of the buffering adjustment mode based on the user's viewing history data and personalized emotional preferences to adapt to the emotional response patterns of different users, making advertising delivery more personalized and intelligent. Through this method, the advertising system can intelligently adjust the presentation of advertising content when the user's emotions fluctuate greatly or the attention is reduced, thereby improving the user's advertising acceptance, reducing interference, and improving the overall effectiveness of advertising delivery.

[0091] A second embodiment of the present application provides an electronic device, the electronic device comprising: processor; The memory is used to store a program. When the program is read and executed by the processor, it executes a method for intelligent delivery of dynamic advertising content based on user emotion recognition provided in the first embodiment of the present application.

[0092] The third embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, a method for intelligent delivery of dynamic advertising content based on user emotion recognition provided in the first embodiment of the present application is executed.

[0093] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A method for intelligent delivery of dynamic advertising content based on user emotion recognition, characterized in that: include: The facial image data of the user when watching the advertisement is collected in real time through a camera device, including facial expressions, eye movement trajectories, gaze lingering positions and lingering time; Inputting the facial image data into a computer vision sentiment analysis model to generate gaze focus data and user experience scores including interest, concentration, and emotional tendency; According to the user experience score, selecting the advertising content that best matches the current user status from a preset advertising content library; Dynamically adjust the visual layout and key information element positions of the selected advertising content based on the sight focus data identified by the computer vision sentiment analysis model; Based on the real-time changes in user experience scores, dynamically adjust the presentation of advertising content, including content type, image-text ratio, and information density.

2. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 1, characterized in that: Also includes: Setting a visual heat detection area around the interface displaying the advertisement content, wherein the visual heat detection area does not display the actual advertisement content but can capture the user's marginal visual attention; Predicting the user's visual fatigue level and attention distraction tendency based on the user's gaze behavior in the visual heat detection area; When it is detected that the user's visual fatigue exceeds a preset threshold, the visual complexity and refresh frequency of the advertising content are adjusted to alleviate the user's visual fatigue and optimize the advertising experience.

3. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 1, characterized in that: Also includes: Real-time analysis of the correlation between the user's facial micro-expressions and the rate of eye movement, identifying the user's subconscious interest and resistance points to the advertising content; Map the recognition results with the semantic partitions of the advertising content to generate a user interest heat map; Based on the user interest heat map, the advertisement content is reconstructed in real time, the semantic elements that the user is concerned about are strengthened, and the content elements that cause resistance are weakened or replaced, thereby realizing the imperceptible personalized optimization of the advertisement content.

4. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 1, characterized in that: Also includes: Monitor users’ emotional fluctuations and concentration changes during advertising; When the emotion fluctuation data shows that the user's emotion tends to be negative or the concentration change data shows that the decrease in the user's concentration exceeds a preset threshold, the buffer adjustment mode of the advertising content is automatically triggered; In the buffer adjustment mode, based on the emotion guidance strategy, the color parameters, content rhythm speed and background music characteristics of the advertisement are adjusted to switch the advertisement content from a high-stimulation information flow mode to a low-stimulation soothing mode; Continuously monitor the user's emotional state, and when it is detected that the user's emotional indicators return to the preset normal range, gradually restore the dynamic delivery strategy to the original advertising content.

5. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 1, characterized in that: The computer vision sentiment analysis model includes a visual feature extraction unit, a sentiment state reasoning unit and a user experience score calculation unit; The visual feature extraction unit is used to extract multi-dimensional features from the input facial image data, including facial expression features, eye movement trajectory data and sight stop information; the visual feature extraction unit detects facial key points through a convolutional neural network to obtain the user's eyebrows, eyes, and mouth corners. In combination with the optical flow method, the eye movement trajectory and sight movement direction of the user are calculated; in combination with the viewing angle parameters of the camera device, the user's sight point is spatially mapped to generate accurate sight focus data, and the user's gaze area in the advertising picture is calibrated; The emotional state inference unit is used to synthesize the output data of the visual feature extraction unit to infer the user's real-time emotional state, including interest, concentration and emotional tendency; the emotional state inference unit adopts a time series modeling method based on a long short-term memory network to identify the dynamic changes of the user's emotional state, and combines the self-attention mechanism to optimize the weight allocation of different visual features to improve the accuracy of emotional analysis; by jointly analyzing the change pattern of facial expressions, the stability of eye movement trajectories, the duration and area of ​​gaze retention, the user's interest in the current advertising content is calculated, the user's concentration level is evaluated, and the user's overall emotional tendency is identified; The user experience score calculation unit is used to calculate the final user experience score based on the output result of the emotional state reasoning unit, and optimize the adaptability of the advertising content in combination with the line of sight focus data; the user experience score calculation unit adopts a weighted multi-factor scoring model to comprehensively consider the interest, concentration, emotional tendency and the stability of the user's line of sight focus to generate a user experience score, and makes personalized adjustments in combination with historical viewing behavior and individual preferences.

6. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 5, characterized in that: The user experience score calculation unit uses the following formula 1 to calculate the user experience score: ； in, Score the end-user experience; Respectively represent the user in The interest, concentration and emotional tendency of each time step ranges from ; For users in The degree of deviation of interest, concentration and emotional tendency at a time step from the average level in the past specified time period; For users in The deviation of interest, concentration and emotional tendency at a time step from the average level in the past specified time period; is the time window size used to calculate the user's historical sentiment trend; is a tiny constant that prevents the denominator from going to zero; is the sentiment weight, which is used to dynamically adjust the influence of each time step in the overall score calculation, and is calculated using the following formula 2: ； in, They are the average interest, concentration and emotional tendency of the user in watching ads in the past; is the personalized adjustment coefficient; is the time window size used to calculate the user's historical sentiment trend.

7. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 1, characterized in that: The real-time collection of facial image data of the user when watching the advertisement by means of a camera device includes: After detecting that the user's face enters the advertisement viewing area, the exposure parameters of the camera device are dynamically adjusted based on active light adaptation imaging technology to adapt to different ambient lighting conditions; Combined with an adaptive frame rate control algorithm, the sampling frame rate is increased when the user's eye movement frequency is high to accurately capture the instantaneous attention shift, and the sampling frequency is reduced when the user's gaze remains stable to reduce data redundancy and optimize computing efficiency.

8. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 1, characterized in that: The selecting, according to the user experience score, the advertisement content with the highest matching degree with the current user status from a preset advertisement content library includes: Based on the interest, concentration and emotional tendency in the user experience score, the advertising content in the advertising content library is preliminarily screened to eliminate the types of advertisements that do not match the user's current emotional state, and to give priority to retaining the advertising content for which historical viewing data indicates that the user has positive feedback; For the initially screened advertising content, analyze the user's eye focus data to determine the characteristics of the advertising elements that the user is more likely to focus on during the current advertising viewing process, including image areas, text information or dynamic content, and calculate the matching degree of candidate advertisements in the advertising content library based on the distribution of the user's interest points to improve the accuracy of advertising matching; When the matching degrees of multiple advertising contents are close, the recommendation ranking of the advertising contents is adjusted based on the user's historical viewing behavior, ad click records and preference data of similar users.

9. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 1, characterized in that: The method of dynamically adjusting the visual layout and key information element positions of the selected advertising content based on the sight focus data identified by the computer vision sentiment analysis model includes: Based on the user's sight focus data, determine the user's main attention area and low attention area in the advertising content, wherein when the user's sight is focused on a specific area for a long time, the specific area is determined as a high attention area, and the part where the user's sight is less focused is identified as a low attention area; Optimize the display of key information elements in high-attention areas. If the ad content contains multiple information levels, prioritize the core brand logo, price information or main ad copy to the high-attention area to improve information delivery efficiency. Weaken or reduce non-core content in low-attention areas to reduce visual interference. During the advertising playback process, dynamic advertising elements are adaptively adjusted in combination with the real-time changes in the user's line of sight. When it is detected that the user's line of sight stays in the key information area for a short time, the visual prominence of the area is appropriately enhanced, including enlarging the font, increasing the contrast, or adding animation effects to attract the user's attention. When the user pays attention to the specified content for a long time, excessive dynamic changes are avoided.

10. The method for intelligent delivery of dynamic advertising content based on user emotion recognition according to claim 1, characterized in that: The method of dynamically adjusting the presentation mode of the advertisement content according to the real-time changes of the user experience score includes: Based on the interest, concentration and emotional tendency in the user experience score, the user's current ability to accept information and preference characteristics are judged, and the advertising content adjustment strategy is set according to different score ranges. When the interest and concentration are high, the advertising content with rich information is given priority, and when the interest or concentration is low, the non-core information is reduced to avoid the user's resistance to the advertisement; Optimize the image-text ratio for the adjusted ad content. When the user's concentration is high and interest is low, increase the proportion of image elements in the ad to enhance the visual impact. When the user's interest is high, increase the clarity and content of the text information to meet the user's in-depth reading needs. During the advertising process, the changing trend of the user experience score is continuously monitored, and the information density is dynamically adjusted. When the user's emotional tendency is positive and the concentration is stable, the details of the advertising content are increased. When the user's concentration decreases or the emotion tends to be negative, the information density is reduced to improve the acceptability of the advertisement and the efficiency of information transmission.

Images