Short video content accurate recommendation system based on artificial intelligence image recognition

Abstract

The invention discloses a short video content accurate recommendation system based on artificial intelligence image recognition, particularly relates to the technical field of short video personalized recommendation, and is used for solving the problem of matching of user interest and visual bearing capacity. The method comprises the following steps: extracting short video key frame spatial-temporal characteristics through a multi-scale convolutional neural network, constructing a cognitive load threshold curve in combination with a user micro gesture sequence, representing the tolerance range of a user to visual complexity in different time periods, and generating a visual complexity vector; calculating a visual load matching index by using an adaptive dynamic warping algorithm, and adjusting a candidate video sequence through a load penalty factor to generate an initial recommendation probability; modeling a user dynamic interest vector based on a gated loop unit network in combination with an attention mechanism; and finally, user interests and video semantics are fused through multi-dimensional features, a final recommendation list is generated through a multi-objective optimization algorithm under the constraint of visual load, and personalized recommendation of interest matching degree maximization and visual comfort optimization is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of short video personalized recommendation, in particular to a short video content accurate recommendation system based on artificial intelligence image recognition. BACKGROUND

[0002] With the popularity of intelligent terminals and high-speed mobile networks, short videos have gradually become an important channel for people to obtain information and entertainment in their daily lives. The vast number of short videos has dramatically expanded the selection range for users, and recommendation systems play a key role in this regard. Existing recommendation platforms generally rely on big data mining and deep learning models to analyze users' historical viewing records, likes, and forwarding behaviors, thereby achieving personalized recommendations and improving user stickiness and content exposure.

[0003] However, existing recommendation methods still have obvious defects in practical application. On the one hand, they usually rely on a single click or viewing time indicator, which is difficult to truly reflect users' immediate cognitive state and psychological load during the viewing process, and is prone to problems such as overly complex or overly single recommended content, causing user fatigue or aesthetic fatigue. On the other hand, the recommended results often focus on interest similarity, while ignoring the matching relationship between visual complexity and users' instantaneous cognitive capacity, leading to some recommended videos having high interest relevance but causing additional burden on users in visual perception, thereby reducing the effectiveness of recommendations and user experience. In addition, existing recommendation decisions are mainly based on a single optimization target, lacking comprehensive consideration of multi-dimensional constraint conditions, and are prone to conflicts between recommendation accuracy and viewing comfort, affecting the overall performance of the system. SUMMARY

[0004] To overcome the deficiencies of the prior art, the present application provides a short video content accurate recommendation system based on artificial intelligence image recognition, which solves the problems in the background art.

[0005] To achieve the above object, the application is implemented by the following technical solutions: a short video content accurate recommendation system based on artificial intelligence image recognition, comprising the following modules: a visual load perception module, configured to extract the space-time features of short video key frames through a multi-scale convolutional neural network, generate a visual complexity vector, and construct a user cognitive load threshold curve by collecting user interactive micro-gesture feature sequences, for representing the tolerance range of visual complexity of the user in different time periods; a cognitive matching module, configured to match the visual complexity vector of the candidate short video with the cognitive load threshold curve of the current time period of the user through an adaptive dynamic regularization algorithm, obtain a visual load matching index, when the visual load matching index exceeds a preset threshold range, introduce a load penalty factor to adjust the sorting value of the candidate short video, and generate an initial recommendation probability of the candidate short video; a dynamic interest modeling module, configured to model the user interactive micro-gesture feature sequences in time sequence based on a gated recurrent unit network combined with an attention mechanism, filter noise interactions in the historical sequence through a reset gate, strengthen recent relevant interest representations through an update gate, and weight historical behaviors through attention weights, and output a user dynamic interest vector; and a multi-target recommendation decision module, configured to perform multi-dimensional feature fusion analysis on the user dynamic interest vector and the semantic embedding vector of the candidate short video, obtain a semantic matching degree, introduce the visual load matching index as a constraint condition, and perform weighted decision on the initial recommendation probability of the candidate short video and the semantic matching degree through a multi-objective optimization algorithm with constraints, and output a recommendation list.

[0006] Further, the specific process of extracting the space-time features of short video key frames through a multi-scale convolutional neural network to generate a visual complexity vector is as follows: key frame sampling is performed on the input short video, and the space-time features of the short video key frames are extracted, including scene complexity features, motion intensity features, color contrast features, and texture density features; global scene layout features are extracted through a large-scale branch of the multi-scale convolutional neural network, object motion trajectory features are extracted through a medium-scale branch, and picture detail texture features are extracted through a small-scale branch; the features extracted at different scales are fused across layers, a visual complexity score is calculated through a feature weighting aggregation algorithm, and a multi-dimensional vector representing the visual complexity of the video is generated.

[0007] Further, the specific process of collecting user interactive micro-gesture feature sequences to construct a user cognitive load threshold curve is as follows: micro-gesture interaction data of the user during browsing is continuously monitored, including acceleration change features of touch screen sliding trajectories, distribution features of page dwell time, pupil focus trajectory features, and touch screen pressure features; multi-modal interaction features are time series segmented and standardized, time domain and frequency domain features representing cognitive load are extracted, a mapping relationship from micro-gesture features to cognitive load level is established, a threshold curve reflecting the change law of cognitive load of the user in different time periods is constructed through a time series analysis algorithm, and the curve dynamically represents the real-time tolerance range of visual complexity of the user.

[0008] Further, the specific process of obtaining the visual load matching index by matching the visual complexity vector of the candidate short video with the cognitive load threshold curve of the user in the current time period through the adaptive dynamic time warping algorithm is as follows: calculating the feature distance matrix of the visual complexity vector of the candidate short video and the cognitive load threshold curve of the user at each time point; finding the optimal matching path between the two sequences through the dynamic time warping algorithm, wherein the warping window size is adaptively adjusted according to the fluctuation degree of the curve; calculating the matching degree score according to the cumulative distance of the optimal matching path, and normalizing the path length and the curvature to generate the visual load matching index representing the matching degree of the video content and the user's cognitive state.

[0009] Further, when the visual load matching index exceeds the preset threshold range, a load penalty factor is introduced to adjust the ranking value of the candidate short video, and the specific process of generating the initial recommendation probability of the candidate short video is as follows: according to the deviation degree of the visual load matching index and the preset threshold, the load penalty factor value is calculated through a nonlinear function, and the load penalty factor and the original ranking value are weighted and adjusted, wherein the penalty factor weight is dynamically adjusted according to the current cognitive load level of the user; the adjusted ranking value is mapped to the probability space through the Sigmoid function to generate the initial recommendation probability of the candidate short video, which reflects the matching degree of the video content and the user's cognitive bearing capacity.

[0010] Further, the logical process of modeling the time sequence of the user interaction micro-gesture feature sequence based on the gated recurrent unit network combined with the attention mechanism is as follows: inputting the user interaction micro-gesture feature sequence into the gated recurrent unit network to capture the time sequence dependency in the sequence through the progression of time steps; introducing the attention mechanism based on the hidden layer output of the gated recurrent unit network to calculate the attention weight of each time step output; weighting and summing the hidden states of each time step according to the attention weight to generate a latent vector representation representing the user's dynamic interest, and fusing the latent vector representation with the input feature at the current time to output the time sequence modeling result of the user's dynamic interest.

[0011] Further, the specific process of outputting the user dynamic interest vector by filtering the noise interaction in the historical sequence through the reset gate, strengthening the recent relevant interest representation through the update gate, and weighting the historical behavior through the attention weight is as follows: calculating the retention proportion of the historical hidden state through the reset gate to filter the noise interaction information irrelevant to the current interest; controlling the fusion proportion of the current time input information and the historical information through the update gate to strengthen the recent relevant interest representation; calculating the importance weight of each time step in the historical sequence through the attention mechanism, and aggregating the weighted hidden states; concatenating and linearly transforming the feature vectors processed through the gating mechanism and the attention weighting to generate the user dynamic interest vector.

[0012] Further, the specific process of obtaining the semantic matching degree by multi-dimensional feature fusion analysis of the user dynamic interest vector and the candidate short video semantic embedding vector is as follows: the interaction weight between the user dynamic interest vector and the candidate short video semantic embedding vector is calculated through the cross-modal attention mechanism, and the association mapping of user interest and video content is established; the two vectors are fused through the combination of feature splicing and element-level product, the linear and nonlinear relationship between user interest and video content is captured, and the high-order cross features are extracted through the deep nonlinear transformation of the fused feature vector by the multilayer perception; the high-order cross features are mapped to the semantic matching degree score through the fully connected layer, and the score represents the correlation degree of user interest and video content.

[0013] Further, the visual load matching index is introduced as a constraint condition, the initial recommendation probability of the candidate short video and the semantic matching degree are weighted and decided by the multi-objective optimization algorithm with constraints, and the specific process of outputting the recommendation list is as follows: a double-objective optimization function is established to maximize the semantic matching degree and minimize the visual load, the visual load matching index is taken as a hard constraint condition, the target function is solved based on the multi-objective optimization algorithm of Pareto optimality, and a candidate solution set is generated; in the candidate solution set, the multi-objective is converted into a single objective according to the weighted summation strategy, the initial recommendation probability and the semantic matching degree are weighted and fused to determine the final recommendation score, and the recommendation list is output in descending order of the score.

[0014] The present application has the following beneficial effects: (1) The short video content accurate recommendation system based on artificial intelligence image recognition can consider the video content complexity and the user cognitive bearing capacity simultaneously in the video recommendation process by setting the visual load perception module and the cognitive matching module. The visual load perception module can realize accurate quantization of the visual complexity of the video content, and the adaptive dynamic normalization algorithm in the cognitive matching module can effectively evaluate the matching degree of the video content and the real-time cognitive state of the user. When it is detected that the visual load exceeds the current tolerance range of the user, the system dynamically adjusts the recommendation ranking by introducing the load penalty factor, significantly reduces the cognitive fatigue of the user caused by high visual complexity content, and improves the comfort and safety of the content recommendation. The existing recommendation method effectively solves the problem of lack of coordination between video complexity and user cognitive state, avoids the user visual fatigue and content loss caused by the recommended video with too high complexity, and improves the comfort and stability of the recommendation system.

[0015] (2) The short video content accurate recommendation system based on artificial intelligence image recognition can realize dynamic evolution of user interest and deep fusion of semantic content in the recommendation process through setting a dynamic interest modeling module and a multi-target recommendation decision module. Through the gated recurrent unit network and attention mechanism in the dynamic interest modeling module, accurate capture of real-time interest changes of the user and noise filtering are realized. In combination with the constraint optimization algorithm in the multi-target recommendation decision module, the visual load matching index is taken as a hard constraint condition to realize multi-target balanced optimization of user interest preferences and cognitive capacity while ensuring content relevance, so as to improve the recommendation accuracy while ensuring the comfort of user experience. The method solves the problem that the existing method has a single recommendation result and cannot consider the user's immediate state, so that the recommendation list can not only meet the user's interest focus, but also meet the user's cognitive capacity in different time periods, thereby significantly improving the accuracy of the recommendation result and the user experience.

[0016] Of course, implementing any product of the present application does not necessarily require all the advantages described above to be achieved at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The short video content accurate recommendation system based on artificial intelligence image recognition of the present application is shown in the flowchart. DETAILED DESCRIPTION

[0018] The short video content accurate recommendation system based on artificial intelligence image recognition of the present application solves the problem that the existing recommendation method lacks coordination between video complexity and user cognitive state, has a single recommendation result, and cannot consider the user's immediate interest and cognitive comfort.

[0019] The scheme in the present application embodiment has the following general idea: First, the visual load perception module and the cognitive matching module are used to jointly model the visual complexity of the short video content and the cognitive bearing range of the user, so as to realize dynamic matching of the video and the user state. Second, the dynamic interest vector reflecting the evolution law of the user's interest is obtained through the dynamic interest modeling module, and is deeply fused with the candidate short video semantic embedding vector. Finally, the visual load matching index is introduced as a constraint condition in the multi-target recommendation decision module, and the recommendation probability and the semantic matching degree are weighed and decided based on the multi-target optimization algorithm, so as to generate a recommendation list considering accuracy and comfort.

[0020] Please refer to Figure 1The embodiment of the application provides a technical scheme: a short video content accurate recommendation system based on artificial intelligence image recognition, comprising the following modules: a visual load perception module, which is used for extracting the space-time features of short video key frames through a multi-scale convolutional neural network, generating a visual complexity vector, collecting a user interactive micro-gesture feature sequence, and constructing a user cognitive load threshold curve for representing the tolerance range of the user to visual complexity in different time periods; a cognitive matching module, which is used for matching the visual complexity vector of a candidate short video with the cognitive load threshold curve of the user in the current time period through an adaptive dynamic regularization algorithm, obtaining a visual load matching index, introducing a load penalty factor to adjust the candidate short video ranking value when the visual load matching index exceeds a preset threshold range, and generating a candidate short video initial recommendation probability; a dynamic interest modeling module, which is used for modeling the user interactive micro-gesture feature sequence in time sequence based on a gated recurrent unit network combined with an attention mechanism, filtering noise interaction in a historical sequence through a reset gate, strengthening a recent related interest representation through an update gate, weighting historical behaviors through an attention weight, and outputting a user dynamic interest vector; and a multi-target recommendation decision module, which is used for performing multi-dimensional feature fusion analysis on the user dynamic interest vector and a candidate short video semantic embedding vector, obtaining a semantic matching degree, introducing the visual load matching index as a constraint condition, performing weighted decision on the candidate short video initial recommendation probability and the semantic matching degree through a multi-target optimization algorithm with constraints, and outputting a recommendation list.

[0021] In this embodiment, a visual load perception module is used to extract the spatio-temporal features of the key frames of the short video through a multi-scale convolutional neural network (CNN, a commonly used deep learning image feature extraction method). Based on these features, a visual complexity vector is generated, which represents the information density, color change rate, and object quantity of the video frame, reflecting the visual complexity of the video. At the same time, the system collects the user's interactive micro-gesture feature sequence (such as sliding pause, slight click, fingertip drag, etc.), infers the user's cognitive load state, and constructs a user cognitive load threshold curve, which describes the user's tolerance range for video visual complexity at different time periods. This can achieve dynamic perception of user visual fatigue and cognitive comfort. The cognitive matching module uses a self-adaptive dynamic warping algorithm (an improved algorithm of Dynamic Time Warping, used to compare the similarity of two non-fully aligned time series) to match the visual complexity vector of the candidate short video with the user's cognitive load threshold curve, obtaining a visual load matching index, which measures the adaptation of the candidate video to the user's current cognitive state. When the index exceeds the pre-set reasonable range, the system will introduce a load penalty factor, i.e. adding a negative correction term to the video ranking score to reduce the priority of videos that are too complex or too monotonous in the recommendation list, thereby generating an initial recommendation probability that meets the user's visual endurance. The dynamic interest modeling module uses a gated recurrent unit network (GRU, a recurrent neural network structure for processing time series data) combined with an attention mechanism to model the user's interactive micro-gesture feature sequence in time series. By resetting the gate to filter out irrelevant noise interaction data in the historical sequence, by updating the gate to strengthen the recent interest signals related to the candidate video, and by assigning different weights to different historical interaction behaviors through attention weights, a dynamic interest vector of the user is obtained. This vector not only reflects the user's long-term preferences, but also emphasizes the user's recent interest changes, improving the real-time and accuracy of the recommendation. The multi-objective recommendation decision module performs multi-dimensional feature fusion on the user's dynamic interest vector and the semantic embedding vector of the candidate short video (numerical representation of video content semantic information through a deep learning model), obtaining a semantic matching degree. On this basis, the aforementioned visual load matching index is introduced as a constraint condition, and a multi-objective optimization algorithm with constraints (i.e. an algorithm that optimizes both the recommendation probability and the semantic matching degree while being limited by the user's cognitive load threshold) is used to make a weighted decision on the initial recommendation probability and semantic matching degree of the candidate video, and finally output a recommendation list that meets the user's interest and cognitive comfort.By having the above modules work together, this invention can not only ensure that the recommendation results are highly relevant to the user's interests, but also effectively avoid viewing fatigue or boredom caused by excessively high or low video complexity, thereby improving the user experience and the overall performance of the recommendation system.

[0022] Specifically, the process of extracting spatiotemporal features of keyframes from short videos and generating a visual complexity vector using a multi-scale convolutional neural network is as follows: Keyframe sampling is performed on the input short video to extract spatiotemporal features of the keyframes, including scene complexity features, motion intensity features, color contrast features, and texture density features; global scene layout features are extracted through the large-scale branch of the multi-scale convolutional neural network, body motion trajectory features are extracted through the medium-scale branch, and image detail texture features are extracted through the small-scale branch; the features extracted at each scale are fused across levels, and a visual complexity score is calculated through a feature weighted aggregation algorithm to generate a multi-dimensional vector representing the visual complexity of the video.

[0023] In this implementation scheme, firstly, keyframe sampling is performed on the input short video to obtain a frame sequence. ,in This represents the number of keyframes obtained through sampling. Feature extraction is performed on each frame, yielding four types of features: scene complexity, motion intensity, color contrast, and texture density. Further, a multi-scale convolutional neural network is constructed, comprising three branches: a large-scale branch for extracting global scene layout features; a medium-scale branch for extracting object motion trajectory features; and a small-scale branch for extracting image detail texture features. Let the scale be... Next frame The extracted feature vector is Then the overall feature set is represented as: ;in, This represents the global scene features extracted from large-scale branches; This represents the motion trajectory features extracted by the mesoscale branch; This represents the texture detail features extracted from the small-scale branch. Subsequently, the features at different scales are fused across layers. A feature-weighted aggregation algorithm is used to calculate the overall visual complexity score. ;in: Keyframe Visual complexity score; This represents a nonlinear transformation function used to map features at various scales to a unified scoring space. represent the weighted coefficients of each scale; the determination method of the weighted coefficients is: correlation analysis is performed on the user subjective score results of a large number of video samples in the training set and the features extracted by the model, and the least mean square error optimization method is used to minimize the error between the predicted complexity score and the average user subjective score, so as to obtain the optimal weighted coefficients. After obtaining the complexity scores of each key frame, the visual complexity vector of the entire short video is calculated: ; wherein, represents a multi-dimensional vector representing the visual complexity of the short video; represents the complexity scores of each key frame arranged in time sequence, which can comprehensively reflect the complexity change of the video in the time and space dimensions.

[0024] Specifically, the specific process of constructing the user cognitive load threshold curve by collecting the user interactive micro-gesture feature sequence is as follows: continuously monitoring the micro-gesture interaction data of the user during browsing, including the acceleration change feature of the touch screen sliding track, the distribution feature of the page dwell time, the pupil focus trajectory feature and the touch screen pressure feature; time sequence segmentation and standardization processing are performed on the multi-modal interaction features, time domain and frequency domain features representing cognitive load are extracted, a mapping relationship from micro-gesture features to cognitive load level is established, and a threshold curve reflecting the change law of cognitive load of the user in different time periods is constructed by a time sequence analysis algorithm, which dynamically represents the real-time tolerance range of the user to the visual complexity.

[0025] In the embodiment, specifically, the specific process of constructing the user cognitive load threshold curve by collecting the user interactive micro-gesture feature sequence is as follows: first, continuously monitoring the micro-gesture interaction data of the user during browsing the short video, the data including: the acceleration change feature of the touch screen sliding track; the distribution feature of the page dwell time; the pupil focus trajectory feature; the touch screen pressure feature. Time sequence segmentation and standardization processing are performed on the above multi-modal interaction features to obtain the feature sequence: ; wherein, represents the sliding acceleration feature value at time ; represents the page dwell time feature value at time ; represents the pupil focus trajectory feature value at time ; represents the touch screen pressure feature value at time . Then, based on the above multi-modal features, time domain and frequency domain features are extracted to obtain a feature vector representing the cognitive load level: ; wherein, represents the cognitive load score at time ; represents a time-frequency domain feature extraction function, including statistical quantities such as mean, variance and frequency energy ratio. is the weighted coefficient representing the multimodal feature; the determination method of the weighted coefficient is: regression fitting is performed on the actual interaction data of the user in the training set and the subjective cognitive load label result, and the least square method is used to estimate the coefficient value, so as to ensure that the error between the cognitive load score output by the model and the real cognitive load level is minimized. After obtaining the sequence , a time series smoothing analysis method (such as exponential weighted moving average) is used to construct a user cognitive load threshold curve: ; wherein: is the cognitive load threshold at time ; is a smoothing factor, the value range of which is determined by experimental data; is the threshold result of the previous time. The curve obtained through the above process can dynamically reflect the real-time tolerance range of the user to the visual complexity in different time periods, and provide constraint conditions for subsequent visual load matching.

[0026] Specifically, the specific process of obtaining the visual load matching index by matching the visual complexity vector of the candidate short video with the cognitive load threshold curve of the user in the current time period through the adaptive dynamic warping algorithm is as follows: the feature distance matrix of the visual complexity vector of the candidate short video and the cognitive load threshold curve of the user at each time point is calculated; the optimal matching path between the two sequences is found through the dynamic time warping algorithm, wherein the warping window size is adaptively adjusted according to the fluctuation degree of the curve; the matching degree score is calculated according to the cumulative distance of the optimal matching path, and the path length and the bending degree are normalized to generate the visual load matching index representing the matching degree of the video content and the user's cognitive state.

[0027] In this embodiment, first, the visual complexity vector of the candidate short video on the key frame sequence is: ; wherein: is the visual complexity value corresponding to the th key frame; is the number of sampling key frames in the candidate short video. At the same time, the cognitive load threshold curve of the user in the current time period is: ; wherein: is the cognitive load threshold at the th time point; is the number of sampling points of the threshold curve. Then, the feature distance matrix between the visual complexity sequence of the candidate video and the threshold curve of the user is constructed: ; wherein: is the visual complexity of the th frame of the candidate video and the cognitive load threshold of the user at the ​the difference of the time point threshold. On this basis, the dynamic time warping (DTW) method is used to calculate the optimal matching path between the two sequences. The recursive formula is: ; wherein: represents the minimum cumulative distance matched to the sequence position ; the initial condition is . In order to avoid distortion caused by excessive bending path, the warping window size ω is set, and the curve fluctuation degree is adaptively adjusted: ; wherein: ω represents the window size of dynamic warping; γ represents the adjustment coefficient; represents the standard deviation of the threshold curve. When the threshold curve fluctuates greatly, the window range is automatically expanded to capture more nonlinear matching relationships. The final matching path is P, and the cumulative distance is M(m, n). In order to eliminate the influence of path length and bending degree, the matching result is normalized to obtain the visual load matching index: ; wherein: Ω represents the visual load matching index, and the value closer to 1 indicates that the video complexity is more suitable for the user's cognitive threshold; represents the length of the optimal matching path; represents the bending degree of the matching path; κ represents the bending degree penalty coefficient, and the determination method is: through cross-validation experiment, the optimal value which can stably maintain the recommendation accuracy in different user groups is selected. Through the above steps, the visual load matching index obtained can accurately represent the adaptation degree between the video content and the current cognitive state of the user, and provide accurate basis for the screening and sorting of candidate videos.

[0028] Specifically, when the visual load matching index exceeds the preset threshold range, a load penalty factor is introduced to adjust the candidate short video sorting value, and the specific process of generating the candidate short video initial recommendation probability is as follows: according to the deviation degree of the visual load matching index and the preset threshold, the load penalty factor value is calculated through a nonlinear function, the load penalty factor and the original sorting value are weighted and adjusted, wherein the penalty factor weight is dynamically adjusted according to the current cognitive load level of the user; the adjusted sorting value is mapped to the probability space through the Sigmoid function to generate the candidate short video initial recommendation probability, and the probability value reflects the matching degree between the video content and the user's cognitive bearing capacity.

[0029] In the embodiment, the visual load matching index is , and the visual load threshold preset by the system is: ; wherein: represents the matching degree between the video content and the user's cognitive threshold, and the value range is , and the larger the value is, the more suitable it is; The threshold upper limit of the user's cognitive acceptance is represented, and the determination method is as follows: according to historical user experimental data, the average upper limit of the user's endurance when watching videos of different complexities is counted, and the stable value is obtained through quantile method. When the two are deviated, the deviation degree is calculated: ; wherein: The difference between the matching index and the threshold value is represented, and the greater the value, the heavier the cognitive burden of the video on the user. Based on the deviation degree, a nonlinear function is introduced to calculate the load penalty factor: ; wherein: The load penalty factor is represented, and the value range ; The adjustment coefficient is represented, and the determination method is as follows: under multiple user experimental data, the optimal value of the video recommendation result that can effectively distinguish between mild and heavy load is selected. Then, the penalty factor and the original ranking value of the candidate video are weighted and adjusted: ; wherein: The original ranking value of the candidate short video is represented; The adjusted ranking value is represented; The penalty weight coefficient is represented, and the determination method is as follows: according to the real-time cognitive load level of the user, when the user's interactive performance shows fatigue characteristics, the value increases, and when the user is in a state of full energy, the value decreases. Finally, the adjusted ranking value is mapped to the probability space through the Sigmoid function to obtain the initial recommendation probability of the candidate short video: ; wherein: The initial recommendation probability of the candidate short video is represented, and the closer the value is to 1, the more likely the video is to be recommended; The scaling coefficient is represented, and the determination method is as follows: through grid search or cross-validation, the optimal value that can make the prediction result distribution most balanced on the training set is selected. Through the above process, the system can fine-tune the recommendation result under the premise of considering the user's cognitive endurance, so that the recommendation probability not only depends on the semantic relevance of the video, but also reflects the real-time cognitive burden state of the user, thereby avoiding recommending too complex or too stimulating content.

[0030] Specifically, based on the gated recurrent unit network combined with the attention mechanism, the logical process of time series modeling of the user interaction micro-gesture feature sequence is as follows: the user interaction micro-gesture feature sequence is input into the gated recurrent unit network, and the time step is calculated to capture the time series dependence in the sequence; the attention mechanism is introduced based on the hidden layer output of the gated recurrent unit network, and the attention weight of each time step output is calculated; the hidden states of each time step are weighted and summed according to the attention weight, to generate a latent vector representation representing the user's dynamic interest, and the latent vector representation is fused with the input feature at the current time to output the time series modeling result of the user's dynamic interest.

[0031] In this embodiment, specifically, the logic process of modeling the sequence of user interaction micro-gesture features based on the gated recurrent unit network combined with the attention mechanism is as follows: first, the sequence of micro-gesture features generated by the user in the short video browsing process is input into the gated recurrent unit network (GRU) step by step, and the network captures the time sequence dependency in the sequence through progressive calculation, so that the historical behavior information can effectively affect the interest modeling at the current time. In this process, the GRU selectively retains the historical state related to the current interest and filters out irrelevant or noise interactions through the internal reset gate and update gate mechanism, thereby ensuring that the dynamic interest vector can truly reflect the user's attention trend. Subsequently, the attention mechanism is introduced based on the output of the GRU network hidden layer, the hidden states at different time steps are weighted and evaluated, and the contribution of each time step to the overall interest expression is calculated, so that the model can automatically focus on the recent or key user interaction behavior. By weighting and summing the hidden states at each time step according to the attention weight, a latent vector representation representing the user's dynamic interest is generated. This vector not only retains the long-term trend of the user's historical behavior, but also reflects the interest preference in the current time period. Finally, the latent vector representation is fused with the input features at the current time to form a complete user dynamic interest representation, which is input into the subsequent recommendation decision module, thereby realizing accurate recommendation of candidate short videos. The entire process logically realizes a continuous processing link from historical behavior filtering, time sequence dependency modeling, to key behavior weighting, and finally to dynamic interest output, so that the user interest modeling is both detailed and dynamically responsive to real-time behavior changes of the user.

[0032] Specifically, the specific process of outputting the user dynamic interest vector through the reset gate to filter noise interactions in the historical sequence, the update gate to strengthen the representation of recent relevant interest, and the attention weight to weight the historical behavior is as follows: the reset gate is used to calculate the retention ratio of the historical hidden state, filter the noise interaction information irrelevant to the current interest, and the update gate is used to control the fusion ratio of the current time input information and the historical information, and strengthen the representation of recent relevant interest. The importance weight of each time step in the historical sequence is calculated through the attention mechanism, and the weighted hidden state is aggregated. The feature vector processed by the gating mechanism and attention weighting is spliced and linearly transformed to generate the user dynamic interest vector.

[0033] In this embodiment, the input sequence and the hidden state initialization are as follows: wherein: is the input feature vector at the i-th time step, including touch swipe, dwell time, and pupil trajectory features; is the total length of the user interaction sequence. The hidden state is initialized as: wherein: ​The initial hidden state is set to zero vector. The reset gate filters out the noise interactions at each time step, and the calculation formula of the reset gate is: ; wherein: : the reset gate vector, the value range is , which represents the retention ratio of historical information; : the weight matrix of input features to the reset gate, and the parameter initialization can use the Xavier method; : the weight matrix of the previous hidden state to the reset gate; : the bias vector of the reset gate; : the Sigmoid activation function. Effect: if value is small, the current step will shield more historical information, thereby filtering out the noise interactions irrelevant to user interest. The update gate strengthens the recent interest, and the calculation formula of the update gate is: ; wherein: : the update gate vector, the value range is , which represents the fusion ratio of current input and historical information; : the weight matrix of input features to the update gate; : the weight matrix of the previous hidden state to the update gate; : the bias vector of the update gate. The candidate hidden state is: ; wherein: : the candidate hidden state; : the weight matrix of input features to the candidate hidden state; : the weight matrix of historical state to the candidate hidden state; : the bias vector of the candidate hidden state; : the vector element-wise multiplication operation. The final hidden state update formula is: ; wherein: : the hidden state of the th time step, which comprehensively considers historical interest and current input features. After the attention mechanism calculates the weight to obtain the hidden state sequence , the attention mechanism is introduced: ; wherein: : the attention weight of the mth time step, reflecting the contribution of the time step to user interest; : the weight matrix of attention calculation; : the bias vector of attention calculation; : the context weight vector, which is determined by: in the training process, it is automatically learned by minimizing the recommendation error. The aggregated interest representation is: ; wherein: The attention-weighted aggregation vector represents the weighted interest information of all historical behaviors. The dynamic interest vector output splices the final hidden state and the attention aggregation vector and performs linear transformation: ; wherein: : the user dynamic interest vector; : the splicing result of the interest aggregation vector and the final hidden state; : the weight matrix of linear transformation; : the bias vector of linear transformation.

[0034] Specifically, the specific process of obtaining the semantic matching degree by performing multi-dimensional feature fusion analysis on the user dynamic interest vector and the candidate short video semantic embedding vector is as follows: the cross-modal attention mechanism is used to calculate the interaction weight between the user dynamic interest vector and the candidate short video semantic embedding vector, and an associated mapping of user interest and video content is established; the two vectors are fused by combining feature splicing and element-level multiplication, the linear and nonlinear relationships between user interest and video content are captured, and high-order cross features are extracted by performing deep nonlinear transformation on the fused feature vector through a multilayer perception machine; the high-order cross features are mapped to a semantic matching degree score through a fully connected layer, and the score represents the relevance of user interest and video content.

[0035] In the embodiment, the cross-modal attention calculates the interaction weight by using the cross-modal attention mechanism to calculate the interaction weight between the user dynamic interest vector and the candidate short video semantic embedding vector , and establishes an associated mapping of user interest and video content. The formula is: ; the parameters are as follows: : the user dynamic interest vector, output by the gating recurrent unit network combined with the attention mechanism of the previous module; : the candidate short video semantic embedding vector, generated by the video content feature encoder; : the attention weight matrix, which can be randomly initialized initially and updated adaptively through training; : the interaction weight matrix, normalized by softmax, representing the association strength of each dimension of the user interest vector and the video semantic vector; the key of this step is to quantify the associated mapping of user interest and video features through the attention mechanism, providing a weight reference for subsequent feature fusion. Feature fusion fuses the user interest vector and the video semantic vector, and captures linear and nonlinear relationships by combining feature splicing and element-level multiplication. The formula is: ; the parameters are as follows: : the fused feature vector; : the vector splicing operation connects and in the feature dimension; : element-wise product, representing the multiplication of corresponding dimensions to capture nonlinear interaction features; Note: the fusion operation can simultaneously preserve the original feature information and interaction information, improving the feature representation capability. High-order nonlinear feature transformation is performed on the fused feature vector to extract high-order cross features. The formula is as follows: ; Parameter description: : high-order cross feature vector : the layer activation function (such as ReLU) is used to introduce nonlinearity : the layer weight matrix is randomly generated initially and optimized through training : the layer bias vector : the number of layers of the multi-layer perception is determined by experiment, generally 2-4 layers; Note: through multiple nonlinear transformations, the model can capture the high-order relationship between complex user interests and video semantics. Map to semantic matching degree score input the fully connected layer to generate the final semantic matching degree score , representing the relevance of user interest and video content. The formula is as follows: ; Parameter description: : semantic matching degree score, usually ranging from 0 to 1 : Sigmoid activation function, used to map to the probability space : fully connected layer weight matrix : fully connected layer bias term; Weight and threshold determination method: : the weight matrix is randomly generated initially and optimized through back propagation and iterative training data; the bias is initially zero and can be trained and adjusted; the Sigmoid output is naturally mapped to 0-1.

[0036] Specifically, the visual load matching index is introduced as a constraint condition, and the initial recommendation probability and semantic matching degree of the candidate short video are weighted and decided by a multi-objective optimization algorithm with constraints, and the specific process of outputting the recommendation list is as follows: a double-objective optimization function is established to maximize the semantic matching degree and minimize the visual load, the visual load matching index is taken as a hard constraint condition, the multi-objective optimization algorithm based on Pareto optimality is used to solve the objective function, and a candidate solution set is generated; in the candidate solution set, the multi-objective is converted into a single objective according to the weighted sum strategy, the initial recommendation probability and the semantic matching degree are weighted and fused to determine the final recommendation score, and the recommendation list is output in descending order of score.

[0037] In this embodiment, a double-objective optimization function is established to maximize the semantic matching degree and minimize visual load To optimize the objective, and using the visual load matching index as a hard constraint, a candidate solution set is generated. Formula: Parameter description: : Recommendation decision vector for candidate short videos, representing the recommendation probability allocation for each candidate video; The semantic matching degree function maps the semantic matching degree score calculated in the previous step to the recommendation probability space. Visual load function, used to quantify the impact of the visual complexity of video content on the user's cognitive load; The visual load threshold is dynamically determined by the user's cognitive load level (e.g., based on moving averages, extreme values, or percentile statistical methods). Note: The core of this step is to simultaneously incorporate "content relevance" and "visual load" into the decision-making process, ensuring that the recommended list aligns with the user's interests without overwhelming their senses. The Pareto optimal solution candidate set is generated by solving the objective function using the Pareto multi-objective optimization algorithm. Each solution represents a balance between the two objectives. Formula: Parameter description: : Candidate solution set, containing recommendation vectors that satisfy the Pareto optimality condition; ParetoOpt: Pareto optimality function, used to select the best solution from the best solutions. Maximize and Minimize the vector that cannot be dominated by other solutions; The set of all possible recommendation decision vectors; Note: This step ensures that the generated candidate solutions achieve an optimal balance between "interest matching degree" and "visual load". Multi-objective weighted summation transforms into a single objective in the candidate solution set, converting the dual-objective approach into a single objective, combined with the initial recommendation probability. and semantic matching degree Weighted fusion is performed to determine the final recommendation score. . formula: Parameter description: : Final recommendation score, used to sort and output the recommendation list; The initial recommendation probability of candidate short videos is generated from the ranking value adjusted for visual load. Semantic matching score, generated by the fusion of multi-dimensional features from the previous step; Visual load matching index; : Weighting coefficients, representing the contributions of initial probability, semantic matching degree, and visual load to the final recommendation score; the optimal ratio can be determined by optimizing recommendation accuracy and user load satisfaction on the training set through grid search or cross-validation; Note: Weighted fusion can integrate two objectives into a single score while preserving the constraint on user visual load. Output recommendation list based on the final recommendation score. The candidate short videos are sorted in descending order to generate a final recommendation list. Formula: ; Parameter description: : The final sorted short video recommendation list; : The final recommendation score of the i-th candidate video; : The total number of candidate short videos; Description: The sorted list takes into account the user interest, video content relevance and visual load constraint, ensuring that the recommended results are scientific and reasonable.

[0038] In summary, the present application has at least the following effects: The short video content precise recommendation system based on artificial intelligence image recognition extracts the spatiotemporal features of the key frames of the short video through a multi-scale convolutional neural network, and constructs a cognitive load threshold curve by combining user interactive micro-gesture features, thereby realizing precise modeling of the visual tolerance range of the user at different time periods and effectively avoiding the problems of visual overload or insufficient interest matching degree caused by a single recommendation strategy. The user interactive micro-gesture sequence is sequentially modeled based on a gated recurrent unit network combined with an attention mechanism, the reset gate filters noise interaction, the update gate strengthens recent related interests, and the attention weights historical behavior, and a user dynamic interest vector is output, so that the system can reflect the changes in user interest in real time and improve the relevance and real-time performance of the recommended content. The user dynamic interest vector and the semantic embedding vector of the candidate short video are fused through a cross-modal attention mechanism, feature splicing and a multi-layer perception machine, high-order cross features are extracted, and a semantic matching degree score is generated, thereby enhancing the relevance between user interest and video content and improving the precision and personalization level of the recommendation. By introducing a visual load matching index as a constraint condition and using a multi-objective optimization algorithm with constraints to weight the initial recommendation probability and semantic matching degree of the candidate short video, the balance between maximum interest matching degree and minimum visual load is realized, so that the final recommendation list meets the user interest and ensures the visual comfort, thereby improving the overall satisfaction of the user experience.

[0039] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems or computer program products. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) having computer-usable program code embodied therein.

[0040] ​The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the one or more flowcharts and / or blocks

[0041] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the one or more flowcharts and / or blocks

[0042] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the one or more flowcharts and / or blocks

[0043] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments described and shown, and it is therefore intended that the application cover any and all variations of the preferred embodiments which fall within the scope of the present application. Accordingly, the appended claims are intended to cover all such modifications and variations as falling within the scope of the application.

[0044] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover the modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A short video content accurate recommendation system based on artificial intelligence image recognition, characterized in that, The method comprises the following modules: A visual load perception module is configured to extract the spatiotemporal features of key frames of a short video through a multi-scale convolutional neural network, generate a visual complexity vector, and construct a user cognitive load threshold curve by collecting a user interaction micro-gesture feature sequence, the user cognitive load threshold curve being used to represent the tolerance range of the user to visual complexity at different time periods; A cognitive matching module is configured to match the visual complexity vector of a candidate short video with the cognitive load threshold curve of the user at the current time period through an adaptive dynamic normalization algorithm, obtain a visual load matching index, introduce a load penalty factor to adjust the ranking value of the candidate short video when the visual load matching index exceeds a preset threshold range, and generate an initial recommendation probability of the candidate short video; A dynamic interest modeling module is configured to model the user interaction micro-gesture feature sequence in time series based on a gated recurrent unit network combined with an attention mechanism, filter noise interactions in the historical sequence through a reset gate, strengthen recent relevant interest representations through an update gate, and weight historical behaviors through an attention weight, and output a user dynamic interest vector; A multi-target recommendation decision module is configured to perform multi-dimensional feature fusion analysis on the user dynamic interest vector and a semantic embedding vector of a candidate short video to obtain a semantic matching degree, introduce the visual load matching index as a constraint condition, and perform weighted decision on the initial recommendation probability of the candidate short video and the semantic matching degree through a multi-objective optimization algorithm with constraints, and output a recommendation list.

2. The short video content precise recommendation system based on artificial intelligence image recognition according to claim 1, characterized in that: The specific process of extracting the spatiotemporal features of key frames of a short video through a multi-scale convolutional neural network to generate a visual complexity vector is as follows: Key frames of the input short video are sampled to extract the spatiotemporal features of the key frames, including scene complexity features, motion intensity features, color contrast features, and texture density features; Global scene layout features are extracted through a large-scale branch of the multi-scale convolutional neural network, object motion trajectory features are extracted through a medium-scale branch, and picture detail texture features are extracted through a small-scale branch; The features extracted at different scales are fused across layers, a visual complexity score is calculated through a feature weighting aggregation algorithm, and a multi-dimensional vector representing the visual complexity of the video is generated.

3. The short video content precise recommendation system based on artificial intelligence image recognition according to claim 2, characterized in that: The specific process of collecting a user interaction micro-gesture feature sequence to construct a user cognitive load threshold curve is as follows: Micro-gesture interaction data of the user during browsing is continuously monitored, including acceleration change features of touch screen sliding trajectories, distribution features of page dwell time, pupil focus trajectory features, and touch screen pressure features; Multi-modal interaction features are subjected to time series segmentation and standardization processing, time domain and frequency domain features representing cognitive load are extracted, a mapping relationship from micro-gesture features to cognitive load levels is established, a threshold curve reflecting the change law of the cognitive load of the user at different time periods is constructed through a time series analysis algorithm, and the threshold curve dynamically represents the real-time tolerance range of the user to visual complexity.

4. The short video content precise recommendation system based on artificial intelligence image recognition of claim 1, characterized in that: The specific process of matching the visual complexity vector of a candidate short video with the cognitive load threshold curve of the user at the current time period through an adaptive dynamic normalization algorithm to obtain a visual load matching index is as follows: A feature distance matrix of the visual complexity vector of the candidate short video and the cognitive load threshold curve of the user at each time point is calculated; An optimal matching path between two sequences is found by a dynamic time warping algorithm, wherein a warping window size is adaptively adjusted according to a curve fluctuation degree; A matching degree score is calculated according to a cumulative distance of the optimal matching path, and is normalized by combining a path length and a bending degree to generate a visual load matching index representing a matching degree of video content and a user cognitive state.

5. The short video content precise recommendation system based on artificial intelligence image recognition according to claim 4, characterized in that: When the visual load matching index exceeds a preset threshold range, a load penalty factor is introduced to adjust a candidate short video ranking value, and a specific process of generating an initial recommendation probability of the candidate short video is as follows: According to a deviation degree of the visual load matching index from the preset threshold, a load penalty factor value is calculated by a nonlinear function, and the load penalty factor is weighted and adjusted with an original ranking value, wherein a penalty factor weight is dynamically adjusted according to a current cognitive load level of the user; An adjusted ranking value is mapped to a probability space by a Sigmoid function to generate an initial recommendation probability of the candidate short video, and the probability value reflects a matching degree of video content and a user cognitive bearing capacity.

6. The short video content precise recommendation system based on artificial intelligence image recognition according to claim 1, characterized in that: A logic process of time series modeling of a user interactive micro-gesture feature sequence based on a gated recurrent unit network combined with an attention mechanism is as follows: The user interactive micro-gesture feature sequence is input into the gated recurrent unit network to capture time series dependence in the sequence by time step progression calculation; An attention mechanism is introduced based on a hidden layer output of the gated recurrent unit network to calculate an attention weight of each time step output; A hidden state of each time step is weighted and summed according to the attention weight to generate a latent vector representation representing a user dynamic interest, and the latent vector representation is fused with an input feature of a current time to output a time series modeling result of the user dynamic interest.

7. The short video content precise recommendation system based on artificial intelligence image recognition according to claim 6, characterized in that: A specific process of outputting a user dynamic interest vector by filtering noise interaction in a historical sequence through a reset gate, strengthening a recent related interest representation through an update gate, and weighting historical behaviors through an attention weight is as follows: A reservation proportion of a historical hidden state is calculated by the reset gate to filter noise interaction information irrelevant to a current interest; A fusion proportion of current time input information and historical information is controlled by the update gate to strengthen a recent related interest representation; Importance weights of each time step in a historical sequence are calculated by the attention mechanism, and the weighted hidden states are aggregated; The feature vectors processed by the gating mechanism and the attention weighting are spliced and linearly transformed to generate the user dynamic interest vector.

8. The short video content precise recommendation system based on artificial intelligence image recognition of claim 1, characterized in that: A specific process of obtaining a semantic matching degree by performing multi-dimensional feature fusion analysis on the user dynamic interest vector and a candidate short video semantic embedding vector is as follows: An interaction weight between the user dynamic interest vector and the candidate short video semantic embedding vector is calculated by a cross-modal attention mechanism to establish an associated mapping of user interest and video content; The two vectors are fused by splicing and element-level multiplication to capture linear and nonlinear relationships between user interest and video content, and a high-order cross feature is extracted by performing deep nonlinear transformation on the fused feature vector through a multilayer perceptron; The high-order cross feature is mapped to a semantic matching degree score through a fully connected layer, and the score represents a correlation degree of user interest and video content. 9.The short video content recommendation system based on artificial intelligence image recognition according to claim 8, characterized in that: The specific process of outputting the recommendation list is as follows: A double-objective optimization function is established to maximize the semantic matching degree and minimize the visual load, the visual load matching index is taken as a hard constraint condition, and a multi-objective optimization algorithm based on Pareto optimality is used to solve the objective function to generate a candidate solution set; In the candidate solution set, the multi-objective is converted into a single objective according to the weighted summation strategy, the initial recommendation probability and the semantic matching degree are weighted and fused to determine the final recommendation score, and the recommendation list is output in descending order of score.

Images