Short video content accurate recommendation system based on artificial intelligence image recognition

Abstract

The application discloses a short video content accurate recommendation system based on artificial intelligence image recognition, and particularly relates to the technical field of short video personalized recommendation, and is used for solving the matching problem of user interest and visual bearing capacity. Through a multi-scale convolutional neural network, the spatiotemporal features of key frames of a short video are extracted, a cognitive load threshold curve is constructed in combination with a user micro-gesture sequence, the tolerance range of a user to visual complexity in different time periods is represented, and a visual complexity vector is generated; an adaptive dynamic normalization algorithm is used to calculate a visual load matching index, and an initial recommendation probability is generated by adjusting the ranking of candidate videos through a load penalty factor; a user dynamic interest vector is modeled based on a gated recurrent unit network in combination with an attention mechanism; finally, multi-dimensional feature fusion is performed on user interest and video semantics, and a final recommendation list is generated through a multi-objective optimization algorithm under the constraint of visual load, so that personalized recommendation with maximum interest matching degree and optimized visual comfort is realized.

Description

Technical Field

[0001] This invention relates to the field of personalized recommendation technology for short videos, specifically a precise recommendation system for short video content based on artificial intelligence image recognition. Background Technology

[0002] With the widespread adoption of smart devices and high-speed mobile networks, short videos have gradually become an important channel for people to obtain information and entertainment. The sheer number of short videos has dramatically expanded users' choices, making recommendation systems a crucial element. Existing recommendation platforms generally rely on big data mining and deep learning models to analyze users' historical viewing records, likes, and sharing behaviors to achieve personalized recommendations, thereby improving user engagement and content exposure.

[0003] However, existing recommendation methods still have significant shortcomings in practical applications. On the one hand, they typically rely on single metrics such as clicks or viewing time, making it difficult to accurately reflect the user's immediate cognitive state and psychological load during viewing. This can easily lead to overly complex or overly simplistic recommended content, causing user fatigue or aesthetic apathy. On the other hand, recommendation results often emphasize interest similarity while neglecting the matching relationship between visual complexity and the user's instantaneous cognitive capacity. This results in some recommended videos, although highly relevant to interests, placing an additional visual burden on the user, thereby reducing the effectiveness of the recommendation and the user experience. Furthermore, existing recommendation decisions are mostly based on a single optimization objective, lacking comprehensive consideration of multi-dimensional constraints. This can easily create a conflict between recommendation accuracy and viewing comfort, affecting the overall performance of the system. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a precise recommendation system for short video content based on artificial intelligence image recognition, thus solving the problems mentioned in the background.

[0005] To achieve the above objectives, this invention provides the following technical solution: a short video content accurate recommendation system based on artificial intelligence image recognition, comprising the following modules: a visual load perception module, used to extract spatiotemporal features of key frames in short videos through a multi-scale convolutional neural network to generate a visual complexity vector, and to construct a user cognitive load threshold curve by collecting user interaction micro-gesture feature sequences to characterize the user's tolerance range for visual complexity at different time periods; and a cognitive matching module, used to match the visual complexity vector of candidate short videos with the user's cognitive load threshold curve for the current time period using an adaptive dynamic warping algorithm to obtain a visual load matching index, and to introduce load when the visual load matching index exceeds a preset threshold range. The penalty factor adjusts the ranking values ​​of candidate short videos to generate initial recommendation probabilities. The dynamic interest modeling module, based on a gated recurrent unit network and attention mechanism, performs temporal modeling of user interaction micro-gesture feature sequences. It filters noisy interactions in historical sequences through a reset gate, strengthens recently relevant interest representations through an update gate, and weights historical behaviors with attention weights to output a user dynamic interest vector. The multi-objective recommendation decision module performs multi-dimensional feature fusion analysis on the user dynamic interest vector and the semantic embedding vector of candidate short videos to obtain semantic matching degree. It introduces a visual load matching index as a constraint and uses a constrained multi-objective optimization algorithm to weight the initial recommendation probability and semantic matching degree of candidate short videos, outputting a recommendation list.

[0006] Furthermore, the specific process of extracting spatiotemporal features of keyframes in short videos and generating visual complexity vectors through multi-scale convolutional neural networks is as follows: Keyframe sampling is performed on the input short video to extract spatiotemporal features of the keyframes, including scene complexity features, motion intensity features, color contrast features, and texture density features; global scene layout features are extracted through the large-scale branch of the multi-scale convolutional neural network, body motion trajectory features are extracted through the medium-scale branch, and image detail texture features are extracted through the small-scale branch; the features extracted at each scale are fused across levels, and a visual complexity score is calculated through a feature weighted aggregation algorithm to generate a multi-dimensional vector representing the visual complexity of the video.

[0007] Furthermore, the specific process of constructing a user cognitive load threshold curve by collecting user interaction micro-gesture feature sequences is as follows: continuously monitor the user's micro-gesture interaction data during browsing, including the acceleration change characteristics of touch screen swipe trajectory, the distribution characteristics of page dwell time, pupil focusing trajectory characteristics, and touch screen pressure characteristics; perform time series segmentation and standardization processing on multimodal interaction features, extract time domain and frequency domain features representing cognitive load, establish a mapping relationship from micro-gesture features to cognitive load level, and construct a threshold curve reflecting the user's cognitive load change pattern in different time periods through time series analysis algorithms. This curve dynamically represents the user's real-time tolerance range for visual complexity.

[0008] Furthermore, the visual load matching index is obtained by matching the visual complexity vector of the candidate short video with the cognitive load threshold curve of the user's current time period using an adaptive dynamic warping algorithm. The specific process is as follows: calculate the feature distance matrix between the visual complexity vector of the candidate short video and the user's cognitive load threshold curve at each time point; find the optimal matching path between the two sequences using a dynamic time warping algorithm, wherein the warping window size is adaptively adjusted according to the degree of curve fluctuation; calculate the matching degree score based on the cumulative distance of the optimal matching path, and perform normalization processing by combining the path length and curvature to generate a visual load matching index that represents the degree of matching between the video content and the user's cognitive state.

[0009] Furthermore, when the visual load matching index exceeds a preset threshold, a load penalty factor is introduced to adjust the ranking value of candidate short videos. The specific process for generating the initial recommendation probability of candidate short videos is as follows: Based on the degree of deviation between the visual load matching index and the preset threshold, the load penalty factor value is calculated through a nonlinear function. The load penalty factor is then weighted and adjusted with the original ranking value, where the penalty factor weight is dynamically adjusted according to the user's current cognitive load level. The adjusted ranking value is mapped to the probability space through the Sigmoid function to generate the initial recommendation probability of candidate short videos. This probability value reflects the degree of matching between the video content and the user's cognitive tolerance.

[0010] Furthermore, the logical process of temporal modeling of user interaction micro-gesture feature sequences based on gated recurrent unit networks (GRN networks) combined with attention mechanisms is as follows: The user interaction micro-gesture feature sequences are input into the GRN network, and temporal dependencies in the sequence are captured through progressive calculations at time steps; an attention mechanism is introduced based on the hidden layer output of the GRN network to calculate the attention weights of the output at each time step; the hidden states at each time step are weighted and summed according to the attention weights to generate a latent vector representation representing the user's dynamic interests; the latent vector representation is fused with the input features at the current moment to output the temporal modeling result of the user's dynamic interests.

[0011] Furthermore, the specific process of outputting the user's dynamic interest vector by filtering noisy interactions in the historical sequence through the reset gate, strengthening the representation of recent relevant interests through the update gate, and weighting historical behavior with attention weights is as follows: The reset gate calculates the retention ratio of historical hidden states to filter out noisy interaction information that is irrelevant to the current interest; the update gate controls the fusion ratio of the current input information and historical information to strengthen the representation of recent relevant interests; the attention mechanism calculates the importance weight of each time step in the historical sequence and aggregates the weighted hidden states; the feature vectors processed by the gating mechanism and attention weighting are concatenated and linearly transformed to generate the user's dynamic interest vector.

[0012] Furthermore, the semantic matching degree is obtained by performing multi-dimensional feature fusion analysis on the user dynamic interest vector and the candidate short video semantic embedding vector as follows: The interaction weights between the user dynamic interest vector and the candidate short video semantic embedding vector are calculated through a cross-modal attention mechanism to establish a mapping between user interests and video content; the two vectors are fused by a combination of feature concatenation and element-wise multiplication to capture the linear and non-linear relationships between user interests and video content; a deep non-linear transformation is performed on the fused feature vector using a multilayer perceptron to extract high-order cross features; and the high-order cross features are mapped to a semantic matching degree score through a fully connected layer, which represents the degree of correlation between user interests and video content.

[0013] Furthermore, a visual load matching index is introduced as a constraint. A weighted decision is made on the initial recommendation probability and semantic matching degree of candidate short videos using a constrained multi-objective optimization algorithm to output the recommendation list. The specific process is as follows: A bi-objective optimization function is established with the goal of maximizing semantic matching degree and minimizing visual load. The visual load matching index is used as a hard constraint. The objective function is solved based on the Pareto optimal multi-objective optimization algorithm to generate a candidate solution set. In the candidate solution set, the multi-objective is transformed into a single objective according to the weighted summation strategy. The initial recommendation probability and semantic matching degree are weighted and fused to determine the final recommendation score. The recommendation list is then output in descending order of score.

[0014] The present invention has the following beneficial effects:

[0015] (1) The short video content accurate recommendation system based on artificial intelligence image recognition, by setting up a visual load perception module and a cognitive matching module, can simultaneously consider the complexity of video content and the user's cognitive tolerance during the video recommendation process. The visual load perception module realizes the accurate quantification of the visual complexity of video content, and combined with the adaptive dynamic warping algorithm in the cognitive matching module, it can effectively evaluate the degree of matching between video content and the user's real-time cognitive state. When the visual load is detected to exceed the user's current tolerance range, the system dynamically adjusts the recommendation ranking by introducing a load penalty factor, which significantly reduces the cognitive fatigue caused by high visual complexity content to users and improves the comfort and safety of content recommendation. It effectively solves the problem of lack of coordination between video complexity and user cognitive state in existing recommendation methods, avoids user visual fatigue and content loss caused by recommending videos with excessive complexity, and thus improves the comfort and stability of the recommendation system.

[0016] (2) The short video content accurate recommendation system based on artificial intelligence image recognition, by setting up a dynamic interest modeling module and a multi-objective recommendation decision module, can realize the deep integration of user interest dynamic evolution and semantic content during the recommendation process. Through the gated recurrent unit network and attention mechanism in the dynamic interest modeling module, the system can accurately capture and filter noise of real-time changes in user interests. Combined with the constraint optimization algorithm in the multi-objective recommendation decision module, while ensuring content relevance, the system uses the visual load matching index as a hard constraint condition, realizing the multi-objective balance optimization of user interest preferences and cognitive tolerance, thereby improving the accuracy of recommendation while ensuring the comfort of user experience. This solves the problem that the recommendation results of existing methods have a single objective and cannot take into account the user's real-time state, so that the recommendation list can not only match the user's interest focus, but also meet the user's cognitive tolerance at different time periods, thereby significantly improving the accuracy of recommendation results and user experience.

[0017] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0018] Figure 1 This is a flowchart of the short video content accurate recommendation system based on artificial intelligence image recognition, as described in this invention. Detailed Implementation

[0019] This application's embodiments address the problems of existing recommendation methods, such as a lack of coordination between video complexity and user cognitive state, a single-target recommendation result, and difficulty in balancing user immediate interests and cognitive comfort, through a short video content accurate recommendation system based on artificial intelligence image recognition.

[0020] The overall concept of the solution in this application embodiment is as follows:

[0021] First, by using a visual load perception module and a cognitive matching module, the visual complexity of short video content and the user's cognitive tolerance range are jointly modeled to achieve dynamic matching between the video and the user's state. Second, a dynamic interest modeling module is used to obtain dynamic interest vectors that reflect the evolution of user interests, and these vectors are deeply fused with the semantic embedding vectors of candidate short videos. Finally, a visual load matching index is introduced as a constraint in the multi-objective recommendation decision module, and a multi-objective optimization algorithm is used to weigh the recommendation probability and semantic matching degree to generate a recommendation list that balances accuracy and comfort.

[0022] Please see Figure 1This invention provides a technical solution: a short video content accurate recommendation system based on artificial intelligence image recognition, comprising the following modules: a visual load perception module, used to extract spatiotemporal features of key frames of short videos through a multi-scale convolutional neural network to generate a visual complexity vector, and to construct a user cognitive load threshold curve by collecting user interaction micro-gesture feature sequences to characterize the user's tolerance range for visual complexity at different time periods; and a cognitive matching module, used to match the visual complexity vector of candidate short videos with the user's cognitive load threshold curve for the current time period through an adaptive dynamic warping algorithm to obtain a visual load matching index, and to introduce a load penalty factor when the visual load matching index exceeds a preset threshold range. The system adjusts the ranking values ​​of candidate short videos to generate initial recommendation probabilities. A dynamic interest modeling module, based on a gated recurrent unit network and attention mechanism, performs temporal modeling of user interaction micro-gesture feature sequences. This is achieved by resetting the gate to filter noisy interactions in historical sequences, updating the gate to strengthen recent relevant interest representations, and weighting historical behavior with attention weights, outputting a user dynamic interest vector. A multi-objective recommendation decision module performs multi-dimensional feature fusion analysis on the user dynamic interest vector and the semantic embedding vector of candidate short videos to obtain semantic matching degree. A visual load matching index is introduced as a constraint, and a constrained multi-objective optimization algorithm is used to weight the initial recommendation probability and semantic matching degree of candidate short videos, outputting a recommendation list.

[0023] In this implementation, a visual load perception module extracts spatiotemporal features of keyframes from short videos using a multi-scale convolutional neural network (CNN, a commonly used deep learning image feature extraction method). Based on these features, a visual complexity vector is generated, representing dimensions such as information density, color change rate, and number of objects in the video frame, reflecting the video's visual complexity. Simultaneously, the system infers the user's cognitive load state by collecting user interaction micro-gesture feature sequences (e.g., swipe pauses, slight clicks, fingertip drags, etc.) and constructs a user cognitive load threshold curve. This curve describes the user's tolerance range for video visual complexity at different time periods. This enables dynamic perception of user visual fatigue and cognitive comfort. A cognitive matching module uses an adaptive dynamic warping algorithm (an improved version of Dynamic Time Warping, used to compare the similarity of two imperfectly aligned time series) to match the visual complexity vector of candidate short videos with the user's cognitive load threshold curve, obtaining a visual load matching index. This index measures the degree of fit between the candidate video and the user's current cognitive state. When the index exceeds a preset reasonable range, the system introduces a load penalty factor, which adds a negative correction term to the video ranking score to reduce the priority of overly complex or monotonous videos in the recommendation list, thereby generating an initial recommendation probability that better suits the user's visual tolerance. The dynamic interest modeling module, based on a Gated Recurrent Unit (GRU, a recurrent neural network structure for processing time-series data) combined with an attention mechanism, performs time-series modeling of the user's interactive micro-gesture feature sequences. It filters noisy interaction data irrelevant to the current interest in the historical sequence through a reset gate, strengthens recent interest signals related to candidate videos through an update gate, and assigns differentiated weights to different historical interaction behaviors through attention weights, thus obtaining the user's dynamic interest vector. This vector not only reflects the user's long-term preferences but also emphasizes recent changes in the user's interests, improving the real-time performance and accuracy of recommendations. The multi-objective recommendation decision module fuses the user's dynamic interest vector with the semantic embedding vector of candidate short videos (a numerical representation of video content semantic information through a deep learning model) to obtain a semantic matching degree. Based on this, the aforementioned visual load matching index is introduced as a constraint. A constrained multi-objective optimization algorithm (i.e., an algorithm that simultaneously optimizes recommendation probability and semantic matching degree and is limited by the user's cognitive load threshold) is used to make a weighted decision on the initial recommendation probability and semantic matching degree of candidate videos, and finally outputs a recommendation list that meets the user's interests and cognitive comfort.By having the above modules work together, this invention can not only ensure that the recommendation results are highly relevant to the user's interests, but also effectively avoid viewing fatigue or boredom caused by excessively high or low video complexity, thereby improving the user experience and the overall performance of the recommendation system.

[0024] Specifically, the process of extracting spatiotemporal features of keyframes from short videos and generating a visual complexity vector using a multi-scale convolutional neural network is as follows: Keyframe sampling is performed on the input short video to extract spatiotemporal features of the keyframes, including scene complexity features, motion intensity features, color contrast features, and texture density features; global scene layout features are extracted through the large-scale branch of the multi-scale convolutional neural network, body motion trajectory features are extracted through the medium-scale branch, and image detail texture features are extracted through the small-scale branch; the features extracted at each scale are fused across levels, and a visual complexity score is calculated through a feature weighted aggregation algorithm to generate a multi-dimensional vector representing the visual complexity of the video.

[0025] In this implementation scheme, firstly, keyframe sampling is performed on the input short video to obtain a frame sequence. ,in This represents the number of keyframes obtained through sampling. Feature extraction is performed on each frame, yielding four types of features: scene complexity, motion intensity, color contrast, and texture density. Further, a multi-scale convolutional neural network is constructed, comprising three branches: a large-scale branch for extracting global scene layout features; a medium-scale branch for extracting object motion trajectory features; and a small-scale branch for extracting image detail texture features. Let the scale be... Next frame The extracted feature vector is Then the overall feature set is represented as: ;in, This represents the global scene features extracted from large-scale branches; This represents the motion trajectory features extracted by the mesoscale branch; This represents the texture detail features extracted from the small-scale branch. Subsequently, the features at different scales are fused across layers. A feature-weighted aggregation algorithm is used to calculate the overall visual complexity score. ;in: Keyframe Visual complexity score; This represents a nonlinear transformation function used to map features at various scales to a unified scoring space. This represents the weighting coefficients for each scale. The weighting coefficients are determined by performing correlation analysis on the user subjective ratings of a large number of video samples in the training set and the features extracted by the model. A minimum mean square error optimization method is then used to minimize the error between the predicted complexity score and the average user subjective rating, thus obtaining the optimal weighting coefficients. After obtaining the complexity scores for each keyframe, the visual complexity vector of the entire short video is calculated. ;in, A multidimensional vector representing the visual complexity of a short video; This represents the complexity score of each keyframe arranged in chronological order, which can comprehensively reflect the changes in the complexity of the video in the spatiotemporal dimensions.

[0026] Specifically, the process of constructing a user cognitive load threshold curve by collecting user interaction micro-gesture feature sequences is as follows: Continuously monitor the user's micro-gesture interaction data during browsing, including the acceleration change characteristics of touch screen swipe trajectory, the distribution characteristics of page dwell time, pupil focusing trajectory characteristics, and touch screen pressure characteristics; perform time series segmentation and standardization on the multimodal interaction features, extract the time domain and frequency domain features representing cognitive load, establish a mapping relationship from micro-gesture features to cognitive load level, and construct a threshold curve reflecting the user's cognitive load change pattern in different time periods through time series analysis algorithms. This curve dynamically represents the user's real-time tolerance range for visual complexity.

[0027] In this implementation plan, specifically, the process of constructing a user cognitive load threshold curve by collecting user interaction micro-gesture feature sequences is as follows: First, continuously monitor the micro-gesture interaction data of users during short video browsing. The data includes: acceleration change characteristics of touch screen swipe trajectories; distribution characteristics of page dwell time; pupil focusing trajectory characteristics; and touch screen pressure characteristics. The above multimodal interaction features are then subjected to time series segmentation and standardization to obtain the feature sequence: ;in: Indicates at time The characteristic value of sliding acceleration; Indicates at time Page dwell time characteristic value; Indicates at time The characteristic value of the pupil focusing trajectory; Indicates at time The touchscreen pressure feature value was then obtained. Subsequently, based on the aforementioned multimodal features, time-domain and frequency-domain features were extracted to obtain a feature vector representing the cognitive load level. ;in: Indicates at time Cognitive load score; This represents the time-frequency domain feature extraction function, including statistics such as mean, variance, and frequency-energy ratio. The weighting coefficients represent the multimodal features. The weighting coefficients are determined by performing regression fitting between the actual user interaction data and the subjective cognitive load annotation results in the training set, and then estimating the coefficient values ​​using the least squares method. This ensures that the error between the cognitive load score output by the model and the actual cognitive load level is minimized. (This is done after obtaining the sequence...) Then, time series smoothing analysis methods (such as exponentially weighted moving average) were used to construct the national value curve of user cognitive load: ;in: Indicates at time Cognitive load threshold; This represents the smoothing factor, and its range is determined by experimental data. This represents the threshold result from the previous time step. It is obtained through the above process. The curve can dynamically reflect the user's real-time tolerance range for visual complexity at different time periods, providing constraints for subsequent visual load matching.

[0028] Specifically, the process of matching the visual complexity vector of candidate short videos with the cognitive load threshold curve of the user's current time period using an adaptive dynamic warping algorithm to obtain the visual load matching index is as follows: Calculate the feature distance matrix between the visual complexity vector of the candidate short videos and the user's cognitive load threshold curve at each time point; find the optimal matching path between the two sequences using a dynamic time warping algorithm, where the warping window size is adaptively adjusted according to the degree of curve fluctuation; calculate the matching degree score based on the cumulative distance of the optimal matching path, and normalize it by combining the path length and curvature to generate a visual load matching index that represents the degree of matching between the video content and the user's cognitive state.

[0029] In this implementation scheme, firstly, let the visual complexity vector of the candidate short video on the keyframe sequence be: ;in: Indicates the first The visual complexity value corresponding to each keyframe; This indicates the number of keyframes sampled in the candidate short video. Meanwhile, the user's cognitive load threshold curve for the current time period is as follows: ;in: Indicates the first Cognitive load threshold at each time point; This represents the number of sampling points on the threshold curve. Next, the feature distance matrix between the candidate video complexity sequence and the user threshold curve is constructed: ;in: Indicates the candidate video number Frame and user in the first The difference in threshold values ​​at each time point is then used. Based on this, the Dynamic Time Warping (DTW) method is employed to calculate the optimal matching path between the two sequences. The recursive formula is as follows: ;in: Indicates the position matched in the sequence. Minimum cumulative distance at time; initial conditions are To avoid distortion caused by excessively curved paths, a regularized window size ω is set and adaptively adjusted based on the degree of curve undulation. Where: ω represents the dynamically adjusted window size; γ represents the adjustment coefficient; This represents the standard deviation of the threshold curve. When the threshold curve fluctuates significantly, the window range automatically expands to capture more non-linear matching relationships. The final matching path is P, and its cumulative distance is M(m,n). To eliminate the influence of path length and curvature, the matching results are normalized to obtain the visual load matching index: Where: Ω represents the visual load matching index, and the closer the value is to 1, the better the video complexity matches the user's cognitive threshold; Indicates the length of the optimal matching path; The visual load matching index represents the curvature of the matching path; κ represents the curvature penalty coefficient, which is determined by selecting the optimal value that can stably maintain recommendation accuracy across different user groups through cross-validation experiments. Through the above steps, the obtained visual load matching index can accurately characterize the degree of fit between video content and the user's current cognitive state, providing a precise basis for the selection and ranking of candidate videos.

[0030] Specifically, when the visual load matching index exceeds a preset threshold, a load penalty factor is introduced to adjust the ranking value of candidate short videos. The specific process for generating the initial recommendation probability of candidate short videos is as follows: Based on the degree of deviation between the visual load matching index and the preset threshold, the load penalty factor value is calculated through a nonlinear function. The load penalty factor is then weighted and adjusted with the original ranking value, where the penalty factor weight is dynamically adjusted according to the user's current cognitive load level. The adjusted ranking value is mapped to the probability space through the Sigmoid function to generate the initial recommendation probability of candidate short videos. This probability value reflects the degree of matching between the video content and the user's cognitive tolerance.

[0031] In this implementation scheme, firstly, the visual load matching index is: The system's preset visual load threshold is: ;in: This indicates the degree of match between the video content and the user's cognitive threshold, with a value range of [value range missing]. A larger value indicates a better fit; This represents the upper limit of the threshold that users can accept cognitively. It is determined by: statistically analyzing historical user experiment data to calculate the average upper bound of user tolerance when watching videos of different complexities, and then obtaining a stable value using the quantile method. Then, calculate the degree of deviation between the two: ;in: This represents the difference between the matching index and the threshold; a larger value indicates a heavier cognitive burden on the user from the video. Based on the degree of deviation, a non-linear function is introduced to calculate the load penalty factor: ;in: This represents the load penalty factor, with a value range of... ; The adjustment coefficient is determined by selecting the optimal value among multiple user experimental data sets that effectively distinguishes between mild and severe video recommendation loads. Then, the penalty factor is weighted and adjusted based on the original ranking values ​​of the candidate videos. ;in: This represents the original ranking value of the candidate short videos; This indicates the adjusted sort value; The penalty weight coefficient is determined by dynamically adjusting it based on the user's real-time cognitive load level. The value increases when the user's interaction shows signs of fatigue and decreases when the user is energetic. Finally, the adjusted ranking values ​​are mapped to the probability space using the Sigmoid function to obtain the initial recommendation probability of the candidate short videos. ;in: This represents the initial recommendation probability of the candidate short video; the closer the value is to 1, the more likely the video is to be recommended. The scaling factor is determined by selecting the optimal value on the training set that best balances the distribution of prediction results through grid search or cross-validation. Through this process, the system can finely control the recommendation results while considering the user's cognitive capacity. This ensures that the recommendation probability depends not only on the semantic relevance of the video but also reflects the user's real-time cognitive burden, thereby avoiding the recommendation of overly complex or stimulating content.

[0032] Specifically, the logical process of performing temporal modeling on user interaction micro-gesture feature sequences based on gated recurrent unit networks (GRN networks) combined with an attention mechanism is as follows: The user interaction micro-gesture feature sequences are input into the GRN network, and temporal dependencies in the sequence are captured through progressive calculations at time steps; an attention mechanism is introduced based on the hidden layer output of the GRN network to calculate the attention weight of the output at each time step; the hidden states at each time step are weighted and summed according to the attention weights to generate a latent vector representation representing the user's dynamic interests; the latent vector representation is fused with the input features at the current moment to output the temporal modeling result of the user's dynamic interests.

[0033] In this implementation scheme, specifically, the logical process of temporal modeling of user interaction micro-gesture feature sequences based on Gated Recurrent Unit (GRU) network combined with an attention mechanism is as follows: First, the micro-gesture feature sequences generated by the user during short video browsing are input into the GRU network step by step. The network captures the temporal dependencies in the sequence through progressive calculation, enabling historical behavior information to effectively influence the interest modeling at the current moment. During this process, the GRU selectively retains historical states related to the current interest and filters out irrelevant or noisy interactions through its internal reset and update gate mechanisms, thereby ensuring that the dynamic interest vector can truly reflect the user's attention trend. Subsequently, an attention mechanism is introduced based on the output of the hidden layer of the GRU network to perform weighted evaluation of the hidden states at different time steps, calculating the contribution of each time step to the overall interest expression, enabling the model to automatically focus on recent or key user interaction behaviors. By weighted summing of the hidden states at each time step according to the attention weights, a latent vector representation that comprehensively represents the user's dynamic interests is generated. This vector retains both the long-term trend of the user's historical behavior and reflects the interest preferences in the current time period. Finally, this latent vector representation is fused with the input features at the current moment to form a complete dynamic user interest representation, which serves as the input for the subsequent recommendation decision module, thereby achieving accurate recommendations for candidate short videos. The entire process logically implements a continuous processing chain from historical behavior filtering and temporal dependency modeling to key behavior weighting and dynamic interest output, making user interest modeling both detailed and dynamically responsive to real-time changes in user behavior.

[0034] Specifically, the process of outputting the user's dynamic interest vector by filtering noisy interactions in the historical sequence through the reset gate, strengthening the representation of recent relevant interests through the update gate, and weighting historical behavior with attention weights is as follows: The reset gate calculates the retention ratio of historical hidden states to filter out noisy interaction information that is irrelevant to the current interest; the update gate controls the fusion ratio of the current input information and historical information to strengthen the representation of recent relevant interests; the attention mechanism calculates the importance weight of each time step in the historical sequence and aggregates the weighted hidden states; the feature vectors processed by the gating mechanism and attention weighting are concatenated and linearly transformed to generate the user's dynamic interest vector.

[0035] In this implementation scheme, the input sequence and hidden state initialization are set as follows: The user interaction micro-gesture feature sequence is: ;in: :No. The input feature vector for each time step includes touch screen swipe, dwell time, and pupil trajectory features; The total length of the user interaction sequence. Hidden state is initialized as follows: ;in: The initial hidden state is set to zero vector. The reset gate filters noise interactions at each time step; the calculation formula for the reset gate is: ;in: : Reset the gate vector, the numerical range is , indicates the percentage of historical information retained; : Input features into the weight matrix of the reset gate; parameter initialization can be performed using the Xavier method. The weight matrix from the previous hidden state to the reset gate; :Reset the gate's bias vector; The Sigmoid activation function. Its function is to activate the Sigmoid function if... If the value is small, the current step will filter out more historical information, thus filtering out noisy interactions irrelevant to the user's interests. The formula for calculating the recent interest update gate is: ;in: Update the gate vector, with values ​​ranging from 1 to 2. , indicates the ratio of current input to historical information; The weight matrix from the input features to the update gate; The weight matrix from the previous hidden state to the update gate; Update the gate bias vector. The candidate hidden states are: ;in: Candidate hidden state; The weight matrix from input features to candidate hidden states; The weight matrix from historical states to candidate hidden states; : The bias vector of the candidate hidden state; Element-wise multiplication of vectors. The final hidden state update formula is: ;in: :No. The hidden states at each time step are calculated, taking into account both historical interests and current input features. An attention mechanism is used to compute weights to obtain the hidden state sequence. Then, an attention mechanism is introduced: ;in: Attention weight at time step m, reflecting the contribution to user interest at that moment; The weight matrix for attention calculation; The bias vector used for attention calculation; The context weight vector is determined by automatically learning it during training by minimizing the recommendation error. The aggregated interest representation is as follows: ;in: The attention-weighted aggregation vector represents the weighted interest information of all historical behaviors. The dynamic interest vector output concatenates the final hidden state with the attention aggregation vector and performs a linear transformation. ;in: User dynamic interest vectors; The result of concatenating the interest aggregation vector with the final hidden state; The weight matrix of the linear transformation; : Bias vector of linear transformation.

[0036] Specifically, the process of performing multi-dimensional feature fusion analysis on user dynamic interest vectors and candidate short video semantic embedding vectors to obtain semantic matching degree is as follows: The interaction weights between user dynamic interest vectors and candidate short video semantic embedding vectors are calculated using a cross-modal attention mechanism to establish a mapping between user interests and video content; the two vectors are fused using a combination of feature concatenation and element-wise multiplication to capture the linear and non-linear relationships between user interests and video content; a deep non-linear transformation is performed on the fused feature vectors using a multilayer perceptron to extract high-order cross features; and the high-order cross features are mapped to a semantic matching degree score through a fully connected layer, which represents the degree of correlation between user interests and video content.

[0037] In this implementation scheme, the cross-modal attention calculation interaction weights are used to calculate the user's dynamic interest vector through a cross-modal attention mechanism. With candidate short video semantic embedding vectors The interaction weights between these factors establish a mapping between user interests and video content. The formula is as follows: Parameter description: User dynamic interest vectors are output by the gated recurrent unit network of the previous module combined with the attention mechanism. : Candidate short video semantic embedding vectors, generated by the video content feature encoder; Attention weight matrix, which can be randomly initialized initially and adaptively updated during training; The interaction weight matrix, normalized using softmax, represents the correlation strength between the user interest vector and each dimension of the video semantic vector. The key step here is to quantify the correlation mapping between user interests and video features through an attention mechanism, providing a weight reference for subsequent feature fusion. Feature fusion combines the user interest vector and the video semantic vector, using a combination of feature concatenation and element-wise multiplication to capture both linear and non-linear relationships. The formula is as follows: Parameter description: : The fused feature vector; Vector concatenation operation, which... and Connect along the feature dimension; Element-wise multiplication represents multiplying corresponding dimensions to capture nonlinear interaction features; Note: Fusion operations can simultaneously preserve both original feature information and interaction information, improving feature representation capabilities. Higher-order nonlinear feature transformations are applied to the fused feature vector using a multilayer perceptron (MLP). A deep nonlinear transformation is performed to extract high-order cross features. The formula is as follows: Parameter description: : Higher-order cross feature vectors; : No. Layer activation functions (such as ReLU) are used to introduce nonlinearity; : No. The layer weight matrix is ​​initially randomly generated and optimized through training. : No. Layer bias vector; The number of layers in the multilayer perceptron is determined experimentally, typically 2-4 layers. Explanation: Through multilayer nonlinear transformations, the model can capture complex high-order relationships between user interests and video semantics. Mapping to semantic matching scores will convert the high-order feature vectors... Input to a fully connected layer to generate the final semantic matching score. This represents the relevance between user interests and video content. The formula is as follows: Parameter description: Semantic matching score, typically ranging from 0 to 1; The sigmoid activation function is used to map to a probability space. : Weight matrix of fully connected layer; Bias terms for fully connected layers; methods for determining weights and thresholds: The weight matrix is ​​initially randomly generated and iteratively optimized using backpropagation and training data; the bias... It starts at zero and can be adjusted through training; the Sigmoid output naturally maps to 0~1.

[0038] Specifically, a visual load matching index is introduced as a constraint. A constrained multi-objective optimization algorithm is used to make a weighted decision on the initial recommendation probability and semantic matching degree of candidate short videos, and the specific process of outputting the recommendation list is as follows: A bi-objective optimization function is established with the goal of maximizing semantic matching degree and minimizing visual load. The visual load matching index is used as a hard constraint. The objective function is solved based on the Pareto optimal multi-objective optimization algorithm to generate a candidate solution set. In the candidate solution set, the multi-objective is transformed into a single objective according to the weighted summation strategy. The initial recommendation probability and semantic matching degree are weighted and fused to determine the final recommendation score. The recommendation list is then output in descending order of score.

[0039] In this implementation plan, the step of establishing a bi-objective optimization function is to maximize the semantic matching degree. and minimize visual load To optimize the objective, the visual load matching index is used as a hard constraint to generate a candidate solution set. Formula: Parameter description: : Recommendation decision vector for candidate short videos, representing the recommendation probability allocation for each candidate video; The semantic matching degree function maps the semantic matching degree score calculated in the previous step to the recommendation probability space. Visual load function, used to quantify the impact of the visual complexity of video content on the user's cognitive load; The visual load threshold is dynamically determined by the user's cognitive load level (e.g., based on moving averages, extreme values, or percentile statistical methods). Note: The core of this step is to simultaneously incorporate "content relevance" and "visual load" into the decision-making process, ensuring that the recommended list aligns with the user's interests without overwhelming their senses. The Pareto optimal solution candidate set is generated by solving the objective function using the Pareto multi-objective optimization algorithm. Each solution represents a balance between the two objectives. Formula: Parameter description: : Candidate solution set, containing recommendation vectors that satisfy the Pareto optimality condition; ParetoOpt: Pareto optimality function, used to select the optimal solution from the candidate vectors. Maximize and Minimize the vector that cannot be dominated by other solutions; The set of all possible recommendation decision vectors; Note: This step ensures that the generated candidate solutions achieve an optimal balance between "interest matching degree" and "visual load". Multi-objective weighted summation transforms into a single objective in the candidate solution set, converting the dual-objective approach into a single objective, combined with the initial recommendation probability. and semantic matching degree Weighted fusion is performed to determine the final recommendation score. . formula: Parameter description: : Final recommendation score, used to sort and output the recommendation list; The initial recommendation probability of candidate short videos is generated from the ranking value adjusted for visual load. Semantic matching score, generated by the fusion of multidimensional features from the previous step; Visual load matching index; : Weighting coefficients, representing the contributions of initial probability, semantic matching degree, and visual load to the final recommendation score; the optimal ratio can be determined by optimizing recommendation accuracy and user load satisfaction on the training set through grid search or cross-validation; Note: Weighted fusion can integrate two objectives into a single score while preserving the constraint on user visual load. Output recommendation list based on the final recommendation score. The candidate short videos are sorted in descending order to generate the final recommendation list. Formula: Parameter description: The final sorted list of short video recommendations; : No. The final recommendation score for each candidate video; Total number of candidate short videos; Note: The sorted list takes into account user interests, video content relevance, and visual load constraints to ensure that the recommendation results are scientific and reasonable.

[0040] In summary, this application has at least the following effects:

[0041] This AI-based image recognition-based short video content recommendation system extracts spatiotemporal features from keyframes of short videos using a multi-scale convolutional neural network and constructs a cognitive load threshold curve by combining user interaction micro-gesture features. This enables accurate modeling of users' visual tolerance range at different time periods, effectively avoiding visual overload or insufficient interest matching caused by a single recommendation strategy. The system uses a gated recurrent unit network combined with an attention mechanism to perform temporal modeling of user interaction micro-gesture sequences. By resetting the gate to filter noisy interactions, updating the gate to strengthen recently relevant interests, and weighting historical behaviors with attention, it outputs a dynamic user interest vector. This allows the system to reflect changes in user interests in real time, improving the relevance and real-time performance of recommended content. Finally, the system fuses the dynamic user interest vector with the semantic embedding vector of candidate short videos through a cross-modal attention mechanism, feature concatenation, and a multilayer perceptron to extract high-order cross features and generate a semantic matching score. This enhances the relevance between user interests and video content, improving the accuracy and personalization of recommendations. By introducing a visual load matching index as a constraint and using a constrained multi-objective optimization algorithm to make weighted decisions on the initial recommendation probability and semantic matching degree of candidate short videos, a balance between maximizing interest matching degree and minimizing visual load is achieved. This ensures that the final recommendation list not only meets user interests but also guarantees visual comfort, thereby improving the overall user experience satisfaction.

[0042] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0043] This invention is described with reference to flowchart illustrations and / or block diagrams of systems, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0044] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0045] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0046] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0047] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A short video content accurate recommendation system based on artificial intelligence image recognition, characterized in that, Includes the following modules: The visual load perception module is used to extract the spatiotemporal features of key frames in short videos through multi-scale convolutional neural networks, generate a visual complexity vector, and construct a user cognitive load threshold curve by collecting user interaction micro-gesture feature sequences, which is used to characterize the user's tolerance range for visual complexity at different time periods. The cognitive matching module is used to match the visual complexity vector of candidate short videos with the cognitive load threshold curve of the user's current time period through an adaptive dynamic warping algorithm to obtain a visual load matching index. When the visual load matching index exceeds the preset threshold range, a load penalty factor is introduced to adjust the ranking value of candidate short videos and generate the initial recommendation probability of candidate short videos. The dynamic interest modeling module is used to perform temporal modeling of user interaction micro-gesture feature sequences based on gated recurrent unit network and attention mechanism. It outputs user dynamic interest vector by resetting the gate to filter noisy interactions in the historical sequence, updating the gate to strengthen recent relevant interest representations, and weighting historical behavior with attention weights. The multi-objective recommendation decision module is used to perform multi-dimensional feature fusion analysis on user dynamic interest vectors and candidate short video semantic embedding vectors to obtain semantic matching degree. The visual load matching index is introduced as a constraint condition. The constrained multi-objective optimization algorithm is used to make a weighted decision on the initial recommendation probability and semantic matching degree of candidate short videos and output a recommendation list. The specific process of matching the visual complexity vector of candidate short videos with the cognitive load threshold curve of the user's current time period using an adaptive dynamic warping algorithm to obtain the visual load matching index is as follows: Calculate the feature distance matrix between the visual complexity vector of the candidate short video and the user cognitive load threshold curve at each time point; The optimal matching path between two sequences is found by using a dynamic time warping algorithm, where the size of the warping window is adaptively adjusted according to the degree of curve fluctuation. The matching score is calculated based on the cumulative distance of the optimal matching path, and then normalized by combining the path length and curvature to generate a visual load matching index that represents the degree of matching between video content and user cognitive state. When the visual load matching index exceeds a preset threshold, a load penalty factor is introduced to adjust the candidate short video ranking value. The specific process for generating the initial recommendation probability of the candidate short video is as follows: Based on the degree of deviation between the visual load matching index and the preset threshold, the load penalty factor value is calculated through a nonlinear function. The load penalty factor is then weighted and adjusted with the original ranking value. The weight of the load penalty factor is dynamically adjusted according to the user's current cognitive load level. The adjusted ranking values ​​are mapped to the probability space using the Sigmoid function to generate the initial recommendation probability of candidate short videos. The value of this probability reflects the degree of matching between the video content and the user's cognitive tolerance.

2. The short video content accurate recommendation system based on artificial intelligence image recognition according to claim 1, characterized in that: The specific process of extracting the spatiotemporal features of keyframes in short videos and generating visual complexity vectors using multi-scale convolutional neural networks is as follows: Keyframe sampling is performed on the input short video to extract the spatiotemporal features of the keyframes, including scene complexity features, motion intensity features, color contrast features, and texture density features. The large-scale branch of the multi-scale convolutional neural network extracts global scene layout features, the medium-scale branch extracts body motion trajectory features, and the small-scale branch extracts image detail texture features. Features extracted from each scale are fused across levels, and a visual complexity score is calculated using a feature weighted aggregation algorithm to generate a multidimensional vector representing the visual complexity of the video.

3. The short video content accurate recommendation system based on artificial intelligence image recognition according to claim 2, characterized in that: The specific process of constructing a user cognitive load threshold curve by collecting user interaction micro-gesture feature sequences is as follows: Continuously monitor users' micro-gesture interaction data during browsing, including the acceleration change characteristics of touch screen swipe trajectories, the distribution characteristics of page dwell time, pupil focusing trajectory characteristics, and touch screen pressure characteristics; Multimodal interaction features are segmented and standardized over time to extract time-domain and frequency-domain features that characterize cognitive load. A mapping relationship from micro-gesture features to cognitive load levels is established. A threshold curve reflecting the changes in cognitive load of users in different time periods is constructed through time-series analysis algorithms. This curve dynamically characterizes the real-time tolerance range of users to visual complexity.

4. The short video content accurate recommendation system based on artificial intelligence image recognition according to claim 1, characterized in that: The logical process of performing temporal modeling of user interaction micro-gesture feature sequences based on gated recurrent unit networks combined with attention mechanisms is as follows: The user interaction micro-gesture feature sequence is input into a gated recurrent unit network, and the temporal dependencies in the sequence are captured by progressive calculation of time steps. An attention mechanism is introduced based on the hidden layer output of the gated recurrent unit network to calculate the attention weight of the output at each time step; The hidden states at each time step are weighted and summed according to the attention weights to generate a latent vector representation that represents the user's dynamic interests. The latent vector representation is then fused with the input features at the current time step to output the temporal modeling results of the user's dynamic interests.

5. The short video content accurate recommendation system based on artificial intelligence image recognition according to claim 4, characterized in that: The specific process of outputting a user's dynamic interest vector by resetting the gate to filter noisy interactions in the historical sequence, updating the gate to strengthen recent relevant interest representations, and weighting historical behavior with attention weights is as follows: The retention rate of historical hidden states is calculated by resetting the gate, and noisy interaction information that is irrelevant to the current interest is filtered out. By updating the fusion ratio of current input information and historical information in the gate control, the representation of recent relevant interests is strengthened; The importance weights of each time step in the historical sequence are calculated using an attention mechanism, and the weighted hidden states are then aggregated. The feature vectors, which have undergone gating and attention weighting, are concatenated and linearly transformed to generate dynamic interest vectors for users.

6. The short video content accurate recommendation system based on artificial intelligence image recognition according to claim 1, characterized in that: The specific process of performing multi-dimensional feature fusion analysis between user dynamic interest vectors and candidate short video semantic embedding vectors to obtain semantic matching degree is as follows: By calculating the interaction weights between user dynamic interest vectors and candidate short video semantic embedding vectors through a cross-modal attention mechanism, a correlation mapping between user interests and video content is established. By combining feature concatenation and element-level multiplication, the user dynamic interest vector and the candidate short video semantic embedding vector are fused to capture the linear and non-linear relationship between user interests and video content. Then, a multilayer perceptron is used to perform a deep non-linear transformation on the fused feature vector to extract high-order cross features. The high-order cross features are mapped to semantic matching scores through fully connected layers, which represent the relevance between user interests and video content.

7. The short video content accurate recommendation system based on artificial intelligence image recognition according to claim 6, characterized in that: Introducing a visual load matching index as a constraint, a constrained multi-objective optimization algorithm is used to weight the initial recommendation probability and semantic matching degree of candidate short videos, and the specific process of outputting the recommendation list is as follows: A bi-objective optimization function is established with the goal of maximizing semantic matching degree and minimizing visual load. The visual load matching index is used as a hard constraint. The objective function is solved based on the Pareto optimal multi-objective optimization algorithm to generate a candidate solution set. In the candidate solution set, the multi-objective is transformed into a single objective according to the weighted summation strategy. The initial recommendation probability and semantic matching degree are weighted and fused to determine the final recommendation score, and the recommendation list is output in descending order of score.

Images