A user behavior prediction method and system for virtual reality technology

Abstract

The present invention discloses a user behavior prediction method and system for virtual reality technology, and relates to the field of computer vision technology. A user behavior prediction system for virtual reality technology includes: a prediction model construction module, a user behavior prediction module and a user behavior prediction feedback module. The present invention can effectively capture the general characteristics and patterns of user behavior by constructing an initial user behavior prediction model based on big data user dynamic videos, and further optimize the initial user behavior prediction model through the behavior data of the target user, so that the model can be closer to the behavior characteristics of the target user, thereby improving the accuracy of user behavior prediction; by collecting the dynamic video of the target user in a preset virtual reality scene and extracting the user behavior vector, the constructed target user behavior prediction model can be trained and predicted for the specific behavior pattern of the individual user, thereby realizing personalized services.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a user behavior prediction method and system for virtual reality technology. Background Art

[0002] Virtual reality technology has been greatly developed in recent years, which enables users to interact in a simulated three-dimensional environment. However, in order to provide a realistic interactive experience, it is necessary to be able to accurately predict the user's behavior.

[0003] By predicting user behavior, we can generate and render scenes that users may see in advance, thereby reducing latency and improving user experience. In addition, we can also provide personalized content and services by predicting user behavior. Existing user behavior prediction methods mainly predict users' future behavior by analyzing their historical behavior data, but the amount of data is too small to include all possible user behaviors. If users begin to show new behaviors that are inconsistent with historical data, this method may not be able to accurately predict. Predicting user behavior should be able to comprehensively utilize big data analysis, especially the rich information contained in dynamic video data, and use deep learning technology to build an initial model that can capture the common characteristics and patterns of user behavior to predict user behavior.

[0004] Therefore, we urgently need to provide a user behavior prediction method and system for virtual reality technology based on deep learning and multi-source feature fusion. Summary of the invention

[0005] The present invention aims to provide a user behavior prediction method and system for virtual reality technology to improve the accuracy of user behavior prediction.

[0006] A user behavior prediction method for virtual reality technology comprises the following steps:

[0007] S1. Collect dynamic videos of target users in N preset virtual reality scenes to obtain dynamic videos of target users V n , n=1, 2, ..., N; each target user dynamic video V n contains K preset frames, and the target user dynamic video V n The kth preset frame image in is recorded as the target user dynamic image P nk , k=1, 2, ..., K; the target user dynamic image P nk Input into the user behavior decomposition model to extract information and obtain the target user behavior vector A nk , based on the target user's dynamic video V n All target user behavior vectors A generated nkRecorded as a set of training samples, we get the target user behavior prediction training sample Y n ; Predict all target user behavior training samples Y n Combine and obtain a target user behavior prediction training set; build a user behavior prediction model based on big data user dynamic videos to obtain a user behavior prediction initial model; update the user behavior prediction initial model based on the target user behavior prediction training set to obtain a target user behavior prediction model;

[0008] S2. According to the preset frames, the dynamic images of the target user in the virtual reality scene are obtained in real time to obtain the dynamic images to be predicted; when K dynamic images to be predicted are obtained, the K dynamic images to be predicted are input into the target user behavior prediction model for prediction to obtain the predicted target user behavior; the virtual reality scene is preloaded based on the predicted target user behavior;

[0009] S3. Real-time acquisition of the next frame of the target user's dynamic image in the virtual reality scene to obtain the dynamic image to be evaluated; when M dynamic images to be evaluated are collected, the M dynamic images to be evaluated are divided into γ dynamic image sets O to be evaluated. θ ; θ=1, 2, …, γ; dynamic image set O to be evaluated θ contains K consecutive dynamic images to be evaluated, recording the dynamic image set O to be evaluated θ The actual target user behavior corresponding to the last dynamic image to be evaluated in is obtained. θ , the dynamic image set O to be evaluated θ and actual target user behavior F θ A user behavior prediction feedback set is obtained by combining; and parameters of a target user behavior prediction model are updated based on the user behavior prediction feedback set.

[0010] As a preferred technical solution of the present invention, the user behavior decomposition model includes an image processing layer, a position information extraction layer, an environmental information extraction layer, a body information extraction layer, an action information extraction layer, an emotion information extraction layer and a result output layer;

[0011] The image processing layer is used to transform the target user dynamic image P nk Perform preprocessing to obtain the preprocessed target user dynamic image C nk ;

[0012] The position information extraction layer is used to extract the position information of the target user according to the pre-processed dynamic image C nk Perform position tracking and obtain the position information vector L nk ;

[0013] The environmental information extraction layer is used to extract the dynamic image C of the target user according to the preprocessing nk Extract environmental information and obtain the environmental information vector Hnk ;

[0014] The body information extraction layer is used to extract the dynamic image C of the target user according to the preprocessing nk Perform skeleton fitting on the target user to obtain the body information vector S nk , S nk ={(α nk1 , β nk1 ),…,(α nkz , β nkz ),…,(α nkZ , β nkZ )} T , Z is the number of key points of the target user's head and limbs; (α nkz , β nkz ) represents the horizontal and vertical coordinates of the z-th key point of the target user fitted in the k-th frame;

[0015] The action information extraction layer is used to extract the target user dynamic image C according to the preprocessing nk Use the skeleton-based action recognition algorithm to perform action recognition and obtain the action information vector D nk ;

[0016] The emotion information extraction layer is used to extract the emotion information based on the preprocessed target user dynamic image C nk Perform emotion recognition and obtain the emotion information vector E nk ;

[0017] The resulting output layer is used to transform the position information vector L nk , environmental information vector H nk , body information vector S nk , action information vector D nk and the emotion information vector E nk Combine them to get the target user behavior vector A nk , A nk =(L nk , H nk , S nk , D nk , E nk ).

[0018] As a preferred technical solution of the present invention, the target user behavior prediction model includes a data preprocessing layer, a feature selection layer, a behavior prediction layer and a result output layer;

[0019] The data preprocessing layer is used to input K dynamic images to be predicted into the user behavior decomposition model for information extraction, and obtain K target user behavior vectors X to be predicted. k ;

[0020] The feature selection layer is used to select all target user behavior vectors X to be predicted.k Use decision trees to perform feature selection and obtain a set of prediction selection feature components;

[0021] The behavior prediction layer is used to perform feature fusion based on the prediction selection feature component set to obtain the predicted target user behavior;

[0022] The result output layer is used to output the predicted target user behavior.

[0023] As a preferred technical solution of the present invention, the specific steps of constructing a user behavior prediction model based on big data user dynamic videos in step S1 to obtain the user behavior prediction initial model include:

[0024] Collect dynamic videos of several big data users in a preset virtual reality scene to obtain a big data dynamic video set; divide each dynamic video in the big data dynamic video set into K frames according to preset frames, and combine them to obtain several dynamic image sets; send the several dynamic image sets one by one into the user behavior decomposition model for information extraction to obtain several big data user training samples, perform feature fusion processing on the several big data user training samples, and obtain several preset user behaviors; combine the several big data user training samples and the corresponding preset user behaviors to obtain the initial training set for user behavior prediction, and use the preset user behaviors corresponding to the big data user training samples as the target values ​​of a group of big data user training samples;

[0025] The CNN model is used as the basic model of the initial model for user behavior prediction. The initial training set for user behavior prediction is input into the CNN model to perform model training with the target value as the target to obtain the initial model for user behavior prediction. The initial model for user behavior prediction is evaluated to obtain the initial model evaluation result. If the evaluation result of the initial model is passed, the trained initial model for user behavior prediction is output; otherwise, the initial training set for user behavior prediction is used to continue model training.

[0026] As a preferred technical solution of the present invention, the specific steps of updating the initial user behavior prediction model based on the target user behavior prediction training set to obtain the target user behavior prediction model include:

[0027] For the target user behavior prediction training sample Y in the target user behavior prediction training set n Perform feature fusion to obtain the preset user behavior R n ; Predict the target user behavior training sample Y n and the corresponding preset user behavior R n Combine to get the updated target user behavior prediction training set; set the preset user behavior R n As the target user behavior prediction training sample Y in the target user behavior prediction training set, n The training target value of

[0028] The updated target user behavior prediction training set is input into the user behavior prediction initial model to update the model parameters with the training target value as the target, and a new user behavior prediction initial model is obtained; the new user behavior prediction initial model is evaluated to obtain an updated evaluation result; if the updated evaluation result is passed, the trained new user behavior prediction initial model is output, and the new user behavior prediction initial model is used as the target user behavior prediction model; otherwise, the user behavior prediction initial model is continued to be updated using the target user behavior prediction training set.

[0029] As a preferred technical solution of the present invention, the specific steps of updating the parameters of the target user behavior prediction model based on the user behavior prediction feedback set in step S3 include:

[0030] S31. The dynamic image set O to be evaluated in the user behavior prediction feedback set θ Input into the target user behavior prediction model for behavior prediction, and obtain the target user behavior U to be evaluated θ ;

[0031] S32. Calculate the target user behavior U to be evaluated θ and actual target user behavior F θ The Euclidean distance of θ ;

[0032] S33. Calculate all evaluation similarities J θ The average evaluation similarity is obtained by taking the mean of the average evaluation similarity; if the average evaluation similarity is within the preset similarity threshold, the target user behavior prediction model will continue to be used for behavior prediction; otherwise, the user behavior prediction feedback set will be used to update the parameters of the target user behavior prediction model.

[0033] As a preferred technical solution of the present invention, the specific steps of updating the parameters of the target user behavior prediction model using the user behavior prediction feedback set in step S33 include:

[0034] Construct I behavior prediction hyperparameter individuals Q i , i=1,2,…,I; each behavior prediction hyperparameter individual Q i contains a set of hyperparameters for adjusting the user behavior prediction model; i Combine to obtain the behavior prediction hyperparameter iteration population; set the maximum number of iterations;

[0035] Iterate the behavior prediction hyperparameters of individuals Q in the population according to the behavior prediction hyperparameters i The user behavior prediction feedback set is simulated and calculated, and the hyperparameter individual Q is predicted according to the behavior. iThe target user behavior prediction model is adjusted by adjusting the hyper parameters in to obtain a simulated target user behavior prediction model;

[0036] All the dynamic images to be evaluated in the user behavior prediction feedback set O θ Input into the simulated target user behavior prediction model for behavior prediction, and obtain γ simulated target user predicted behaviors; compare the γ simulated target user predicted behaviors with the corresponding actual target user behaviors F θ Perform similarity calculation to obtain γ simulated evaluation similarities; calculate the average of the γ simulated evaluation similarities to obtain the simulated average prediction accuracy; use the simulated average prediction accuracy as the behavior prediction hyperparameter individual Q i The fitness W i ;

[0037] When the maximum number of iterations is reached, the behavior prediction hyperparameter individual Q corresponding to the maximum fitness is output i , which is the optimal behavior prediction hyperparameter individual. The target user behavior prediction model is updated according to the adjusted hyperparameters in the optimal behavior prediction hyperparameter individual to obtain a new target user behavior prediction model.

[0038] A user behavior prediction system for virtual reality technology, comprising:

[0039] The prediction model building module is used to collect dynamic videos of target users in N preset virtual reality scenes and obtain the dynamic video V of the target user. n , n=1, 2, ..., N; each target user dynamic video V n contains K preset frames, and the target user dynamic video V n The kth preset frame image in is recorded as the target user dynamic image P nk , k=1, 2, ..., K; the target user dynamic image P nk Input into the user behavior decomposition model to extract information and obtain the target user behavior vector A nk , based on the target user's dynamic video V n All target user behavior vectors A generated nk Recorded as a set of training samples, we get the target user behavior prediction training sample Y n ; Predict all target user behavior training samples Y n Combine and obtain a target user behavior prediction training set; build a user behavior prediction model based on big data user dynamic videos to obtain a user behavior prediction initial model; update the user behavior prediction initial model based on the target user behavior prediction training set to obtain a target user behavior prediction model;

[0040] The user behavior prediction module is used to obtain the dynamic images of the target user in the virtual reality scene in real time according to the preset frames to obtain the dynamic images to be predicted; when K dynamic images to be predicted are obtained, the K dynamic images to be predicted are input into the target user behavior prediction model for prediction to obtain the predicted target user behavior; and the virtual reality scene is preloaded based on the predicted target user behavior;

[0041] The user behavior prediction feedback module is used to obtain the next frame of the target user's dynamic image in the virtual reality scene in real time to obtain the dynamic image to be evaluated; when M dynamic images to be evaluated are collected, the M dynamic images to be evaluated are divided into γ dynamic image sets O to be evaluated. θ ; θ=1, 2, …, γ; dynamic image set O to be evaluated θ contains K consecutive dynamic images to be evaluated, recording the dynamic image set O to be evaluated θ The actual target user behavior corresponding to the last dynamic image to be evaluated in the , and the actual target user behavior F θ , the dynamic image set O to be evaluated θ and actual target user behavior F θ A user behavior prediction feedback set is obtained by combining; and parameters of a target user behavior prediction model are updated based on the user behavior prediction feedback set.

[0042] The present invention has the following advantages:

[0043] 1. The present invention can effectively capture the general characteristics and patterns of user behavior by constructing an initial user behavior prediction model based on big data user dynamic videos. Further, the initial user behavior prediction model is optimized through the behavior data of the target user, so that the model can be closer to the behavior characteristics of the target user, thereby improving the accuracy of user behavior prediction; by collecting dynamic videos of the target user in a preset virtual reality scene and extracting user behavior vectors, the constructed target user behavior prediction model can be trained and predicted for the specific behavior patterns of individual users, thereby realizing personalized services; the target user behavior prediction model can make instant predictions based on dynamic images acquired in real time, which provides smooth and seamless interaction for the user experience in the virtual reality scene and reduces waiting and loading time; by collecting dynamic images to be evaluated in real time and forming a feedback set, the target user behavior prediction model can regularly update parameters, so that the prediction accuracy can be continuously optimized to adapt to changes in user behavior.

[0044] 2. The present invention decomposes user behavior through multiple levels such as image processing, location tracking, environmental analysis, body movement and emotion recognition, and combined with multi-dimensional information fusion, it can provide a more comprehensive and in-depth understanding of user behavior; by integrating information at different levels, the user behavior prediction model can more accurately predict the user's behavior and possible next action; the extracted body information vector and action information vector can be used to identify the target user's personal habits and behavior patterns, thereby providing more personalized prediction operations; the environmental information extraction layer and the location information extraction layer enable the model to understand the user's physical environment and context, which helps to make more appropriate predictions in specific situations; the addition of the emotional information extraction layer enables the user behavior prediction model to not only understand the user's physical movements, but also capture the user's emotional state, which is particularly important for using emotional information to assist in behavior prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the structure of a user behavior prediction system for virtual reality technology adopted in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to enable persons skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0047] Embodiment 1, a method for predicting user behavior for virtual reality technology, comprising the following steps:

[0048] S1. Collect dynamic videos of target users in N preset virtual reality scenes to obtain dynamic videos of target users V n , n=1, 2, ..., N; each target user dynamic video V n contains K preset frames, and the target user dynamic video V n The kth preset frame image in is recorded as the target user dynamic image P nk , k=1, 2, ..., K; the target user dynamic image P nk Input into the user behavior decomposition model to extract information and obtain the target user behavior vector A nk , based on the target user's dynamic video V n All target user behavior vectors A generated nk Recorded as a set of training samples, we get the target user behavior prediction training sample Y n ; Predict all target user behavior training samples Y nCombine and obtain a target user behavior prediction training set; build a user behavior prediction model based on big data user dynamic videos to obtain a user behavior prediction initial model; update the user behavior prediction initial model based on the target user behavior prediction training set to obtain a target user behavior prediction model;

[0049] In a preset virtual reality scene, the target user wears a virtual reality device and uses the virtual reality device to collect dynamic videos; the value of the preset frame is set by professional technicians according to the actual situation; the dynamic video of the target user is converted into a specific frame image through methods such as frame rate control and key frame extraction;

[0050] The user behavior decomposition model includes an image processing layer, a location information extraction layer, an environmental information extraction layer, a body information extraction layer, an action information extraction layer, an emotion information extraction layer, and a result output layer;

[0051] The image processing layer is used to transform the target user dynamic image P nk Perform preprocessing to obtain the preprocessed target user dynamic image C nk ; Various image preprocessing techniques, such as grayscale conversion, histogram equalization, noise removal, image enhancement, scale transformation, etc., can be used to improve image quality and prepare data for subsequent processing steps;

[0052] The position information extraction layer is used to extract the position information of the target user according to the pre-processed dynamic image C nk Perform position tracking and obtain the position information vector L nk ; Position tracking is usually achieved using target tracking algorithms such as Kalman filters, particle filters or more modern deep learning methods;

[0053] The environmental information extraction layer is used to extract the dynamic image C of the target user according to the preprocessing nk Extract environmental information and obtain the environmental information vector H nk Environmental information can be extracted through scene classification algorithms or object detection frameworks. These methods may include using CNN for scene recognition or using pre-trained models such as YOLO and Faster R-CNN to identify environmental elements in images.

[0054] The body information extraction layer is used to extract the dynamic image C of the target user according to the preprocessing nk Perform skeleton fitting on the target user to obtain the body information vector S nk , S nk ={(α nk1 , β nk1 ),…,(α nkz , β nkz ),…,(α nkZ , β nkZ )} T, Z is the number of key points of the target user's head and limbs; (α nkz , β nkz ) represents the horizontal and vertical coordinates of the zth key point of the target user fitted in the kth frame; skeleton fitting usually uses a pose estimation algorithm, such as OpenPose or AlphaPose, which can identify the key points of the human body and output coordinates;

[0055] The action information extraction layer is used to extract the target user dynamic image C according to the preprocessing nk Use the skeleton-based action recognition algorithm to perform action recognition and obtain the action information vector D nk ;Action recognition can be done using skeleton-based action recognition algorithms, which analyze the changes of body key points over time and recognize specific action patterns;

[0056] The emotion information extraction layer is used to extract the emotion information based on the preprocessed target user dynamic image C nk Perform emotion recognition and obtain the emotion information vector E nk Emotion recognition can be done through facial expression analysis, using methods such as Facial Action Coding System (FACS) or deep learning methods to identify and classify the user's emotional state;

[0057] The resulting output layer is used to transform the position information vector L nk , environmental information vector H nk , body information vector S nk , action information vector D nk and the emotion information vector E nk Combine them to get the target user behavior vector A nk , A nk =(L nk , H nk , S nk , D nk , E nk );

[0058] The target user behavior prediction model includes a data preprocessing layer, a feature selection layer, a behavior prediction layer, and a result output layer;

[0059] The data preprocessing layer is used to input K dynamic images to be predicted into the user behavior decomposition model for information extraction, and obtain K target user behavior vectors X to be predicted. k ; Preprocessing can improve the quality of data, reduce the impact of noise, and improve the efficiency of model training and the accuracy of prediction;

[0060] The feature selection layer is used to select all target user behavior vectors X to be predicted. kUse decision trees to perform feature selection to obtain a set of predictive selection feature components; decision trees, especially ensemble methods like random forests or gradient boosted decision trees, can be used to assess the importance of features and select the features that are most helpful for prediction; feature selection can reduce model complexity, increase training speed, reduce the risk of overfitting, and may improve the generalization ability of the model;

[0061] The behavior prediction layer is used to perform feature fusion based on the prediction selection feature component set to obtain the predicted target user behavior; the behavior prediction layer can provide behavior prediction based on historical data and real-time analysis;

[0062] The result output layer is used to output the predicted target user behavior;

[0063] The specific steps of building a user behavior prediction model based on big data user dynamic videos in step S1 to obtain the initial user behavior prediction model include:

[0064] Collect dynamic videos of several big data users in a preset virtual reality scene to obtain a big data dynamic video set; divide each dynamic video in the big data dynamic video set into K frames according to preset frames, and combine them to obtain several dynamic image sets; send the several dynamic image sets one by one into the user behavior decomposition model for information extraction to obtain several big data user training samples, perform feature fusion processing on the several big data user training samples, and obtain several preset user behaviors; combine the several big data user training samples and the corresponding preset user behaviors to obtain the initial training set for user behavior prediction, and use the preset user behaviors corresponding to the big data user training samples as the target values ​​of a group of big data user training samples;

[0065] The CNN model is used as the basic model of the initial user behavior prediction model, and the initial user behavior prediction training set is input into the CNN model to perform model training with the target value as the target, so as to obtain the initial user behavior prediction model; the initial user behavior prediction model is evaluated to obtain the initial model evaluation result; if the initial model evaluation result is passed, the trained initial user behavior prediction model is output; otherwise, the initial user behavior prediction training set is used to continue the model training;

[0066] Model evaluation can be performed by using cross-validation or calculating the model prediction accuracy and precision to obtain the initial model evaluation results; collecting a large amount of user dynamic video data can ensure that the model covers a variety of user behavior scenarios, which helps to improve the generalization ability of the model; segmenting the video into frame images allows the model to use time series information to more accurately capture the dynamic characteristics of user behavior; using the user behavior decomposition model to extract information from the dynamic image set one by one ensures that deep and discriminative features are extracted from the original data; fusing features from different sources can extract more powerful and representative action features and enhance the accuracy of the prediction model;

[0067] The specific steps of updating the initial user behavior prediction model based on the target user behavior prediction training set to obtain the target user behavior prediction model include:

[0068] For the target user behavior prediction training sample Y in the target user behavior prediction training set n Perform feature fusion to obtain the preset user behavior R n ; Predict the target user behavior training sample Y n and the corresponding preset user behavior R n Combine to get the updated target user behavior prediction training set; set the preset user behavior R n As the target user behavior prediction training sample Y in the target user behavior prediction training set, n The training target value of

[0069] Input the updated target user behavior prediction training set into the user behavior prediction initial model, update the model parameters with the training target value as the target, and obtain a new user behavior prediction initial model; perform model evaluation on the new user behavior prediction initial model to obtain an updated evaluation result; if the updated evaluation result is passed, output the trained new user behavior prediction initial model, and use the new user behavior prediction initial model as the target user behavior prediction model; otherwise, continue to update the user behavior prediction initial model using the target user behavior prediction training set;

[0070] The preset user behavior is obtained by feature fusion based on the target user behavior prediction training samples, such as action labels such as dodge and retreat. Through feature fusion processing, data features from different sources and types can be integrated to extract feature combinations that contribute more to predicting the target user behavior, thereby improving the accuracy of the prediction;

[0071] S2. According to the preset frames, the dynamic images of the target user in the virtual reality scene are obtained in real time to obtain the dynamic images to be predicted; when K dynamic images to be predicted are obtained, the K dynamic images to be predicted are input into the target user behavior prediction model for prediction to obtain the predicted target user behavior; the virtual reality scene is preloaded based on the predicted target user behavior; in the preset virtual reality scene, the available virtual reality scenes are labeled and classified, and indexes are established according to the content, type, theme and other attributes of the scene to facilitate matching according to the user behavior prediction results; based on the user behavior prediction results, a preloading strategy is formulated; this includes determining which scenes should be preloaded first and the timing of preloading; for example, preloading can be triggered when the user is about to make a certain action;

[0072] S3. Real-time acquisition of the next frame of the target user's dynamic image in the virtual reality scene to obtain the dynamic image to be evaluated; when M dynamic images to be evaluated are collected, the M dynamic images to be evaluated are divided into γ dynamic image sets O to be evaluated. θ ; θ=1, 2, …, γ; dynamic image set O to be evaluated θ contains K consecutive dynamic images to be evaluated, recording the dynamic image set O to be evaluated θ The actual target user behavior corresponding to the last dynamic image to be evaluated in is obtained. θ , the dynamic image set O to be evaluated θ and actual target user behavior F θ Combining to obtain a user behavior prediction feedback set; updating parameters of a target user behavior prediction model based on the user behavior prediction feedback set;

[0073] The specific steps of updating the parameters of the target user behavior prediction model based on the user behavior prediction feedback set in step S3 include:

[0074] S31. The dynamic image set O to be evaluated in the user behavior prediction feedback set θ Input into the target user behavior prediction model for behavior prediction, and obtain the target user behavior U to be evaluated θ ;

[0075] S32. Calculate the target user behavior U to be evaluated θ and actual target user behavior F θ The Euclidean distance of θ ;

[0076] S33. Calculate all evaluation similarities J θThe average evaluation similarity is obtained by taking the average evaluation similarity; if the average evaluation similarity is within the preset similarity threshold, the target user behavior prediction model is continued to be used for behavior prediction; otherwise, the user behavior prediction feedback set is used to update the parameters of the target user behavior prediction model;

[0077] By collecting users' dynamic images in real time and comparing them with the output of the target user behavior prediction model, the accuracy of the prediction can be continuously corrected and improved; the Euclidean distance between the actual behavior and the predicted behavior is calculated to evaluate the current effectiveness of the model, so that the model can adjust itself according to the latest behavior of the target user; when the accuracy of the prediction is not within the preset similarity threshold range, the feedback data set can be used to update the model. Such a dynamic update mechanism ensures that the model remains efficient and relevant over time; the preset similarity threshold range is set by professional technicians based on actual conditions; by making timely adjustments to the prediction model, unnecessary data processing and storage can be avoided, thereby saving computing resources and improving overall efficiency;

[0078] The specific steps of updating the parameters of the target user behavior prediction model using the user behavior prediction feedback set in step S33 include:

[0079] Construct I behavior prediction hyperparameter individuals Q i , i=1,2,…,I; each behavior prediction hyperparameter individual Q i contains a set of hyperparameters for adjusting the user behavior prediction model; i Combine to obtain the behavior prediction hyperparameter iteration population; set the maximum number of iterations; the maximum number of iterations is set by professional technicians based on actual conditions;

[0080] Iterate the behavior prediction hyperparameters of individuals Q in the population according to the behavior prediction hyperparameters i The user behavior prediction feedback set is simulated and calculated, and the hyperparameter individual Q is predicted according to the behavior. i The target user behavior prediction model is adjusted by adjusting the hyper parameters in to obtain a simulated target user behavior prediction model;

[0081] All the dynamic images to be evaluated in the user behavior prediction feedback set O θ Input into the simulated target user behavior prediction model for behavior prediction, and obtain γ simulated target user predicted behaviors; compare the γ simulated target user predicted behaviors with the corresponding actual target user behaviors F θ Perform similarity calculation to obtain γ simulated evaluation similarities; calculate the average of the γ simulated evaluation similarities to obtain the simulated average prediction accuracy; use the simulated average prediction accuracy as the behavior prediction hyperparameter individual Q i The fitness W i ;

[0082] The behavior prediction hyperparameter iteration population may also be updated to expand the number of the behavior prediction hyperparameter iteration population;

[0083] When the maximum number of iterations is reached, the behavior prediction hyperparameter individual Q corresponding to the maximum fitness is output i , which is the optimal behavior prediction hyperparameter individual, the target user behavior prediction model is updated according to the adjustment hyperparameters in the optimal behavior prediction hyperparameter individual to obtain a new target user behavior prediction model;

[0084] By constructing behavior prediction hyperparameter individuals and performing iterative simulation calculations, not only can the model be fine-tuned to improve prediction accuracy, but the optimal hyperparameter configuration can also be selected through fitness evaluation. This method increases the flexibility and adaptability of the model, enabling it to better reflect and adapt to the complexity and variability of user behavior; in addition, through iterative population update operations, the hyperparameter space can be further explored, thereby improving the generalization ability of the model. Ultimately, when the model is fully trained and optimized, it can more accurately predict user behavior and provide users with a more personalized and satisfying virtual reality experience.

[0085] Example 2, a user behavior prediction system for virtual reality technology, see Figure 1 As shown, including:

[0086] The prediction model building module is used to collect dynamic videos of target users in N preset virtual reality scenes and obtain the dynamic video V of the target user. n , n=1, 2, ..., N; each target user dynamic video V n contains K preset frames, and the target user dynamic video V n The kth preset frame image in is recorded as the target user dynamic image P nk , k=1, 2, ..., K; the target user dynamic image P nk Input into the user behavior decomposition model to extract information and obtain the target user behavior vector A nk , based on the target user's dynamic video V n All target user behavior vectors A generated nk Recorded as a set of training samples, we get the target user behavior prediction training sample Y n ; Predict all target user behavior training samples Y n Combine and obtain a target user behavior prediction training set; build a user behavior prediction model based on big data user dynamic videos to obtain a user behavior prediction initial model; update the user behavior prediction initial model based on the target user behavior prediction training set to obtain a target user behavior prediction model;

[0087] The user behavior prediction module is used to obtain the dynamic images of the target user in the virtual reality scene in real time according to the preset frames to obtain the dynamic images to be predicted; when K dynamic images to be predicted are obtained, the K dynamic images to be predicted are input into the target user behavior prediction model for prediction to obtain the predicted target user behavior; and the virtual reality scene is preloaded based on the predicted target user behavior;

[0088] The user behavior prediction feedback module is used to obtain the next frame of the target user's dynamic image in the virtual reality scene in real time to obtain the dynamic image to be evaluated; when M dynamic images to be evaluated are collected, the M dynamic images to be evaluated are divided into γ dynamic image sets O to be evaluated. θ ; θ=1, 2, …, γ; dynamic image set O to be evaluated θ contains K consecutive dynamic images to be evaluated, recording the dynamic image set O to be evaluated θ The actual target user behavior corresponding to the last dynamic image to be evaluated in the , and the actual target user behavior F θ , the dynamic image set O to be evaluated θ and actual target user behavior F θ A user behavior prediction feedback set is obtained by combining; and parameters of a target user behavior prediction model are updated based on the user behavior prediction feedback set.

[0089] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A user behavior prediction method for virtual reality technology, characterized in that: The following steps are involved: S1. Collect dynamic videos of target users in N preset virtual reality scenes to obtain dynamic videos of target users V n , n=1, 2, ..., N; each target user dynamic video V n contains K preset frames, and the target user dynamic video V n The kth preset frame image in is recorded as the target user dynamic image P nk , k=1,2,…,K; The target user dynamic image P nk Input into the user behavior decomposition model to extract information and obtain the target user behavior vector A nk , based on the target user's dynamic video V n All target user behavior vectors A generated nk Recorded as a set of training samples, we get the target user behavior prediction training sample Y n ; Predict all target user behavior training samples Y n Combine and obtain a target user behavior prediction training set; build a user behavior prediction model based on big data user dynamic videos to obtain a user behavior prediction initial model; update the user behavior prediction initial model based on the target user behavior prediction training set to obtain a target user behavior prediction model; S2. According to the preset frames, the dynamic images of the target user in the virtual reality scene are obtained in real time to obtain the dynamic images to be predicted; when K dynamic images to be predicted are obtained, the K dynamic images to be predicted are input into the target user behavior prediction model for prediction to obtain the predicted target user behavior; Preload VR scenes based on predicted target user behavior; S3. Real-time acquisition of the next frame of the target user's dynamic image in the virtual reality scene to obtain the dynamic image to be evaluated; when M dynamic images to be evaluated are collected, the M dynamic images to be evaluated are divided into γ dynamic image sets O to be evaluated. θ ; θ=1, 2, …, γ; dynamic image set O to be evaluated θ contains K consecutive dynamic images to be evaluated, recording the dynamic image set O to be evaluated θ The actual target user behavior corresponding to the last dynamic image to be evaluated in the , and the actual target user behavior F θ , the dynamic image set O to be evaluated θ and actual target user behavior F θ Combining to obtain a user behavior prediction feedback set; updating parameters of a target user behavior prediction model based on the user behavior prediction feedback set; The user behavior decomposition model includes an image processing layer, a location information extraction layer, an environmental information extraction layer, a body information extraction layer, an action information extraction layer, an emotion information extraction layer, and a result output layer; The image processing layer is used to transform the target user dynamic image P nk Perform preprocessing to obtain the preprocessed target user dynamic image C nk ; The position information extraction layer is used to extract the position information of the target user according to the pre-processed dynamic image C nk Perform position tracking and obtain the position information vector L nk ; The environmental information extraction layer is used to extract the dynamic image C of the target user according to the preprocessing nk Extract environmental information and obtain the environmental information vector H nk ; The body information extraction layer is used to extract the dynamic image C of the target user according to the preprocessing nk Perform skeleton fitting on the target user to obtain the body information vector S nk , S nk ={(α nk1 , β nk1 ),…,(α nkz , β nkz ),…,(α nkZ , β nkZ )} T , Z is the number of key points of the target user's head and limbs; (α nkz , β nkz ) represents the horizontal and vertical coordinates of the z-th key point of the target user fitted in the k-th frame; The action information extraction layer is used to extract the target user dynamic image C according to the preprocessing nk Use the skeleton-based action recognition algorithm to perform action recognition and obtain the action information vector D nk ; The emotion information extraction layer is used to extract the emotion information based on the preprocessed target user dynamic image C nk Perform emotion recognition and obtain the emotion information vector E nk ; The resulting output layer is used to transform the position information vector L nk , environmental information vector H nk , body information vector S nk , action information vector D nk and the emotion information vector E nk Combine them to get the target user behavior vector A nk , A nk =(L nk , H nk , S nk , D nk , E nk ).

2. A user behavior prediction method for virtual reality technology according to claim 1, characterized in that: The target user behavior prediction model includes a data preprocessing layer, a feature selection layer, a behavior prediction layer, and a result output layer; The data preprocessing layer is used to input K dynamic images to be predicted into the user behavior decomposition model for information extraction, and obtain K target user behavior vectors X to be predicted. k ; The feature selection layer is used to select all target user behavior vectors X to be predicted. k Use decision trees to perform feature selection and obtain a set of prediction selection feature components; The behavior prediction layer is used to perform feature fusion based on the prediction selection feature component set to obtain the predicted target user behavior; The result output layer is used to output the predicted target user behavior.

3. The user behavior prediction method for virtual reality technology according to claim 2, characterized in that: The specific steps of building a user behavior prediction model based on big data user dynamic videos in step S1 to obtain the initial user behavior prediction model include: Collect dynamic videos of several big data users in a preset virtual reality scene to obtain a big data dynamic video set; divide each dynamic video in the big data dynamic video set into K frames according to preset frames, and combine them to obtain several dynamic image sets; send the several dynamic image sets one by one into the user behavior decomposition model for information extraction to obtain several big data user training samples, perform feature fusion processing on the several big data user training samples, and obtain several preset user behaviors; combine the several big data user training samples and the corresponding preset user behaviors to obtain the initial training set for user behavior prediction, and use the preset user behaviors corresponding to the big data user training samples as the target values ​​of a group of big data user training samples; The CNN model is used as the basic model of the initial model for user behavior prediction. The initial training set for user behavior prediction is input into the CNN model to perform model training with the target value as the target to obtain the initial model for user behavior prediction. The initial model for user behavior prediction is evaluated to obtain the initial model evaluation result. If the evaluation result of the initial model is passed, the trained initial model for user behavior prediction is output; otherwise, the initial training set for user behavior prediction is used to continue model training.

4. The user behavior prediction method for virtual reality technology according to claim 3, characterized in that: The specific steps of updating the initial user behavior prediction model based on the target user behavior prediction training set to obtain the target user behavior prediction model include: For the target user behavior prediction training sample Y in the target user behavior prediction training set n Perform feature fusion to obtain the preset user behavior R n ; Predict the target user behavior training sample Y n and the corresponding preset user behavior R n Combine to get the updated target user behavior prediction training set; set the preset user behavior R n As the target user behavior prediction training sample Y in the target user behavior prediction training set, n The training target value of The updated target user behavior prediction training set is input into the user behavior prediction initial model to update the model parameters with the training target value as the target, and a new user behavior prediction initial model is obtained; the new user behavior prediction initial model is evaluated to obtain an updated evaluation result; if the updated evaluation result is passed, the trained new user behavior prediction initial model is output, and the new user behavior prediction initial model is used as the target user behavior prediction model; otherwise, the user behavior prediction initial model is continued to be updated using the target user behavior prediction training set.

5. The user behavior prediction method for virtual reality technology according to claim 4, characterized in that: The specific steps of updating the parameters of the target user behavior prediction model based on the user behavior prediction feedback set in step S3 include: S31. The dynamic image set O to be evaluated in the user behavior prediction feedback set θ Input into the target user behavior prediction model for behavior prediction, and obtain the target user behavior U to be evaluated θ ; S32. Calculate the target user behavior U to be evaluated θ and actual target user behavior F θ The Euclidean distance of θ ; S33. Calculate all evaluation similarities J θ The average evaluation similarity is obtained by taking the mean of the two sets of similarities; if the average evaluation similarity is within the preset similarity threshold, the target user behavior prediction model will continue to be used for behavior prediction; otherwise, the user behavior prediction feedback set will be used to update the parameters of the target user behavior prediction model.

6. The user behavior prediction method for virtual reality technology according to claim 5, characterized in that: The specific steps of updating the parameters of the target user behavior prediction model using the user behavior prediction feedback set in step S33 include: Construct I behavior prediction hyperparameter individuals Q i , i=1,2,…,I; each behavior prediction hyperparameter individual Q i contains a set of hyperparameters for adjusting the user behavior prediction model; i Combine to obtain the behavior prediction hyperparameter iteration population; set the maximum number of iterations; Iterate the behavior prediction hyperparameters of individuals Q in the population according to the behavior prediction hyperparameters i The user behavior prediction feedback set is simulated and calculated, and the hyperparameter individual Q is predicted according to the behavior. i The target user behavior prediction model is adjusted by adjusting the hyper parameters in to obtain a simulated target user behavior prediction model; All the dynamic images to be evaluated in the user behavior prediction feedback set O θ Input into the simulated target user behavior prediction model for behavior prediction, and obtain γ simulated target user predicted behaviors; compare the γ simulated target user predicted behaviors with the corresponding actual target user behaviors F θ Perform similarity calculation to obtain γ simulated evaluation similarities; calculate the average of the γ simulated evaluation similarities to obtain the simulated average prediction accuracy; use the simulated average prediction accuracy as the behavior prediction hyperparameter individual Q i The fitness W i ; When the maximum number of iterations is reached, the behavior prediction hyperparameter individual Q corresponding to the maximum fitness is output i , which is the optimal behavior prediction hyperparameter individual. The target user behavior prediction model is updated according to the adjusted hyperparameters in the optimal behavior prediction hyperparameter individual to obtain a new target user behavior prediction model.

7. A user behavior prediction system for virtual reality technology, characterized in that: The system applies a user behavior prediction method for virtual reality technology as described in any one of claims 1 to 6, including: The prediction model building module is used to collect dynamic videos of target users in N preset virtual reality scenes and obtain the dynamic video V of the target user. n , n=1, 2, ..., N; each target user dynamic video V n contains K preset frames, and the target user dynamic video V n The kth preset frame image in is recorded as the target user dynamic image P nk , k=1, 2, ..., K; the target user dynamic image P nk Input into the user behavior decomposition model to extract information and obtain the target user behavior vector A nk , based on the target user's dynamic video V n All target user behavior vectors A generated nk Recorded as a set of training samples, we get the target user behavior prediction training sample Y n ; Predict all target user behavior training samples Y n Combine and obtain a target user behavior prediction training set; build a user behavior prediction model based on big data user dynamic videos to obtain a user behavior prediction initial model; update the user behavior prediction initial model based on the target user behavior prediction training set to obtain a target user behavior prediction model; The user behavior prediction module is used to obtain the dynamic images of the target user in the virtual reality scene in real time according to the preset frames to obtain the dynamic images to be predicted; when K dynamic images to be predicted are obtained, the K dynamic images to be predicted are input into the target user behavior prediction model for prediction to obtain the predicted target user behavior; and the virtual reality scene is preloaded based on the predicted target user behavior; The user behavior prediction feedback module is used to obtain the next frame of the target user's dynamic image in the virtual reality scene in real time to obtain the dynamic image to be evaluated; when M dynamic images to be evaluated are collected, the M dynamic images to be evaluated are divided into γ dynamic image sets O to be evaluated. θ ; θ=1, 2, …, γ; dynamic image set O to be evaluated θ contains K consecutive dynamic images to be evaluated, recording the dynamic image set O to be evaluated θ The actual target user behavior corresponding to the last dynamic image to be evaluated in the , and the actual target user behavior F θ , the dynamic image set O to be evaluated θ and actual target user behavior F θ A user behavior prediction feedback set is obtained by combining; and parameters of a target user behavior prediction model are updated based on the user behavior prediction feedback set.

Images