A virtual reality game system and method based on visual tracking technology

Abstract

The present invention relates to the field of virtual reality game technology and discloses a virtual reality game system and method based on visual tracking technology, comprising a multidimensional acquisition module and a virtual reality module. The virtual reality game system and method based on visual tracking technology obtains the interactive data and motion performance of all users through the multidimensional acquisition module, and classifies and forms a data set. The virtual reality module constructs a facial model of each user, marks facial feature points and virtual lines, and then analyzes and generates emotional intensity. The virtual reality module constructs a limb model of each user, marks limb feature points and virtual lines, and then analyzes and generates perceptual intensity. The movement performance of each part is independently optimized, and the split modeling captures high precision. The virtual reality module forms a comprehensive model based on the facial model and limb model, determines the degree of delay when the user's action is substituted into the comprehensive model, promptly detects delays and lags in the game character's action, and intelligently optimizes the user experience for a better experience.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality game technology, and in particular to a virtual reality game system and method based on visual tracking technology. Background Art

[0002] Visual tracking technology is a key research area in computer vision. It aims to monitor the positional changes of objects in real time through video or image sequences and accurately predict their future positions. Visual tracking technology is widely used in a variety of fields, including autonomous driving, security surveillance, drone navigation, human-computer interaction, and virtual reality gaming. In virtual reality gaming, visual tracking accurately captures the player's movements, position, and perspective, enabling characters and environments in the virtual world to react in real time to the player's movements. Head tracking is a core application in virtual reality systems. Built-in sensors typically track the position and posture of the player's head in real time, enabling a 360-degree immersive experience. Head tracking not only enhances user immersion but also allows players to interact more naturally with the virtual environment, such as by turning their head to view the surrounding virtual scene. The player's hand movements are also crucial for interacting with the virtual world. Visual tracking technology uses hand tracking devices to capture the player's gestures, enabling simultaneous grabbing and moving objects in the virtual world, and even performing complex movements or operations. To enable richer movement performance in virtual reality games, many advanced VR systems employ full-body tracking technology. By installing multiple sensors, cameras or markers on the player, the player's whole body movement is captured and more realistic interactions are created, including walking, running, jumping, squatting, etc.

[0003] At present, traditional virtual reality game systems based on visual tracking technology tend to ignore capture errors caused by movement inertia, making it difficult to ensure the detailed performance of game characters' synchronized movements, resulting in poor naturalness and smoothness. In addition, due to the lack of in-depth analysis capabilities, it is difficult to timely detect game character movement delays and lags, resulting in slow optimization response speed and poor user gaming experience. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In response to the shortcomings of the existing technology, the present invention provides a virtual reality game system and method based on visual tracking technology, which has the advantages of high precision in split modeling capture and better user experience through intelligent optimization. It solves the problem that traditional virtual reality game systems based on visual tracking technology easily ignore capture errors caused by motion inertia and have slow optimization response speed.

[0006] (2) Technical solution

[0007] To achieve the above-mentioned object, the present invention provides the following technical solutions: a virtual reality game system based on visual tracking technology, comprising a multi-dimensional acquisition module and a virtual reality module;

[0008] The multi-dimensional acquisition module is composed of a virtual interaction unit and a motion capture unit. The virtual interaction unit collects an interaction data set through a network connection to a camera. The interaction data set includes interaction data of all users. The motion capture unit collects an action data set through a network connection to a sensor device. The action data set includes action performances of all users.

[0009] The virtual reality module consists of an interaction analysis unit, a perception analysis unit and a matching management unit. The interaction analysis unit constructs a facial model MM of each user based on the interaction data set, and marks facial feature points and virtual lines. The interaction analysis unit analyzes the change distance BJ of each facial feature point based on the facial model MM, and then generates a corresponding emotional intensity Qxq in combination with the interaction data set. The perception analysis unit is set with a fixed-length capture period P, and then constructs a limb model ZM of each user in combination with the action data set, and marks limb feature points and virtual lines. The perception analysis unit analyzes the length CD of each virtual line based on the limb model ZM, and then generates a corresponding perception intensity Gzq in combination with the interaction data set. The matching management unit forms a comprehensive model YM of each user based on the facial model MM and the limb model ZM, and then synchronously updates the facial feature points and virtual lines in combination with the interaction data set and the action data set. The matching management unit counts the delay time YCS of the synchronous update based on the interaction data set, the action data set and the comprehensive model YM, and sets a fixed-length delay threshold YCY to judge the degree of delay when the user action is substituted into the comprehensive model YM, and outputs the corresponding judgment result.

[0010] Preferably, the expression of the interactive data set is {Y1 s 、Y2 s 、Y3 s 、...、Yn s}, Y1 s To Yn s They are the interaction data of the first to nth users respectively. The interaction data includes facial images and voice records. s represents the specific time of obtaining the interaction data of each user.

[0011] Preferably, the expression of the action data set is {D1 g 、D2 g 、D3 g 、...、Dn g}, D1 to Dn g are the action performances of the first to nth users respectively, which include hand movements, leg movements and body movements, and g represents the weight of each user.

[0012] Preferably, the facial model MM construction process is as follows:

[0013] S11. Extract the interaction data of the i-th user in the interaction dataset and mark the facial image of the i-th user as MB i ;

[0014] S12: Based on the facial image MB of the i-th user i , using 3D software to build a 1:1 facial model MM i , and in the face model MM i The facial feature points are marked in the figure, including the center point of the hairline, the center point of the left eyebrow, the center point of the right eyebrow, the center point of the left pupil, the center point of the right pupil, the tip of the nose, the endpoint of the left corner of the mouth, the endpoint of the right corner of the mouth and the endpoint of the chin. Each facial feature point corresponds to a three-dimensional coordinate, among which the center point of the hairline, the tip of the nose and the endpoint of the chin form a vertical virtual line one, and the center point of the left pupil and the center point of the right pupil form a parallel virtual line one.

[0015] Preferably, the calculation process of the emotion intensity Qxq is as follows:

[0016] S21, according to the facial model MM i , calculate the change distance BJ of a single facial feature point;

[0017] S22. According to S21, calculate the change distance BJ of the left eyebrow center coordinate, the right eyebrow center coordinate, the left pupil center coordinate, the right pupil center coordinate, the left mouth corner endpoint coordinate, and the right mouth corner endpoint coordinate;

[0018] S23, extract the interaction data of the i-th user in the interaction data set, and count the voice record YU of the i-th user within one minute. i , and then calculate the interactive speech speed YS of the i-th user within one minute i .

[0019] Preferably, the limb model ZM construction process is as follows:

[0020] S31. Extract the action performance of the i-th user within the capture period P according to the action data set, and mark the hand action of the i-th user as SH i , mark the leg action of the i-th user as TU i , mark the body action of the i-th user as ST i ;

[0021] S32, according to the hand movement SH of the i-th user i , leg movement TU i and body movements ST i, using 3D software to build a 1:1 limb model ZM i , and in the limb model ZM i Limb feature points are marked in the figure, including the left shoulder endpoint, right shoulder endpoint, left and right shoulder center points, left hand endpoint, right hand endpoint, left foot endpoint and right foot endpoint. Each limb feature point corresponds to a three-dimensional coordinate, among which the left shoulder endpoint, right shoulder endpoint and left and right shoulder center points form parallel virtual line 2, the left shoulder endpoint and left hand endpoint form vertical virtual line 2, the right shoulder endpoint and right hand endpoint form vertical virtual line 3, the left and right shoulder center points and left foot endpoint form inclined virtual line 1, and the left and right shoulder center points and right foot endpoint form inclined virtual line 2.

[0022] Preferably, the calculation process of the perception intensity Gzq is as follows:

[0023] S41, according to the limb model ZM i , calculate the length CD of a single virtual line;

[0024] S42. According to S41, obtain the lengths CD of the second parallel virtual line, the second perpendicular virtual line, the third perpendicular virtual line, the first inclined virtual line, and the second inclined virtual line;

[0025] S43. Calculate the perception intensity Gzq of the i-th user based on the length CD and the interactive data set i .

[0026] Preferably, the matching management unit sets a vertical straight line of fixed length to connect the i-th user face model MM through three-dimensional software i The jaw endpoint and limb model ZM in i The left and right shoulder center points in the ith user form the complete comprehensive model YM i , and combined with the interaction dataset and action dataset, the facial feature points and virtual lines are updated synchronously.

[0027] Preferably, when the delay duration YCS exceeds the delay threshold YCY, it indicates that the degree of delay when the user action is substituted into the comprehensive model YM is serious, and an upgrade signal is generated and transmitted to the interaction analysis unit and the perception analysis unit. The interaction analysis unit and the perception analysis unit will re-upgrade the facial model MM and the limb model ZM, and the matching management unit will re-form the comprehensive model YM.

[0028] A virtual reality gaming method based on visual tracking technology comprises the following steps:

[0029] Step 1: Connect cameras and sensor devices through the network to obtain all users' interaction data and action performance, and classify them into interaction datasets and action datasets;

[0030] Step 2: Based on the interaction dataset and the interaction dataset, construct a facial model MM for each user, mark the facial feature points and virtual lines, then analyze the change distance BJ of each facial feature point and generate the corresponding emotion intensity Qxq;

[0031] Step 3: Set a fixed capture period P, build a limb model ZM for each user, mark the limb feature points and virtual lines, analyze the length CD of each virtual line, and generate the corresponding perception intensity Gzq;

[0032] Step 4: Based on the facial model MM and the limb model ZM, a comprehensive model YM is constructed for each user. Then, combined with the interaction dataset and the action dataset, the facial feature points and virtual lines are synchronously updated. The delay time YCS of the synchronous update is calculated to determine the degree of delay when the user action is substituted into the comprehensive model YM, and the corresponding judgment result is output.

[0033] Compared with the existing technology, the present invention provides a virtual reality game system and method based on visual tracking technology, which has the following beneficial effects:

[0034] 1. The present invention connects cameras and sensing devices through a multi-dimensional acquisition module network to obtain the interactive data and action performances of all users, and classifies them into interactive data sets and action data sets. The virtual reality module constructs a facial model MM of each user based on the interactive data sets and interactive data sets, marks facial feature points and virtual lines, and then analyzes the change distance BJ of each facial feature point to generate a corresponding emotional intensity Qxq. The virtual reality module sets a fixed-length capture period P, constructs a limb model ZM of each user, and then marks limb feature points and virtual lines, analyzes the length CD of each virtual line, and generates a corresponding perception intensity Gzq. During the dynamic capture process, the head and body may produce different movement patterns due to inertia. Separate modeling helps to independently optimize the movement performance of each part, improve the overall accuracy and naturalness, avoid errors caused by movement inertia, and have high capture accuracy.

[0035] 2. The present invention uses a virtual reality module to form a comprehensive model YM for each user based on the facial model MM and the limb model ZM, and then combines the interactive data set and the action data set to synchronously update the facial feature points and virtual lines, and calculates the delay time YCS of the synchronous update to judge the degree of delay when the user action is substituted into the comprehensive model YM. When the delay time YCS exceeds the delay threshold YCY, it indicates that the degree of delay when the user action is substituted into the comprehensive model YM is serious, and an upgrade signal is generated and transmitted to the interactive analysis unit and the perception analysis unit. The interactive analysis unit and the perception analysis unit will re-upgrade the facial model MM and the limb model ZM, and the matching management unit will re-form the comprehensive model YM, so as to timely discover the game character action delay and freeze problems, quickly optimize and update the virtual model, and intelligently optimize the user experience for a better experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of the system flow of the present invention;

[0037] Figure 2 This is a step diagram of the method of the present invention. DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0039] Because traditional virtual reality game systems based on visual tracking technology tend to ignore capture errors caused by motion inertia, it is difficult to ensure the details of the synchronized movements of game characters, resulting in poor naturalness and smoothness. In addition, due to the lack of in-depth analysis capabilities, it is difficult to timely detect delays and freezes in game character movements, resulting in slow optimization response speed and poor user gaming experience. Therefore, a virtual reality game system and method based on visual tracking technology is provided. Please refer to Figure 1 ,A virtual reality game system based on visual tracking technology, including a multi-dimensional acquisition module and a virtual reality module;

[0040] The multi-dimensional acquisition module consists of a virtual interaction unit and a motion capture unit. The virtual interaction unit collects interactive data sets through a network connection camera. The interactive data set includes the interactive data of all users. The expression of the interactive data set is {Y1 s 、Y2 s 、Y3 s 、...、Yn s}, Y1 s To Yn s The interaction data for the first to nth users, including facial images and voice recordings, are displayed. s represents the specific time at which each user's interaction data was obtained. This provides data support for the subsequent synchronization of the virtual expressions of the game characters. The vivid expression of facial expressions more realistically reflects the player's emotions and reactions, enhancing the immersion and interactivity of the game.

[0041] The motion capture unit collects motion data sets through a network connection to the sensor device. The motion data sets include all the user's motion performances. The expression of the motion data set is {D1 g 、D2 g 、D3 g 、...、Dn g}, D1 g To Dng The motion performances of the first to nth users respectively, including hand movements, leg movements, and body movements. g represents the weight of each user. Collecting user body movements helps to accurately estimate the user's height. Combined with the user's weight, it can quantitatively evaluate the user's three-dimensional spatial perception ability.

[0042] By using interactive and action datasets, users' facial and body movements can be tracked in real time, which can more accurately reflect their emotions and movements. The refined feedback mechanism can enhance users' immersion in the virtual environment, allowing them to express their emotions and movements more naturally and accurately in the game.

[0043] The virtual reality module consists of an interaction analysis unit, a perception analysis unit, and a matching management unit. The interaction analysis unit constructs a facial model MM for each user based on the interaction dataset and marks facial feature points and virtual lines. The construction process is as follows:

[0044] S11. Extract the interaction data of the i-th user in the interaction dataset and mark the facial image of the i-th user as MB i ;

[0045] S12: Based on the facial image MB of the i-th user i , using 3D software to build a 1:1 facial model MM i , and in the facial model MM i Facial feature points are marked in the middle, including the center of the hairline, the center of the left eyebrow, the center of the right eyebrow, the center of the left pupil, the center of the right pupil, the tip of the nose, the endpoints of the left corner of the mouth, the endpoints of the right corner of the mouth, and the endpoints of the chin. Each facial feature point corresponds to a three-dimensional coordinate. The center of the hairline, the tip of the nose, and the endpoints of the chin form a vertical virtual line 1, and the center of the left pupil and the center of the right pupil form a parallel virtual line 1. Specifically, the degree of eyebrow lift, the speed of pupil rotation, and the degree of mouth corner lift are key features that intuitively reflect the user's emotional fluctuations;

[0046] The interaction analysis unit analyzes the change distance BJ of each facial feature point based on the facial model MM, and then combines it with the interaction data set to generate the corresponding emotion intensity Qxq. The calculation process is as follows:

[0047] S21, according to the facial model MM i , calculate the change distance BJ of a single facial feature point, and the calculation formula is as follows:

[0048]

[0049] In the formula, x1(s) represents the time point s when a single facial feature point is in the facial model MM. iThe position on the x-axis, y(s) represents the position of a single facial feature point on the facial model MM at time point s i The position on the y-axis, z1(s) represents the position of a single facial feature point on the facial model MM at time point s i The position on the z-axis, x2(s+1) represents the position of a single facial feature point on the facial model MM at time point s+1 i The position on the x-axis, y2(s+1) represents the position of a single facial feature point on the facial model MM at time point s+1 i The position on the y-axis, z2(s+1) represents the position of a single facial feature point on the facial model MM at time point s+1 i The position on the z-axis, Indicates the change distance of a single facial feature point from time point s to time point s+1 according to the Euclidean distance formula;

[0050] S22. According to S21, calculate the change distance BJ of the left eyebrow center coordinate, the right eyebrow center coordinate, the left pupil center coordinate, the right pupil center coordinate, the left mouth corner endpoint coordinate, and the right mouth corner endpoint coordinate;

[0051] S23, extract the interaction data of the i-th user in the interaction data set, and count the voice record YU of the i-th user within one minute. i , and then calculate the interactive speech speed YS of the i-th user within one minute i , which is calculated as follows:

[0052]

[0053] When users are emotionally excited, their interactive speech speed will also increase, which is also a key feature that directly reflects the user's emotional fluctuations;

[0054] S24, based on the change distance BJ of all facial feature points and the interactive speech speed YS i , calculate the emotional intensity Qxq of the i-th user i , which is calculated as follows:

[0055] QXd i =α1×maxBJ+α2×YS i

[0056] In the formula, maxBJ represents the facial feature point with the largest change distance from time point s to time point s+1 among the hairline center point, left eyebrow center point, right eyebrow center point, left pupil center point, right pupil center point, nose tip point, left mouth corner endpoint, right mouth corner endpoint, and chin center point. α1 represents the evaluation weight for the maximum change distance, and α2 represents the evaluation weight for the interactive speech rate. α1+α2=1, α1×maxBJ+α2×YS iAccording to the weights α1 and α2, the emotional intensity of the i-th user is obtained by combining the change distance and the interaction speed. The higher the intensity value, the greater the user's emotional fluctuation.

[0057] The perception analysis unit is set with a fixed capture period P. It then combines the action dataset to construct a limb model ZM for each user and mark the limb feature points and virtual lines. The construction process is as follows:

[0058] S31. Extract the action performance of the i-th user within the capture period P according to the action data set, and mark the hand action of the i-th user as SH i , mark the leg action of the i-th user as TU i , mark the body action of the i-th user as ST i ;

[0059] S32, according to the hand movement SH of the i-th user i , leg movement TU i and body movements ST i , using 3D software to build a 1:1 limb model ZM i , and in the limb model ZM i Limb feature points are marked in the middle, including the left shoulder endpoint, right shoulder endpoint, left and right shoulder center points, left hand endpoint, right hand endpoint, left foot endpoint, and right foot endpoint. Each limb feature point corresponds to a three-dimensional coordinate. The left shoulder endpoint, right shoulder endpoint, and left and right shoulder center points form parallel virtual line 2, the left shoulder endpoint and left hand endpoint form vertical virtual line 2, the right shoulder endpoint and right hand endpoint form vertical virtual line 3, the left and right shoulder centers and left foot endpoint form inclined virtual line 1, and the left and right shoulder centers and right foot endpoint form inclined virtual line 2. Specifically, shoulder length and arm length can accurately predict the user's height;

[0060] The perception analysis unit analyzes the length CD of each virtual line based on the limb model ZM, and then combines it with the interaction data set to generate the corresponding perception intensity Gzq. The calculation process is as follows:

[0061] S41, according to the limb model ZM i , calculate the length CD of a single virtual line, the calculation formula is as follows:

[0062]

[0063] In the formula, dx1 represents the limb model ZM i In the figure, the position of one end of a single virtual line on the x-axis, dx represents the position of the end of the limb model ZM i In the figure, the position of the other end of the single virtual line on the x-axis, dy1 represents the position of the other end of the single virtual line on the limb model ZM i In the figure, the position of one end of a single virtual line on the y-axis, dy2 represents the position of the end of the limb model ZMi In the figure, the position of the other end of the single virtual line on the y-axis, dz1 represents the position of the other end of the single virtual line on the limb model ZM i In the figure, the position of one end of a single virtual line on the z-axis, dz2 represents the position of the end of the limb model ZM i In the example, the position of the other end of the single virtual line on the z-axis is Indicates the length of a single virtual line obtained according to the Euclidean distance formula;

[0064] S42. According to S41, obtain the lengths CD of the second parallel virtual line, the second perpendicular virtual line, the third perpendicular virtual line, the first inclined virtual line, and the second inclined virtual line;

[0065] S43. Calculate the perception intensity Gzq of the i-th user based on the length CD and the interactive data set i , which is calculated as follows:

[0066]

[0067] In the formula, CD px2 represents the length of the second parallel imaginary line, CD cx2 Indicates the length of vertical imaginary line 2, CD cx3 represents the length of vertical imaginary line 3, 0.12 represents the conversion factor used to convert shoulder width to height, and 0.4 represents the conversion factor used to convert arm length to height. represents the average predicted value of height, β1 represents the evaluation weight for the average predicted value of height, i g represents the weight of the i-th user, β1 represents the evaluation weight for weight, β1+β2=1, According to the weights β1 and β2, the length of the virtual line and the weight are combined to obtain the perception strength of the i-th user, so as to achieve the game performance of the virtual character tailored to each user;

[0068] The matching management unit forms a comprehensive model YM for each user based on the facial model MM and the limb model ZM. Specifically, through the 3D software, a vertical straight line of fixed length is set to connect the facial model MM of the i-th user. i The jaw endpoint and limb model ZM in i The left and right shoulder center points in the ith user form the complete comprehensive model YM i , and combined with the interaction dataset and action dataset, synchronously update the facial feature points and virtual lines;

[0069] The matching management unit calculates the delay time YCS of synchronous updates based on the interactive data set, action data set and comprehensive model YM, and sets a fixed delay threshold YCY to judge the degree of delay when the user action is substituted into the comprehensive model YM. When the delay time YCS exceeds the delay threshold YCY, it indicates that the delay when the user action is substituted into the comprehensive model YM is serious, and an upgrade signal is generated and transmitted to the interactive analysis unit and the perception analysis unit. The interactive analysis unit and the perception analysis unit will re-upgrade the facial model MM and the limb model ZM. The matching management unit will re-compose the comprehensive model YM, monitor and analyze the updated delay time, and adjust the response mechanism of the interactive content in a timely manner to ensure the efficiency and response accuracy of the system, which helps to enhance the real-time interactivity of virtual reality games.

[0070] See also Figure 2 , a virtual reality game method based on visual tracking technology, comprising the following steps:

[0071] Step 1: Connect cameras and sensor devices through the network to obtain all users' interaction data and action performance, and classify them into interaction datasets and action datasets;

[0072] Step 2: Based on the interaction dataset and the interaction dataset, construct a facial model MM for each user, mark the facial feature points and virtual lines, then analyze the change distance BJ of each facial feature point and generate the corresponding emotion intensity Qxq;

[0073] Step 3: Set a fixed capture period P, build a limb model ZM for each user, mark the limb feature points and virtual lines, analyze the length CD of each virtual line, and generate the corresponding perception intensity Gzq. During the dynamic capture process, the head and body may produce different movement patterns due to inertia. Separate modeling helps to independently optimize the movement performance of each part, improve overall accuracy and naturalness, avoid errors caused by movement inertia, and achieve high capture accuracy.

[0074] Step 4: Based on the facial model MM and the limb model ZM, a comprehensive model YM is constructed for each user. Then, combined with the interaction dataset and the action dataset, the facial feature points and virtual lines are synchronously updated. The delay duration YCS of the synchronous update is calculated to determine the degree of delay when the user action is substituted into the comprehensive model YM. The corresponding judgment result is output to promptly detect delays and lags in the game character's actions, quickly optimize and update the virtual model, and intelligently optimize the user experience for a better experience.

[0075] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A virtual reality game system based on visual tracking technology, characterized by: Including multi-dimensional acquisition module and virtual reality module; The multi-dimensional acquisition module is composed of a virtual interaction unit and a motion capture unit. The virtual interaction unit collects an interaction data set through a network connection to a camera. The interaction data set includes interaction data of all users. The motion capture unit collects an action data set through a network connection to a sensor device. The action data set includes action performances of all users. The virtual reality module consists of an interaction analysis unit, a perception analysis unit, and a matching management unit. The interaction analysis unit constructs a facial model of each user based on the interaction data set. , and mark facial feature points and virtual lines, the interactive analysis unit according to the facial model , analyze the change distance of each facial feature point , combined with the interactive data set, to generate the corresponding emotion intensity The perception analysis unit is set with a fixed-length capture cycle , combined with the action dataset, to build a limb model for each user , and mark the limb feature points and virtual lines, the perception analysis unit according to the limb model , analyze the length of each virtual line , combined with the interactive data set, to generate the corresponding perception intensity The matching management unit is based on the facial model and limb models , forming a comprehensive model for each user , combined with the interactive dataset and the action dataset, the facial feature points and virtual lines are updated synchronously, and the matching management unit is based on the interactive dataset, the action dataset and the comprehensive model , statistics synchronization update delay , and set a fixed delay threshold , judge the user action and substitute it into the comprehensive model The degree of delay when outputting the corresponding judgment result; Matching snap-in based on facial model and limb models , forming a comprehensive model for each user Specifically, through 3D software, set a vertical straight line of fixed length to connect the first User face model The jaw endpoint and limb model in The center points of the left and right shoulders form the A complete comprehensive model of each user , and combined with the interaction dataset and action dataset, the facial feature points and virtual lines are updated synchronously.

2. A virtual reality game system based on visual tracking technology according to claim 1, characterized in that: The expression of the interactive data set is , to The first to the Interaction data of each user, including facial images and voice recordings, Indicates the specific time for obtaining each user interaction data.

3. A virtual reality game system based on visual tracking technology according to claim 2, characterized in that: The expression of the action dataset is: , to The first to the The user's motion performance includes hand movements, leg movements and body movements. Represents the weight of each user.

4. A virtual reality game system based on visual tracking technology according to claim 3, characterized in that: The facial model The build process is as follows: S11. Extract the first The interaction data of each user and The facial images of users are labeled ; S12. According to facial images of users , using 3D software to build a 1:1 facial model , and in the facial model The facial feature points are marked in the figure, including the center point of the hairline, the center point of the left eyebrow, the center point of the right eyebrow, the center point of the left pupil, the center point of the right pupil, the tip of the nose, the endpoint of the left corner of the mouth, the endpoint of the right corner of the mouth and the endpoint of the chin. Each facial feature point corresponds to a three-dimensional coordinate, among which the center point of the hairline, the tip of the nose and the endpoint of the chin form a vertical virtual line one, and the center point of the left pupil and the center point of the right pupil form a parallel virtual line one.

5. The virtual reality game system based on visual tracking technology according to claim 4, characterized in that: The intensity of the emotions The calculation process is as follows: S21. Based on facial model , calculate the change distance of a single facial feature point ; S22. According to S21, calculate the change distance of the left eyebrow center coordinate, the right eyebrow center coordinate, the left pupil center coordinate, the right pupil center coordinate, the left mouth corner endpoint coordinate and the right mouth corner endpoint coordinate. ; S23, extract the first The interactive data of each user is counted within one minute. Voice recordings of users , calculate within one minute, User's interactive speech rate , S24, based on the change distance of all facial feature points and interactive speech speed , calculate the The emotional intensity of a user .

6. The virtual reality game system based on visual tracking technology according to claim 5, characterized in that: The limb model The build process is as follows: S31. Extract capture cycle based on action data set within, no. The action performance of each user and The hand movements of users are marked as , will The leg movements of users are marked as , will The user's body movements are marked as ; S32. According to User's hand movements , leg movements and body movements , using 3D software to build a 1:1 limb model , and in the limb model Limb feature points are marked in the figure, including the left shoulder endpoint, right shoulder endpoint, left and right shoulder center points, left hand endpoint, right hand endpoint, left foot endpoint and right foot endpoint. Each limb feature point corresponds to a three-dimensional coordinate, among which the left shoulder endpoint, right shoulder endpoint and left and right shoulder center points form parallel virtual line 2, the left shoulder endpoint and left hand endpoint form vertical virtual line 2, the right shoulder endpoint and right hand endpoint form vertical virtual line 3, the left and right shoulder center points and left foot endpoint form inclined virtual line 1, and the left and right shoulder center points and right foot endpoint form inclined virtual line 2.

7. The virtual reality game system based on visual tracking technology according to claim 6, characterized in that: The perceived intensity The calculation process is as follows: S41, according to the limb model , calculate the length of a single virtual line ; S42. According to S41, the lengths of the second parallel virtual line, the second vertical virtual line, the third vertical virtual line, the first inclined virtual line, and the second inclined virtual line are obtained. ; S43, according to length and interactive datasets, calculate the User's perceived strength .

8. The virtual reality game system based on visual tracking technology according to claim 7, characterized in that: The matching management unit sets a vertical straight line of fixed length to connect the first User face model The jaw endpoint and limb model in The center points of the left and right shoulders form the A complete comprehensive model of each user , and combined with the interaction dataset and action dataset, the facial feature points and virtual lines are updated synchronously.

9. The virtual reality game system based on visual tracking technology according to claim 8, characterized in that: The delay duration Latency threshold exceeded When , it means that the user action is substituted into the comprehensive model The delay is serious, and an upgrade signal is generated and transmitted to the interactive analysis unit and the perception analysis unit, and the interactive analysis unit and the perception analysis unit will re-upgrade the facial model. and limb models , the matching management unit will reconstruct the comprehensive model .

10. A virtual reality game method based on visual tracking technology, applied to a virtual reality game system based on visual tracking technology as claimed in any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Connect cameras and sensor devices through the network to obtain all users' interaction data and action performance, and classify them into interaction datasets and action datasets; Step 2: Build a facial model for each user based on the interaction dataset and the interaction dataset , mark facial feature points and virtual lines, and then analyze the change distance of each facial feature point , and generate the corresponding emotional intensity ; Step 3: Set a fixed capture period , build a limb model for each user , then mark the limb feature points and virtual lines, and analyze the length of each virtual line , and generate the corresponding perceptual intensity ; Step 4: Based on the facial model and limb models , forming a comprehensive model for each user , combined with the interactive dataset and action dataset, synchronously update facial feature points and virtual lines, and calculate the delay time of synchronous update , judge the user action and substitute it into the comprehensive model The delay degree is determined and the corresponding judgment result is output.

Images