Virtual reality game system and method based on visual tracking technology

Abstract

The invention relates to the technical field of virtual reality games, and discloses a virtual reality game system and method based on a visual tracking technology, and the system comprises a multi-dimensional collection module and a virtual reality module. According to the virtual reality game system and method based on the visual tracking technology, interaction data and action expressions of all users are obtained through a multi-dimensional collection module, a data set is formed through classification, a virtual reality module constructs a face model of each user, facial feature points and virtual lines are marked, then emotion intensity is generated through analysis, and the emotion intensity of each user is obtained. The virtual reality module constructs a limb model of each user, marks limb feature points and virtual lines, then analyzes and generates perception intensity, and independently optimizes movement performance of each part, split modeling is high in capture precision, the virtual reality module forms a comprehensive model according to a face model and the limb models, and the comprehensive model is more accurate. And the delay degree when the user action is substituted into the comprehensive model is judged, the game role action delay and jamming problems are found in time, and the intelligent optimization user experience is better.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality game technology, and in particular to a virtual reality game system and method based on visual tracking technology. Background Art

[0002] Visual tracking technology is one of the important research directions in the field of computer vision. It aims to monitor the position changes of targets in real time through video or image sequences and accurately predict their future positions. Visual tracking technology is widely used in many fields such as autonomous driving, security monitoring, drone navigation, human-computer interaction, virtual reality games, etc. In the field of virtual reality games, visual tracking technology accurately captures the player's movements, positions and perspectives, so that the characters and environment in the virtual world can respond in real time according to the player's actual movements. Head tracking is one of the core applications in virtual reality systems. Usually, built-in sensors track the position and posture changes of the player's head in real time to achieve a 360-degree panoramic immersive experience. Head tracking not only improves the user's sense of immersion, but also allows players to interact with the virtual environment more naturally, such as turning their heads to view the surrounding virtual scenes. The player's hand movements are also key to interacting with the virtual world. Visual tracking technology captures the player's gestures through hand tracking devices to achieve synchronous grabbing, moving objects, and even performing complex movements or operations in the virtual world. In order to allow players to perform richer movements in virtual reality games, many advanced VR systems use full-body tracking technology. By installing multiple sensors, cameras or markers on the player, the player's whole body movement is captured and more realistic interactions are created, including walking, running, jumping, squatting, etc.

[0003] At present, traditional virtual reality game systems based on visual tracking technology tend to ignore capture errors caused by motion inertia, making it difficult to ensure the detailed performance of game characters' synchronized movements, and the naturalness and smoothness are poor. In addition, the lack of in-depth analysis capabilities makes it difficult to promptly detect game character movement delays and stuttering problems, resulting in slow optimization response speeds and poor user gaming experience. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a virtual reality game system and method based on visual tracking technology, which has the advantages of high precision in split modeling capture and better user experience through intelligent optimization, and solves the problem that traditional virtual reality game systems based on visual tracking technology easily ignore capture errors caused by motion inertia and have slow optimization response speed.

[0006] (II) Technical solution

[0007] To achieve the above-mentioned object, the present invention provides the following technical solutions: a virtual reality game system based on visual tracking technology, comprising a multi-dimensional acquisition module and a virtual reality module;

[0008] The multi-dimensional acquisition module is composed of a virtual interaction unit and a motion capture unit. The virtual interaction unit collects an interaction data set through a network connection camera, and the interaction data set includes the interaction data of all users. The motion capture unit collects an action data set through a network connection sensor device, and the action data set includes the action performance of all users.

[0009] The virtual reality module is composed of an interactive analysis unit, a perception analysis unit and a matching management unit. The interactive analysis unit constructs a facial model MM of each user according to the interactive data set, and marks the facial feature points and virtual lines. The interactive analysis unit analyzes the change distance BJ of each facial feature point according to the facial model MM, and generates the corresponding emotion intensity Qxq in combination with the interactive data set. The perception analysis unit is provided with a fixed-duration capture period P, and constructs a limb model ZM of each user in combination with the action data set, and marks the limb feature points and virtual lines. The perception analysis unit analyzes the length CD of each virtual line according to the limb model ZM, and generates the corresponding perception intensity Gzq in combination with the interactive data set. The matching management unit composes a comprehensive model YM of each user according to the facial model MM and the limb model ZM, and synchronously updates the facial feature points and virtual lines in combination with the interactive data set and the action data set. The matching management unit counts the delay time YCS of the synchronous update according to the interactive data set, the action data set and the comprehensive model YM, and sets a fixed-duration delay threshold YCY to judge the degree of delay when the user action is substituted into the comprehensive model YM, and outputs the corresponding judgment result.

[0010] Preferably, the expression of the interactive data set is {Y1 s 、Y2 s 、Y3 s 、...、Yn s}, Y1 s To Yn s They are the interaction data of the first to the nth user respectively. The interaction data include facial images and voice records. s represents the specific time of obtaining the interaction data of each user.

[0011] Preferably, the expression of the action data set is {D1 g 、D2 g 、D3 g 、...、Dn g}, D1 to Dn g They are the action performances of the first to the nth user respectively, the action performances include hand movements, leg movements and body movements, and g represents the weight of each user.

[0012] Preferably, the facial model MM construction process is as follows:

[0013] S11, extract the interaction data of the i-th user in the interaction data set, and mark the facial image of the i-th user as MB i ;

[0014] S12: Based on the facial image MB of the i-th user i , using 3D software to build a 1:1 facial model MM i , and in the face model MM i The facial feature points are marked in the figure, including the center point of the hairline, the center point of the left eyebrow, the center point of the right eyebrow, the center point of the left pupil, the center point of the right pupil, the tip of the nose, the endpoint of the left corner of the mouth, the endpoint of the right corner of the mouth and the endpoint of the chin. Each facial feature point corresponds to a three-dimensional coordinate, among which the center point of the hairline, the tip of the nose and the endpoint of the chin form a vertical virtual line one, and the center point of the left pupil and the center point of the right pupil form a parallel virtual line one.

[0015] Preferably, the calculation process of the emotion intensity Qxq is as follows:

[0016] S21, according to the facial model MM i , calculate the change distance BJ of a single facial feature point;

[0017] S22, according to S21, calculate the change distance BJ of the left eyebrow center point coordinate, the right eyebrow center point coordinate, the left pupil center point coordinate, the right pupil center point coordinate, the left mouth corner endpoint coordinate and the right mouth corner endpoint coordinate;

[0018] S23, extract the interaction data of the i-th user in the interaction data set, and count the voice record YU of the i-th user within one minute. i , and then calculate the interactive speech rate YS of the i-th user within one minute i .

[0019] Preferably, the limb model ZM construction process is as follows:

[0020] S31. Extract the action performance of the i-th user within the capture period P according to the action data set, and mark the hand action of the i-th user as SH i , mark the leg action of the i-th user as TU i , mark the body action of the i-th user as ST i ;

[0021] S32: According to the hand movement SH of the i-th user i , leg movements TU i and body movements ST i, using 3D software to build a 1:1 limb model ZM i , and in the limb model ZM i Limb feature points are marked in the figure, including the left shoulder endpoint, the right shoulder endpoint, the center point of the left and right shoulders, the left hand endpoint, the right hand endpoint, the left foot endpoint and the right foot endpoint. Each limb feature point corresponds to a three-dimensional coordinate, wherein the left shoulder endpoint, the right shoulder endpoint and the center point of the left and right shoulders form a parallel virtual line 2, the left shoulder endpoint and the left hand endpoint form a vertical virtual line 2, the right shoulder endpoint and the right hand endpoint form a vertical virtual line 3, the center point of the left and right shoulders and the left foot endpoint form an inclined virtual line 1, and the center point of the left and right shoulders and the right foot endpoint form an inclined virtual line 2.

[0022] Preferably, the calculation process of the perception intensity Gzq is as follows:

[0023] S41, according to the limb model ZM i , calculate the length CD of a single virtual line;

[0024] S42, according to S41, obtaining the lengths CD of the parallel virtual line 2, the vertical virtual line 2, the vertical virtual line 3, the inclined virtual line 1 and the inclined virtual line 2;

[0025] S43. Calculate the perception intensity Gzq of the i-th user according to the length CD and the interactive data set. i .

[0026] Preferably, the matching management unit sets a vertical straight line of fixed length to connect the i-th user face model MM through three-dimensional software. i The jaw endpoint and limb model ZM in i The left and right shoulder center points in the ith user form the complete comprehensive model YM i , and combined with the interaction dataset and action dataset, the facial feature points and virtual lines are updated synchronously.

[0027] Preferably, when the delay duration YCS exceeds the delay threshold YCY, it indicates that the degree of delay when the user action is substituted into the comprehensive model YM is serious, and an upgrade signal is generated and transmitted to the interaction analysis unit and the perception analysis unit. The interaction analysis unit and the perception analysis unit will re-upgrade the facial model MM and the limb model ZM, and the matching management unit will re-form the comprehensive model YM.

[0028] A virtual reality game method based on visual tracking technology comprises the following steps:

[0029] Step 1: Connect cameras and sensor devices through the network to obtain the interaction data and action performance of all users, and classify them into interaction data sets and action data sets;

[0030] Step 2: Based on the interactive dataset and the interactive dataset, construct the facial model MM of each user, mark the facial feature points and virtual lines, then analyze the change distance BJ of each facial feature point, and generate the corresponding emotion intensity Qxq;

[0031] Step 3: Set a fixed capture period P, build a limb model ZM for each user, mark limb feature points and virtual lines, analyze the length CD of each virtual line, and generate the corresponding perception intensity Gzq;

[0032] Step 4: Based on the facial model MM and the limb model ZM, a comprehensive model YM of each user is formed. Then, combined with the interaction dataset and the action dataset, the facial feature points and virtual lines are synchronously updated, the delay time YCS of the synchronous update is calculated, the degree of delay when the user action is substituted into the comprehensive model YM is judged, and the corresponding judgment result is output.

[0033] Compared with the prior art, the present invention provides a virtual reality game system and method based on visual tracking technology, which has the following beneficial effects:

[0034] 1. The present invention connects the camera and the sensor device through a multi-dimensional acquisition module network to obtain the interactive data and action performance of all users, and classifies them into interactive data sets and action data sets. The virtual reality module constructs the facial model MM of each user according to the interactive data set and the interactive data set, marks the facial feature points and virtual lines, and then analyzes the change distance BJ of each facial feature point, and generates the corresponding emotional intensity Qxq. The virtual reality module sets a fixed-length capture period P, constructs the limb model ZM of each user, and then marks the limb feature points and virtual lines, analyzes the length CD of each virtual line, and generates the corresponding perception intensity Gzq. In the dynamic capture process, the head and body may produce different movement patterns due to inertia. Separate modeling helps to independently optimize the movement performance of each part, improve the overall accuracy and naturalness, avoid errors caused by movement inertia, and the split modeling has high capture accuracy.

[0035] 2. The present invention forms a comprehensive model YM for each user based on the facial model MM and the limb model ZM through the virtual reality module, and then combines the interactive data set and the action data set to synchronously update the facial feature points and the virtual lines, and counts the delay time YCS of the synchronous update to judge the degree of delay when the user action is substituted into the comprehensive model YM. When the delay time YCS exceeds the delay threshold YCY, it indicates that the degree of delay when the user action is substituted into the comprehensive model YM is serious, and an upgrade signal is generated and transmitted to the interactive analysis unit and the perception analysis unit. The interactive analysis unit and the perception analysis unit will re-upgrade the facial model MM and the limb model ZM, and the matching management unit will re-form the comprehensive model YM, so as to timely discover the game character action delay and jamming problems, quickly optimize and update the virtual model, and intelligently optimize the user experience for a better experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the system flow of the present invention;

[0037] Figure 2 It is a step diagram of the method of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Since the traditional virtual reality game system based on visual tracking technology tends to ignore the capture error caused by the inertia of the action, it is difficult to ensure the detailed performance of the game character's synchronous action, and the naturalness and smoothness are poor. In addition, due to the lack of in-depth analysis capabilities, it is difficult to timely discover the delay and freeze problems of the game character's action, the optimization response speed is slow, and the user's gaming experience is poor. Therefore, a virtual reality game system and method based on visual tracking technology are provided. Please refer to Figure 1 ,A virtual reality game system based on visual tracking technology, includes a multi-dimensional acquisition module and a virtual reality module;

[0040] The multi-dimensional acquisition module consists of a virtual interaction unit and a motion capture unit. The virtual interaction unit collects interactive data sets through a network connection camera. The interactive data set includes the interactive data of all users. The expression of the interactive data set is {Y1 s 、Y2 s 、Y3 s 、...、Yn s}, Y1 s To Yn s The interactive data of the first to nth users respectively, including facial images and voice records. s represents the specific time of obtaining the interactive data of each user, which provides data support for the subsequent synchronization of the virtual expressions of the game characters. The vivid expression of facial expressions can more realistically express the emotions and reactions of players, thus enhancing the immersion and interactivity of the game.

[0041] The motion capture unit collects the motion data set through the network connection sensor device. The motion data set includes the motion performance of all users. The expression of the motion data set is {D1 g 、D2 g 、D3 g 、...、Dn g}, D1 g To Dng They are the motion performances of the first to nth users, including hand motions, leg motions, and body motions. g represents the weight of each user. Collecting user body motions is helpful for accurately estimating user heights in the future. Combined with user weight, the user's three-dimensional spatial perception ability can be quantitatively evaluated.

[0042] By using interactive and action datasets, users’ facial and body movements can be tracked in real time, which can more accurately reflect their emotions and movements. The refined feedback mechanism can enhance users’ immersion in the virtual environment, allowing users’ emotions and movements to be expressed more naturally and accurately in the game.

[0043] The virtual reality module consists of an interactive analysis unit, a perception analysis unit, and a matching management unit. The interactive analysis unit constructs a facial model MM for each user based on the interactive data set, and marks facial feature points and virtual lines. The construction process is as follows:

[0044] S11, extract the interaction data of the i-th user in the interaction data set, and mark the facial image of the i-th user as MB i ;

[0045] S12: Based on the facial image MB of the i-th user i , using 3D software to build a 1:1 facial model MM i , and in the face model MM i The facial feature points are marked in the middle, including the center point of the hairline, the center point of the left eyebrow, the center point of the right eyebrow, the center point of the left pupil, the center point of the right pupil, the tip of the nose, the endpoint of the left corner of the mouth, the endpoint of the right corner of the mouth and the endpoint of the chin. Each facial feature point corresponds to a three-dimensional coordinate. The center point of the hairline, the tip of the nose and the endpoint of the chin form a vertical virtual line 1, and the center point of the left pupil and the center point of the right pupil form a parallel virtual line 1. Specifically, the degree of eyebrow lifting, the speed of pupil rotation and the degree of mouth corner lifting are key features that directly reflect the user's emotional fluctuations;

[0046] The interaction analysis unit analyzes the change distance BJ of each facial feature point based on the facial model MM, and then generates the corresponding emotion intensity Qxq in combination with the interaction data set. The calculation process is as follows:

[0047] S21, according to the facial model MM i , calculate the change distance BJ of a single facial feature point, and the calculation formula is as follows:

[0048]

[0049] In the formula, x1(s) represents the time point s when a single facial feature point is in the facial model MM. iThe position on the x-axis, y(s) represents the position of a single facial feature point on the facial model MM at time point s. i The position on the y-axis, z1(s) represents the position of a single facial feature point on the facial model MM at time point s. i The position on the z-axis, x2(s+1) represents the position of a single facial feature point on the facial model MM at time point s+1 i The position on the x-axis, y2(s+1) represents the position of a single facial feature point on the facial model MM at time point s+1 i The position on the y-axis, z2(s+1) represents the position of a single facial feature point on the facial model MM at time point s+1 i The position on the z-axis, Indicates the change distance of a single facial feature point from time point s to time point s+1 according to the Euclidean distance formula;

[0050] S22, according to S21, calculate the change distance BJ of the left eyebrow center point coordinate, the right eyebrow center point coordinate, the left pupil center point coordinate, the right pupil center point coordinate, the left mouth corner endpoint coordinate and the right mouth corner endpoint coordinate;

[0051] S23, extract the interaction data of the i-th user in the interaction data set, and count the voice record YU of the i-th user within one minute. i , and then calculate the interactive speech rate YS of the i-th user within one minute i , and its calculation formula is as follows:

[0052]

[0053] When users are emotional, their interactive speech speed will also increase, which is also a key feature that directly reflects the user's emotional fluctuations;

[0054] S24, based on the change distance BJ of all facial feature points and the interactive speech speed YS i , calculate the emotional intensity Qxq of the i-th user i , and its calculation formula is as follows:

[0055] QqI i =α1×maxBJ+α2×YS i

[0056] In the formula, maxBJ represents the facial feature point with the largest change distance from time point s to time point s+1 among the center points of the hairline, the center points of the left eyebrow, the center points of the right eyebrow, the center points of the left pupil, the center points of the right pupil, the tip of the nose, the endpoints of the left corner of the mouth, the endpoints of the right corner of the mouth, and the center points of the chin. α1 represents the evaluation weight for the maximum change distance, and α2 represents the evaluation weight for the interactive speech rate. α1+α2=1, α1×maxBJ+α2×YS iIt means that the emotional intensity of the i-th user is obtained according to the weights of α1 and α2, the comprehensive change distance and the interactive speech speed. The higher the intensity value, the greater the emotional fluctuation of the user;

[0057] The perception analysis unit is set with a fixed-length capture cycle P. Combined with the action data set, it constructs the limb model ZM of each user and marks the limb feature points and virtual lines. The construction process is as follows:

[0058] S31. Extract the action performance of the i-th user within the capture period P according to the action data set, and mark the hand action of the i-th user as SH i , mark the leg action of the i-th user as TU i , mark the body action of the i-th user as ST i ;

[0059] S32: According to the hand movement SH of the i-th user i , leg movements TU i and body movements ST i , using 3D software to build a 1:1 limb model ZM i , and in the limb model ZM i Mark limb feature points, including the left shoulder endpoint, the right shoulder endpoint, the center point of the left and right shoulders, the left hand endpoint, the right hand endpoint, the left foot endpoint and the right foot endpoint. Each limb feature point corresponds to a three-dimensional coordinate. The left shoulder endpoint, the right shoulder endpoint and the center point of the left and right shoulders form a parallel virtual line 2, the left shoulder endpoint and the left hand endpoint form a vertical virtual line 2, the right shoulder endpoint and the right hand endpoint form a vertical virtual line 3, the center points of the left and right shoulders and the left foot endpoint form an inclined virtual line 1, and the center points of the left and right shoulders and the right foot endpoint form an inclined virtual line 2. Specifically, the shoulder length and arm length can accurately predict the user's height;

[0060] The perception analysis unit analyzes the length CD of each virtual line according to the limb model ZM, and then combines the interactive data set to generate the corresponding perception intensity Gzq. The calculation process is as follows:

[0061] S41, according to the limb model ZM i , calculate the length CD of a single virtual line, the calculation formula is as follows:

[0062]

[0063] In the formula, dx1 represents the limb model ZM i In the figure, the position of one end of a single virtual line on the x-axis, dx represents the position of the end of the single virtual line on the limb model ZM i In the figure, the position of the other end of the single virtual line on the x-axis, dy1 represents the position of the other end of the single virtual line on the limb model ZM i In the figure, the position of one end of a single virtual line on the y-axis, dy2 represents the position of the end of the single virtual line on the limb model ZMi In the figure, the position of the other end of the single virtual line on the y-axis, dz1 represents the position of the other end of the single virtual line on the limb model ZM i In the figure, the position of one end of a single virtual line on the z-axis, dz2 represents the position of the end of the single virtual line on the z-axis in the limb model ZM i In the figure, the position of the other end of the single virtual line on the z-axis is Indicates the length of a single virtual line obtained according to the Euclidean distance formula;

[0064] S42, according to S41, obtaining the lengths CD of the parallel virtual line 2, the vertical virtual line 2, the vertical virtual line 3, the inclined virtual line 1 and the inclined virtual line 2;

[0065] S43. Calculate the perception intensity Gzq of the i-th user according to the length CD and the interactive data set. i , and its calculation formula is as follows:

[0066]

[0067] In the formula, CD px2 represents the length of the second parallel imaginary line, CD cx2 represents the length of the second vertical imaginary line, CD cx3 represents the length of vertical virtual line 3, 0.12 represents the conversion factor used to convert shoulder width to height, 0.4 represents the conversion factor used to convert arm length to height, represents the average predicted value of height, β1 represents the evaluation weight for the average predicted value of height, i g represents the weight of the i-th user, β1 represents the evaluation weight for weight, β1+β2=1, It means that according to the weights β1 and β2, the virtual line length and weight are combined to obtain the perception strength of the i-th user, so as to achieve the game performance of the virtual character tailored for each user;

[0068] The matching management unit forms a comprehensive model YM for each user based on the facial model MM and the limb model ZM. Specifically, through the three-dimensional software, a vertical straight line of fixed length is set to connect the facial model MM of the i-th user. i The jaw endpoint and limb model ZM in i The left and right shoulder center points in the ith user form the complete comprehensive model YM i , and combined with the interactive dataset and the action dataset, the facial feature points and virtual lines are updated synchronously;

[0069] The matching management unit counts the delay time YCS of synchronous update according to the interactive data set, action data set and comprehensive model YM, and sets a delay threshold YCY of fixed time to judge the degree of delay when the user action is substituted into the comprehensive model YM. When the delay time YCS exceeds the delay threshold YCY, it indicates that the degree of delay when the user action is substituted into the comprehensive model YM is serious. An upgrade signal is generated and transmitted to the interactive analysis unit and the perception analysis unit. The interactive analysis unit and the perception analysis unit will re-upgrade the facial model MM and the limb model ZM. The matching management unit will re-compose the comprehensive model YM, monitor and analyze the updated delay time, and timely adjust the response mechanism of the interactive content to ensure the efficiency and response accuracy of the system, which is helpful to enhance the real-time interactivity of virtual reality games.

[0070] See also Figure 2 , a virtual reality game method based on visual tracking technology, comprising the following steps:

[0071] Step 1: Connect cameras and sensor devices through the network to obtain the interaction data and action performance of all users, and classify them into interaction data sets and action data sets;

[0072] Step 2: Based on the interactive dataset and the interactive dataset, construct the facial model MM of each user, mark the facial feature points and virtual lines, then analyze the change distance BJ of each facial feature point, and generate the corresponding emotion intensity Qxq;

[0073] Step 3: Set a fixed capture period P, build a limb model ZM for each user, mark the limb feature points and virtual lines, analyze the length CD of each virtual line, and generate the corresponding perception intensity Gzq. During the dynamic capture process, the head and body may produce different movement patterns due to inertia. Separate modeling helps to independently optimize the movement performance of each part, improve the overall accuracy and naturalness, avoid errors caused by movement inertia, and achieve high capture accuracy with separate modeling.

[0074] Step 4: Based on the facial model MM and the limb model ZM, a comprehensive model YM is formed for each user. Then, combined with the interaction data set and the action data set, the facial feature points and virtual lines are updated synchronously, and the delay time YCS of the synchronous update is calculated to determine the degree of delay when the user action is substituted into the comprehensive model YM. The corresponding judgment result is output, and the game character action delay and lag problems are discovered in time, the virtual model is quickly optimized and updated, and the user experience is improved through intelligent optimization.

[0075] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A virtual reality game system based on visual tracking technology, characterized in that: Including multi-dimensional acquisition module and virtual reality module; The multi-dimensional acquisition module is composed of a virtual interaction unit and a motion capture unit. The virtual interaction unit collects an interaction data set through a network connection camera, and the interaction data set includes the interaction data of all users. The motion capture unit collects an action data set through a network connection sensor device, and the action data set includes the action performance of all users. The virtual reality module is composed of an interactive analysis unit, a perception analysis unit and a matching management unit. The interactive analysis unit constructs a facial model MM of each user according to the interactive data set, and marks the facial feature points and virtual lines. The interactive analysis unit analyzes the change distance BJ of each facial feature point according to the facial model MM, and generates the corresponding emotion intensity Qxq in combination with the interactive data set. The perception analysis unit is provided with a fixed-duration capture period P, and constructs a limb model ZM of each user in combination with the action data set, and marks the limb feature points and virtual lines. The perception analysis unit analyzes the length CD of each virtual line according to the limb model ZM, and generates the corresponding perception intensity Gzq in combination with the interactive data set. The matching management unit composes a comprehensive model YM of each user according to the facial model MM and the limb model ZM, and synchronously updates the facial feature points and virtual lines in combination with the interactive data set and the action data set. The matching management unit counts the delay time YCS of the synchronous update according to the interactive data set, the action data set and the comprehensive model YM, and sets a fixed-duration delay threshold YCY to judge the degree of delay when the user action is substituted into the comprehensive model YM, and outputs the corresponding judgment result.

2. A virtual reality game system based on visual tracking technology according to claim 1, characterized in that: The expression of the interactive data set is {Y1 s 、Y2 s 、Y3 s 、...、Yn s }, Y1 s To Yn s They are the interaction data of the first to the nth user respectively. The interaction data include facial images and voice records. s represents the specific time of obtaining the interaction data of each user.

3. A virtual reality game system based on visual tracking technology according to claim 2, characterized in that: The expression of the action data set is {D1 g 、D2 g 、D3 g 、...、Dn g }, D1g to Dn g They are the action performances of the first to the nth user respectively, the action performances include hand movements, leg movements and body movements, and g represents the weight of each user.

4. A virtual reality game system based on visual tracking technology according to claim 3, characterized in that: The facial model MM construction process is as follows: S11, extract the interaction data of the i-th user in the interaction data set, and mark the facial image of the i-th user as MB i ; S12: Based on the facial image MB of the i-th user i , using 3D software to build a 1:1 facial model MM i , and in the face model MM i The facial feature points are marked in the figure, including the center point of the hairline, the center point of the left eyebrow, the center point of the right eyebrow, the center point of the left pupil, the center point of the right pupil, the tip of the nose, the endpoint of the left corner of the mouth, the endpoint of the right corner of the mouth and the endpoint of the chin. Each facial feature point corresponds to a three-dimensional coordinate, among which the center point of the hairline, the tip of the nose and the endpoint of the chin form a vertical virtual line one, and the center point of the left pupil and the center point of the right pupil form a parallel virtual line one.

5. A virtual reality game system based on visual tracking technology according to claim 4, characterized in that: The calculation process of the emotion intensity Qxq is as follows: S21, according to the facial model MM i , calculate the change distance BJ of a single facial feature point; S22, according to S21, calculate the change distance BJ of the left eyebrow center point coordinate, the right eyebrow center point coordinate, the left pupil center point coordinate, the right pupil center point coordinate, the left mouth corner endpoint coordinate and the right mouth corner endpoint coordinate; S23, extract the interaction data of the i-th user in the interaction data set, and count the voice record YU of the i-th user within one minute. i , and then calculate the interactive speech rate YS of the i-th user within one minute i , S24, based on the change distance BJ of all facial feature points and the interactive speech speed YS i , calculate the emotional intensity Qxq of the i-th user i .

6. A virtual reality game system based on visual tracking technology according to claim 5, characterized in that: The limb model ZM construction process is as follows: S31. Extract the action performance of the i-th user within the capture period P according to the action data set, and mark the hand action of the i-th user as SH i , label the leg action of the i-th user as TU i , label the body action of the i-th user as ST i ; S32: According to the hand movement SH of the i-th user i , leg movements TU i and body movements ST i , using 3D software to build a 1:1 limb model ZM i , and in the limb model ZM i Limb feature points are marked in the figure, including the left shoulder endpoint, the right shoulder endpoint, the center point of the left and right shoulders, the left hand endpoint, the right hand endpoint, the left foot endpoint and the right foot endpoint. Each limb feature point corresponds to a three-dimensional coordinate, wherein the left shoulder endpoint, the right shoulder endpoint and the center point of the left and right shoulders form a parallel virtual line 2, the left shoulder endpoint and the left hand endpoint form a vertical virtual line 2, the right shoulder endpoint and the right hand endpoint form a vertical virtual line 3, the center point of the left and right shoulders and the left foot endpoint form an inclined virtual line 1, and the center point of the left and right shoulders and the right foot endpoint form an inclined virtual line 2.

7. A virtual reality game system based on visual tracking technology according to claim 6, characterized in that: The calculation process of the perception intensity Gzq is as follows: S41, according to the limb model ZM i , calculate the length CD of a single virtual line; S42, according to S41, obtaining the lengths CD of the parallel virtual line 2, the vertical virtual line 2, the vertical virtual line 3, the inclined virtual line 1 and the inclined virtual line 2; S43. Calculate the perception intensity Gzq of the i-th user according to the length CD and the interactive data set. i .

8. A virtual reality game system based on visual tracking technology according to claim 7, characterized in that: The matching management unit sets a vertical straight line of fixed length to connect the i-th user face model MM through three-dimensional software i The jaw endpoint and limb model ZM in i The left and right shoulder center points in the ith user form the complete comprehensive model YM i , and combined with the interaction dataset and action dataset, the facial feature points and virtual lines are updated synchronously.

9. A virtual reality game system based on visual tracking technology according to claim 8, characterized in that: When the delay duration YCS exceeds the delay threshold YCY, it indicates that the delay in substituting the user action into the comprehensive model YM is serious, and an upgrade signal is generated and transmitted to the interaction analysis unit and the perception analysis unit. The interaction analysis unit and the perception analysis unit will re-upgrade the facial model MM and the limb model ZM, and the matching management unit will re-form the comprehensive model YM.

10. A virtual reality game method based on visual tracking technology, applied to a virtual reality game system based on visual tracking technology as claimed in any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Connect cameras and sensor devices through the network to obtain the interaction data and action performance of all users, and classify them into interaction data sets and action data sets; Step 2: Based on the interactive dataset and the interactive dataset, construct the facial model MM of each user, mark the facial feature points and virtual lines, then analyze the change distance BJ of each facial feature point, and generate the corresponding emotion intensity Qxq; Step 3: Set a fixed capture period P, build a limb model ZM for each user, mark limb feature points and virtual lines, analyze the length CD of each virtual line, and generate the corresponding perception intensity Gzq; Step 4: Based on the facial model MM and the limb model ZM, a comprehensive model YM of each user is formed. Then, combined with the interaction dataset and the action dataset, the facial feature points and virtual lines are synchronously updated, the delay time YCS of the synchronous update is calculated, the degree of delay when the user action is substituted into the comprehensive model YM is judged, and the corresponding judgment result is output.

Images