Multi-source virtual reality man-machine interaction method and system

Abstract

The invention relates to the technical field of image data processing, in particular to a multi-source virtual reality man-machine interaction method and a multi-source virtual reality man-machine interaction system. Predicting an area to which key feature points belong in the user gesture action image at the current moment to obtain a key feature prediction area; and according to the feature points in the key feature prediction region, identifying the gesture action of the user, and mapping the identified gesture action of the user to a virtual reality environment for interactive operation. According to the method, the calculation amount of a computer can be greatly reduced, and the accuracy of key feature points can be kept, so that high-precision gesture recognition is realized, and a user can control an interface in a virtual environment through natural hand actions.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a multi-source virtual reality human-computer interaction method and system. Background Art

[0002] With the rapid development of virtual reality (VR) technology, multimodal virtual reality human-computer interaction (HCI) systems are becoming a hot topic of research. Existing VR systems usually rely on input sources such as handles or sensors to enable users to interact with the virtual environment. However, image processing technology can provide a more intuitive and natural interactive experience by analyzing user gestures in real time, making the virtual environment more responsive to users' dynamic behaviors.

[0003] The existing technology determines the user's gesture by extracting key feature points in the user's gesture video. In order to ensure the accuracy of the key feature points, it is necessary to perform key feature point detection on each frame of the user's gesture video. This operation will greatly increase the computer's computational workload. Summary of the invention

[0004] In order to solve the technical problem of how to reduce the amount of calculation while ensuring the accuracy of key feature points, the purpose of the present invention is to provide a multi-source virtual reality human-computer interaction method and system, and the technical solution adopted is as follows:

[0005] In a first aspect, a multi-source virtual reality human-computer interaction method is provided, the method comprising: obtaining user gesture action images at several historical moments; predicting the area to which the key feature points in the user gesture action images at the current moment belong by analyzing the change trend of the key feature points in the user gesture action images at the several historical moments, and adjusting the area thereof to obtain a key feature prediction area; identifying the user's gesture actions according to the feature points in the key feature prediction area, and mapping the identified user gesture actions to the virtual reality environment for interactive operations.

[0006] This method analyzes the changing trend of key feature points in the historical video of the user's gesture action, predicts and adjusts the area to which the key feature points in the user's gesture action at the current moment belong. The user's gesture action is identified by obtaining the key feature points of the user's gesture action in real time, and the identified user gesture action is mapped to the virtual reality environment for interactive operation.

[0007] Furthermore, the user gesture action images at several historical moments are obtained, and the specific steps are: using a depth camera to capture the user's gestures and generate a user gesture action video; decomposing the user gesture action video into several frames of user gesture action images; and forming a user gesture action image set within a historical time period. The user gesture action video is decomposed into several frames of user gesture action images, and then it also includes: preprocessing the several frames of user gesture action images, and the preprocessed several frames of user gesture action images constitute a user gesture action image set within a historical time period. Preprocessing the several frames of user gesture action images specifically includes: graying each frame of user gesture action image and filtering it using a median filtering algorithm.

[0008] Furthermore, the method for obtaining the key feature prediction area is: obtaining the key feature points in the user gesture action image at each historical moment; analyzing the change trend of the key feature points in the historical time period to obtain the prediction accuracy of the prediction points of the key feature points; and determining all the key feature prediction areas in the user gesture action image at the current moment based on the prediction accuracy of the prediction points.

[0009] Furthermore, key feature points in the user gesture action image at each historical moment are obtained, specifically including: for the user gesture action image at any historical moment, using the SIFT corner point detection algorithm to perform corner point detection on the user gesture action image at the historical moment, and obtaining all corner points in the user gesture action image at the historical moment; for the tth corner point in the user gesture action image at the historical moment, taking the tth corner point as the window center, obtaining a window with a window size of 5*5, and recording the window as the neighborhood range window of the tth corner point; t is a positive integer; according to the relationship between the grayscale value of the tth corner point and the grayscale value in the neighborhood range window of the tth corner point, obtaining the possibility that the tth corner point is a key feature point; if the possibility that the tth corner point is a key feature point is greater than or equal to the key threshold parameter, then the tth corner point is recorded as the key feature point in the user gesture action image at the historical moment; traversing all historical moments in the historical time period, obtaining all key feature points in the user gesture action image at each historical moment.

[0010] Further, analyzing the change trend of the key feature points in the historical time period, and obtaining the prediction accuracy of the prediction points of the key feature points, specifically including: determining the prediction point of the jth key feature point in the user gesture action image at the Nth historical moment according to the coordinate position of the jth key feature point in the user gesture action image at the N-1th historical moment, and the predicted change direction and predicted change distance of the jth key feature point; determining the prediction point of the jth key feature point in the user gesture action image at the Nth historical moment according to the fitting error value of the fitting line of the jth key feature point, the number of all clustering clusters in the clustering result of the jth key feature point, and the data points in the mth cluster in the clustering result of the jth key feature point. The absolute value of the difference between the number of data points in the m+1th cluster in the clustering result of the jth key feature point is used to determine the complexity of the change trend of the jth key feature point; according to the complexity of the change trend of the jth key feature point, the Euclidean distance between the jth key feature point in the user gesture action image at the nth historical moment and the predicted point corresponding to the jth key feature point, the absolute value of the difference between the possibility that the jth key feature point is a key feature point and the possibility that other corner points in the neighborhood corner point set of the jth key feature point are key feature points, the prediction accuracy of the predicted point corresponding to the jth key feature point is determined; m, N and j are all positive integers.

[0011] Further, according to the prediction accuracy of the prediction point, all key feature prediction areas in the user gesture action image at the current moment are determined, specifically including: for the jth key feature point, in the user gesture action image at the current moment, the Euclidean distance between the jth key feature point and the corresponding prediction point of the jth key feature point in the user gesture action image at the Nth historical moment is used as the initial prediction window size of the jth key feature point; the product of the prediction accuracy of the corresponding prediction point of the jth key feature point and the initial prediction window size of the jth key feature point is rounded up to obtain the target prediction window size value U of the jth key feature point j ; According to the coordinate position of the jth key feature point in the user gesture action image at the Nth historical moment, as well as the predicted change direction and predicted change distance of the jth key feature point, determine the predicted point of the jth key feature point in the user gesture action image at the current moment; take the predicted point of the jth key feature point as the window center, and obtain the window size U j ×U j , and record the window as the key feature prediction area of ​​the jth key feature point; traverse all the key feature points in the user gesture action image at the current moment, and determine all the key feature prediction areas in the user gesture action image at the current moment.

[0012] Furthermore, the user's gestures are identified based on the feature points in the key feature prediction area, and the identified user gestures are mapped to the virtual reality environment for interactive operations, specifically including: for any key feature prediction area, using the SIFT corner point detection algorithm to perform corner point detection on the key feature prediction area to obtain the corner points of the key feature prediction area, identifying the user's gestures through the corner points of all the key feature prediction areas, and mapping the identified user gestures to the virtual reality environment for interactive operations.

[0013] In a second aspect, the present invention is a multi-source virtual reality human-computer interaction system, which includes: a data acquisition module: used to acquire user gesture action images at several historical moments; a key feature prediction area analysis module: used to analyze the change trend of key feature points in the user gesture action images at several historical moments, predict the area to which the key feature points in the user gesture action image at the current moment belong, and adjust the area to obtain the key feature prediction area; an interactive operation module: used to identify the user's gesture actions based on the feature points in the key feature prediction area, and map the identified user gesture actions to the virtual reality environment for interactive operations.

[0014] The present invention has the following beneficial effects:

[0015] The present invention analyzes the changing trends of key feature points in historical videos of user gesture actions, and predicts and adjusts the areas to which key feature points in the user gesture actions at the current moment belong. This can not only greatly reduce the amount of computer calculations, but also maintain the accuracy of key feature points, thereby achieving high-precision gesture recognition, allowing users to control the interface in the virtual environment through natural hand movements. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flow chart of a multi-source virtual reality human-computer interaction method provided by an embodiment of the present invention;

[0018] Figure 2 A structural diagram of a multi-source virtual reality human-computer interaction system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a multi-source virtual reality human-computer interaction method and system proposed by the present invention, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0020] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0021] The following is a detailed description of a multi-source virtual reality human-computer interaction method and system provided by the present invention in conjunction with the accompanying drawings.

[0022] See also Figure 1 , which shows a flow chart of a multi-source virtual reality human-computer interaction method provided by an embodiment of the present invention.

[0023] Step S1: Collect user gesture action images at several historical moments.

[0024] In the present invention, there are N historical moments before the current moment, n∈(1,N), and both n and N are positive integers. The nth moment represents any one of the several historical moments, and the Nth moment represents the last historical moment of the several historical moments.

[0025] The specific method for obtaining user gesture action images at several historical moments is:

[0026] 1. Use the motion capture device in the multi-source virtual reality human-computer interaction system to accurately track the movement of the user's hands, and use the depth camera to capture the user's gestures and generate a user gesture action video.

[0027] 2. Use FFmpeg software to decompose the user gesture action video into several frames of user gesture action images.

[0028] 3. Since the collected image is an RGB three-channel image and may contain a certain degree of noise, it needs to be preprocessed. The preprocessing adopted by the present invention is to first grayscale each frame of the user gesture action image to obtain a grayscale image corresponding to each user gesture action image, and then filter it using a median filtering algorithm to obtain a filtered grayscale image; all the preprocessed frames of user gesture action images are recorded as user gesture action images at historical moments, and constitute a user gesture action image set A within the historical time period, specifically:

[0029] A={A1,A2,…,A n ,…,A N}

[0030] In the formula, A n represents the user gesture action image at the nth historical moment; N represents the total number of user gesture action images at all historical moments in the user gesture action image set within the historical time period.

[0031] Step S2: By analyzing the change trend of key feature points in the user gesture action image at several historical moments, the region to which the key feature points in the user gesture action image at the current moment belong is predicted, and the area is adjusted to obtain the key feature prediction region.

[0032] The process of obtaining the key feature prediction area is:

[0033] a. Obtain the key feature points in the user's gesture action image at each historical moment.

[0034] In an image, key feature points usually appear as prominent parts of the image, such as corners, edges or texture areas, which can help identify important structures or details in the image. This also shows that key feature points are used to describe the detailed features of an image. Image details generally occur in areas of drastic changes, so by calculating the stability of the difference in grayscale values ​​between a pixel and other pixels in the local area of ​​a pixel, it is determined whether the area to which the pixel belongs is a detail area. Key feature points generally have obvious characteristics for the neighborhood area, such as local maxima or local minima, so that information is easy to retrieve and not easily lost during subsequent image capture.

[0035] The specific method for obtaining the key feature points in the user gesture action image at each historical moment is:

[0036] 1. For a user gesture action image at any historical moment, use the SIFT corner point detection algorithm to perform corner point detection on the user gesture action image at the historical moment to obtain all corner points in the user gesture action image at the historical moment.

[0037] 2. For the tth corner point in the user gesture action image at the historical moment, take the tth corner point as the window center, obtain a window with a window size of 5*5, and record the window as the neighborhood range window of the tth corner point.

[0038] 3. The calculation formula for obtaining the possibility that the tth corner point is a key feature point is:

[0039]

[0040] Where Pgt Indicates the possibility that the tth corner point is a key feature point; K t represents the number of all pixels in the neighborhood window of the tth corner point; h t represents the gray value of the tth corner point; h t,i Represents the gray value of the i-th pixel in the neighborhood window of the t-th corner point; σ t Represents the variance of the grayscale values ​​of all pixels in the neighborhood window of the t-th corner point; || represents taking the absolute value; norm() represents the linear normalization function; exp() represents the exponential function with a natural constant as the base; Represents the sum of the absolute differences between the grayscale value of the t-th corner point and the grayscale values ​​of all pixels in the neighborhood window of the t-th corner point. In order to ensure that the denominator is not zero, a tuning coefficient ∈ is added to the denominator, ∈=0.01.

[0041] 4. A key threshold parameter T1=0.75 is preset. If the possibility that the tth corner point is a key feature point is greater than or equal to the key threshold parameter T1, the tth corner point is recorded as the key feature point in the user gesture action image at the historical moment; and then all key feature points in the user gesture action image at each historical moment are obtained.

[0042] b. Analyze the changing trend of key feature points in the historical time period, obtain the predicted points of key feature points and the corresponding prediction accuracy.

[0043] The more different the position distance and direction changes of the key feature points in the historical time period are, the greater the amplitude of the user's gesture change. Therefore, the changes of the key feature points in the historical time period can be clustered according to the amplitude of the user's gesture change, and the key feature points in the last historical moment can be obtained, and the prediction points corresponding to the key feature points can be obtained. When the prediction accuracy of the key feature points corresponding to the prediction points is greater, the smaller the area of ​​the prediction area needs to be adjusted. Each key feature point corresponds to a prediction area, and the area of ​​this prediction area is related to the prediction accuracy of the key feature points corresponding to the prediction points. The greater the accuracy, the smaller the area of ​​the prediction area.

[0044] The calculation method for obtaining the prediction points of key feature points and the corresponding prediction accuracy is:

[0045] 1. Since the degree of change in the position distance and direction of the key feature points in the historical time period is more different, it means that the user's gesture change is greater, the difficulty in prediction is greater, and the larger the adjustment of the prediction area is required, so it is necessary to obtain the prediction points of the key feature points and the complexity of the change trend. The specific acquisition method is:

[0046] (1) For the jth key feature point, the position coordinates of the jth key feature point are obtained in the user gesture action images at all historical moments, and a linear fit is performed using the least squares method to obtain the fitting error value of the fitting line of the jth key feature point.

[0047] (2) The sine value of the angle between the straight line connecting the coordinate position of the jth key feature point in the user gesture action image at the nth historical moment and the coordinate position of the jth key feature point in the user gesture action image at the n+1th historical moment and the horizontal line is recorded as the change direction value of the jth key feature point in the user gesture action image at the nth historical moment; the Euclidean distance between the coordinate position of the jth key feature point in the user gesture action image at the nth historical moment and the coordinate position of the jth key feature point in the user gesture action image at the n+1th historical moment is recorded as the change distance of the jth key feature point in the user gesture action image at the nth historical moment.

[0048] (3) A two-dimensional space is constructed through the change distance and change direction values ​​of the key feature points, and the change direction value and change distance of the jth key feature point are obtained from the user gesture action images at all historical moments, and the two-dimensional space is input for clustering to obtain a number of cluster clusters; the cluster cluster to which the jth key feature point in the user gesture action image at the N-1th historical moment belongs is obtained, and recorded as the target cluster cluster; the mean of all change direction values ​​in the target cluster cluster is used as the predicted change direction of the jth key feature point; the mean of all change distances in the target cluster cluster is used as the predicted change distance of the jth key feature point; according to the coordinate position of the jth key feature point in the user gesture action image at the N-1th historical moment, as well as the predicted change direction and predicted change distance of the jth key feature point, the predicted point of the jth key feature point in the user gesture action image at the N-1th historical moment is determined.

[0049] (4) The calculation formula for obtaining the complexity of the change trend of the jth key feature point is:

[0050]

[0051] In the formula, B j Indicates the complexity of the change trend of the jth key feature point; RS j represents the fitting error value of the fitting line of the jth key feature point; M j represents the number of all clusters in the clustering result of the jth key feature point; c j,m represents the number of data points in the mth cluster in the clustering result of the jth key feature point; c j,m+1 Represents the number of data points in the m+1th cluster in the clustering result of the jth key feature point; || represents the absolute value.

[0052] 2. Since the greater the complexity of the change trend of the key feature point, the greater the difficulty of prediction, the lower the corresponding prediction accuracy; in the user gesture action image at the last historical moment, the smaller the distance between the key feature point and the corresponding prediction point of the key feature point, the greater the prediction accuracy of the corresponding prediction point; when the difference between the key feature point and all corner points in its neighborhood corner point set is greater, it means that the key feature point corresponding prediction point is more likely to be the key feature point, and the prediction accuracy is greater at this time. The calculation method for obtaining the prediction accuracy of the jth key feature point corresponding prediction point is:

[0053] (1) In the user gesture action image at the nth historical moment, other corner points are sorted from small to large according to the Euclidean distance between them and the jth key feature point, and the first 10 corner points in the sorted corner point set are used as the neighborhood corner point set of the jth key feature point.

[0054] (2) The calculation method of the prediction accuracy of the jth key feature point corresponding to the prediction point is:

[0055]

[0056] In the formula, Q j Indicates the prediction accuracy of the jth key feature point corresponding to the prediction point; L j Pg represents the number of all corner points in the neighborhood corner point set of the jth key feature point; j,l Pg represents the possibility that the lth corner point in the neighborhood corner point set of the jth key feature point is a key feature point; j Indicates the possibility that the jth key feature point is a key feature point; B j Indicates the complexity of the change trend of the jth key feature point; d N,j It represents the Euclidean distance between the jth key feature point and the corresponding prediction point of the jth key feature point in the user gesture action image at the Nth historical moment. In order to ensure that the denominator is not zero, the adjustment coefficient ∈ is added to the denominator, ∈=0.01. norm() represents the normalization function.

[0057] c. According to the prediction accuracy of the prediction point, determine all key feature prediction areas in the user gesture action image at the current moment.

[0058] The greater the prediction accuracy of the key feature point corresponding to the prediction point, the smaller the area of ​​the prediction region needs to be adjusted.

[0059] The specific method for obtaining all key feature prediction areas in the user gesture action image at the current moment is:

[0060] (1) For the jth key feature point, in the user gesture action image at the current moment, the Euclidean distance between the jth key feature point and the corresponding prediction point of the jth key feature point in the user gesture action image at the Nth historical moment is used as the initial prediction window size R of the jth key feature point. j .

[0061] (2) The specific formula for obtaining the target prediction window size value of the jth key feature point is:

[0062]

[0063] Where U j represents the target prediction window size value of the jth key feature point; Q j Indicates the prediction accuracy of the jth key feature point corresponding to the prediction point; R j Represents the initial prediction window size of the jth key feature point; Indicates rounding up.

[0064] (3) Through the above method, according to the coordinate position of the jth key feature point in the user gesture action image at the Nth historical moment, as well as the predicted change direction and predicted change distance of the jth key feature point, the predicted point of the jth key feature point in the user gesture action image at the current moment is determined; the predicted point of the jth key feature point is used as the window center, and the window size U is obtained. j ×U j , and record the window as the key feature prediction area of ​​the jth key feature point.

[0065] Step S3: identifying the user's gestures based on the feature points in the key feature prediction area, and mapping the identified user gestures to the virtual reality environment for interactive operations.

[0066] For any key feature prediction area, the SIFT corner detection algorithm is used to perform corner detection on the key feature prediction area to obtain the corner points of the key feature prediction area. The user's gestures are identified through the corner points of all key feature prediction areas, and the identified user gestures are mapped to the virtual reality environment for interactive operations.

[0067] Based on the same inventive concept as the above method embodiment, the embodiment of the present invention also provides a multi-source virtual reality human-computer interaction system, such as Figure 2 As shown, for implementing the above method embodiment, the system includes:

[0068] Data acquisition module: used to obtain user gesture action images at several historical moments.

[0069] Key feature prediction area analysis module: used to analyze the change trend of key feature points in the user gesture action image at several historical moments, predict the area to which the key feature points in the user gesture action image at the current moment belong, and adjust the area to obtain the key feature prediction area.

[0070] Interactive operation module: used to identify the user's gestures based on the feature points in the key feature prediction area, and map the identified user gestures to the virtual reality environment for interactive operations.

[0071] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0072] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A multi-source virtual reality human-computer interaction method, characterized in that: The method comprises: Obtain user gesture action images at several historical moments; By analyzing the change trend of key feature points in the user gesture action image at several historical moments, the region to which the key feature points in the user gesture action image at the current moment belong is predicted, and the area is adjusted to obtain the key feature prediction region; The user's gestures are identified based on the feature points in the key feature prediction area, and the identified user gestures are mapped to the virtual reality environment for interactive operations.

2. A multi-source virtual reality human-computer interaction method according to claim 1, characterized in that: Get user gesture action images at several historical moments. The specific steps are as follows: Using a depth camera to capture the user's gestures and generate a user gesture action video; Decompose the user gesture action video into a plurality of frames of user gesture action images; and form a set of user gesture action images within a historical time period.

3. The multi-source virtual reality human-computer interaction method according to claim 2, characterized in that: The user gesture action video is decomposed into a plurality of frames of user gesture action images, and then the method further includes: preprocessing the plurality of frames of user gesture action images, wherein the preprocessed plurality of frames of user gesture action images constitute a set of user gesture action images within a historical time period.

4. The multi-source virtual reality human-computer interaction method according to claim 3, characterized in that: Preprocessing the plurality of frames of user gesture action images specifically includes: Each frame of user gesture action image is grayed out and filtered using a median filtering algorithm.

5. The multi-source virtual reality human-computer interaction method according to claim 1, characterized in that: The method to obtain the key feature prediction area is: Obtain key feature points in the user's gesture action image at each historical moment; Analyze the change trend of the key feature points in the historical time period to obtain the prediction accuracy of the prediction points of the key feature points; According to the prediction accuracy of the prediction point, all key feature prediction areas in the user gesture action image at the current moment are determined.

6. A multi-source virtual reality human-computer interaction method according to claim 5, characterized in that: Obtain the key feature points in the user gesture action image at each historical moment, including: For a user gesture action image at any historical moment, use the SIFT corner point detection algorithm to perform corner point detection on the user gesture action image at the historical moment to obtain all corner points in the user gesture action image at the historical moment; For the tth corner point in the user gesture action image at the historical moment, take the tth corner point as the window center, obtain a window with a window size of 5*5, and record the window as the neighborhood range window of the tth corner point; t is a positive integer; According to the relationship between the gray value of the t-th corner point and the gray value in the neighborhood range window of the t-th corner point, the possibility of the t-th corner point being a key feature point is obtained; If the probability that the t-th corner point is a key feature point is greater than or equal to the key threshold parameter, the t-th corner point is recorded as the key feature point in the user gesture action image at the historical moment; Traverse all historical moments in the historical time period and obtain all key feature points in the user gesture action image at each historical moment.

7. The multi-source virtual reality human-computer interaction method according to claim 6, characterized in that: Analyze the change trend of the key feature points in the historical time period to obtain the prediction accuracy of the prediction points of the key feature points, specifically including: Determine the predicted point of the jth key feature point in the user gesture action image at the Nth historical moment according to the coordinate position of the jth key feature point in the user gesture action image at the N-1th historical moment, and the predicted change direction and predicted change distance of the jth key feature point; Determine the complexity of the change trend of the jth key feature point according to the fitting error value of the fitting straight line of the jth key feature point, the number of all clusters in the clustering result of the jth key feature point, the absolute value of the difference between the number of data points in the mth cluster in the clustering result of the jth key feature point and the number of data points in the m+1th cluster in the clustering result of the jth key feature point; Determine the prediction accuracy of the prediction point corresponding to the jth key feature point according to the complexity of the change trend of the jth key feature point, the Euclidean distance between the jth key feature point in the user gesture action image at the Nth historical moment and the prediction point corresponding to the jth key feature point in the user gesture action image at the Nth historical moment, and the absolute value of the difference between the possibility that the jth key feature point is a key feature point and the possibility that other corner points in the neighborhood corner point set of the jth key feature point are key feature points; m, N and j are all positive integers.

8. The multi-source virtual reality human-computer interaction method according to claim 7, characterized in that: According to the prediction accuracy of the prediction point, all key feature prediction areas in the user gesture action image at the current moment are determined, including: For the jth key feature point, in the user gesture action image at the current moment, the Euclidean distance between the jth key feature point and the corresponding prediction point of the jth key feature point in the user gesture action image at the Nth historical moment is used as the initial prediction window size of the jth key feature point; The product of the prediction accuracy of the prediction point corresponding to the j-th key feature point and the initial prediction window size of the j-th key feature point is rounded up to obtain the target prediction window size value U of the j-th key feature point. j ; According to the coordinate position of the jth key feature point in the user gesture action image at the Nth historical moment, as well as the predicted change direction and predicted change distance of the jth key feature point, determine the predicted point of the jth key feature point in the user gesture action image at the current moment; take the predicted point of the jth key feature point as the window center, and obtain the window size U j ×U j The window is recorded as the key feature prediction area of ​​the jth key feature point; All key feature points in the user gesture action image at the current moment are traversed to determine all key feature prediction areas in the user gesture action image at the current moment.

9. The multi-source virtual reality human-computer interaction method according to claim 1, characterized in that: The user's gestures are identified based on the feature points in the key feature prediction area, and the identified user gestures are mapped to the virtual reality environment for interactive operations, including: For any key feature prediction area, the SIFT corner detection algorithm is used to perform corner detection on the key feature prediction area to obtain the corner points of the key feature prediction area. The user's gestures are identified through the corner points of all key feature prediction areas, and the identified user gestures are mapped to the virtual reality environment for interactive operations.

10. A multi-source virtual reality human-computer interaction system, characterized in that: The system comprises: Data acquisition module: used to obtain user gesture action images at several historical moments; Key feature prediction region analysis module: used to analyze the change trend of key feature points in the user gesture action image at several historical moments, predict the region to which the key feature points in the user gesture action image at the current moment belong, and adjust the area to obtain the key feature prediction region; Interactive operation module: used to identify the user's gestures based on the feature points in the key feature prediction area, and map the identified user gestures to the virtual reality environment for interactive operations.

Images