A scenario-based interactive method and system based on virtual reality

By obtaining situational interaction purpose information in virtual reality, dividing and building interactive scenarios, collecting and reducing dimensionality processing of multi-dimensional interactive information, setting action and assessment requirements, the problem of poor virtual reality skills training is solved and efficient skills are achieved Reduced training results and costs.

CN114840074BActive Publication Date: 2024-12-06SHENZHEN TOUCH SHADOW DESIGN CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210281900.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-22
Publication Date
2024-12-06
Estimated Expiration
2042-03-22

AI Technical Summary

Technical Problem

In skill training, the situational interaction settings of virtual reality cannot fully consider the skill training requirements, resulting in high cost and poor effect of skill training.

Method used

By obtaining situational interaction purpose information, dividing multiple interactive scenarios, collecting multi-dimensional interactive information, performing dimensionality reduction processing, setting interactive actions and assessment requirements, dynamically constructing interactive scenarios, and realizing targeted interactive training.

Benefits of technology

It reduces the intensity of unnecessary interactive training movements, improves the requirements for necessary training movements, improves the effectiveness of skill training, and reduces costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114840074B_ABST
    Figure CN114840074B_ABST
Patent Text Reader

Abstract

The present invention provides a scenario-based interaction method and system based on virtual reality, wherein the method comprises: obtaining current scenario-based interaction purpose information; dividing interaction scenes according to the scenario-based interaction purpose information to obtain a plurality of interaction scenes; collecting and obtaining multi-dimensional interaction information in a plurality of interaction scenes to obtain a plurality of multi-dimensional interaction information sets; performing dimensionality reduction processing on the plurality of multi-dimensional interaction information sets to obtain a plurality of interaction information sets; setting interaction action requirement information in a plurality of interaction scenes according to the plurality of interaction information sets; setting interaction assessment requirement information in a plurality of interaction scenes according to the plurality of interaction information sets; constructing and interacting a plurality of interaction scenes according to the interaction action requirement information and the interaction assessment requirement information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a scenario-based interaction method and system based on virtual reality. Background Art

[0002] Virtual reality refers to the construction of a virtual environment through computer technology, which is similar to the information in the real environment, allowing users to experience the real world in the virtual environment through senses such as sight, hearing, and touch, and even experience a world that does not exist in reality.

[0003] Virtual reality is currently widely used in entertainment, education, training and other fields. In the field of skill training, the application of virtual reality technology is relatively mature.

[0004] Currently, when virtual reality is applied to skill training, the setting of situational interaction cannot fully consider the requirements of skill training, resulting in excessively high skill training costs or poor skill training mastery, and there are technical problems with poor skill training effects. Summary of the invention

[0005] The present application provides a scenario-based interactive method and system based on virtual reality, which is used to solve the technical problem of poor skill training effect when virtual reality is applied to skill training in the prior art.

[0006] In view of the above problems, the present application provides a scenario-based interactive method and system based on virtual reality.

[0007] The first aspect of the present application provides a situational interaction method based on virtual reality, the method comprising: obtaining current situational interaction purpose information; dividing interaction scenes according to the situational interaction purpose information to obtain multiple interaction scenes; collecting and obtaining multi-dimensional interaction information in the multiple interaction scenes to obtain multiple multi-dimensional interaction information sets; performing dimensionality reduction processing on the multiple multi-dimensional interaction information sets to obtain multiple interaction information sets; setting interaction action requirement information in the multiple interaction scenes according to the multiple interaction information sets; setting interaction assessment requirement information in the multiple interaction scenes according to the multiple interaction information sets; constructing and interacting the multiple interaction scenes according to the interaction action requirement information and the interaction assessment requirement information.

[0008] The second aspect of the present application provides a situational interaction system based on virtual reality, the system comprising: a first acquisition unit, used to acquire current situational interaction purpose information; a first processing unit, used to divide the interaction scenes according to the situational interaction purpose information, and obtain multiple interaction scenes; a second acquisition unit, used to collect and acquire multi-dimensional interaction information in the multiple interaction scenes, and obtain multiple multi-dimensional interaction information sets; a second processing unit, used to perform dimensionality reduction processing on the multiple multi-dimensional interaction information sets, and obtain multiple interaction information sets; a third processing unit, used to set interaction action requirement information in the multiple interaction scenes according to the multiple interaction information sets; a fourth processing unit, used to set interaction assessment requirement information in the multiple interaction scenes according to the multiple interaction information sets; a first execution unit, used to construct and interact with the multiple interaction scenes according to the interaction action requirement information and the interaction assessment requirement information.

[0009] The third aspect of the present application provides a situational interactive system based on virtual reality, comprising: a processor, the processor is coupled to a memory, the memory is used to store a program, when the program is executed by the processor, the system executes the steps of the method described in the first aspect.

[0010] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0011] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0012] The technical solution provided by the embodiment of the present application obtains information related to the skill training purpose of the current virtual reality scenario-based interaction, divides multiple interactive scenes, collects multi-dimensional interactive information in multiple interactive scenes, reduces the dimensionality of its multi-dimensional interactive information, obtains multiple interactive information sets, and sets the requirement information of interactive actions in multiple interactive scenes according to the multiple interactive information sets, and sets the assessment requirement information of the interaction in multiple interactive scenes according to the multiple interactive information sets, and then constructs each interactive scene according to the action requirement information and the assessment requirement information in the multiple interactive scenes, and performs scenario-based interaction. The embodiment of the present application obtains multiple interactive scenes according to the purpose information of the current virtual reality scenario-based interaction, can perform interactive training of different actions in each scene, and collects multi-dimensional interactive information in each scene for dimensionality reduction, can reduce the data dimension while retaining the data characteristics of the interactive information, thereby improving the efficiency of constructing each interactive scene, and dynamically and specifically sets the interactive action requirements and assessment requirements in each interactive scene according to the interactive information set, can reduce the intensity of unnecessary interactive training actions, and improve the requirements of necessary interactive training actions, thereby achieving the technical effect of reducing the cost of skill training and improving the effect of scenario-based interactive skill training.

[0013] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of a scenario-based interactive method based on virtual reality provided in this application;

[0015] Figure 2 A schematic diagram of a flow chart of obtaining multiple interactive action requirement information in a scenario-based interactive method based on virtual reality provided in the present application;

[0016] Figure 3 A flowchart of obtaining multiple interactive assessment requirement information in a scenario-based interactive method based on virtual reality provided in this application;

[0017] Figure 4 A schematic diagram of a scenario-based interactive system structure based on virtual reality is provided for this application;

[0018] Figure 5 This is a schematic diagram of the structure of an exemplary electronic device of the present application.

[0019] Explanation of the figure marks: first obtaining unit 11, first processing unit 12, second obtaining unit 13, second processing unit 14, third processing unit 15, fourth processing unit 16, first execution unit 17, electronic device 300, memory 301, processor 302, communication interface 303, bus architecture 304. DETAILED DESCRIPTION

[0020] The present application provides a scenario-based interactive method and system based on virtual reality, which is used to solve the technical problem of poor skill training effect when virtual reality is applied to skill training in the prior art.

[0021] Application Overview

[0022] Virtual reality refers to the construction of a virtual environment through computer technology, which is similar to the information in the real environment, allowing users to experience the real world in the virtual environment through senses such as sight, hearing, and touch, and even experience a world that does not exist in reality.

[0023] Virtual reality is currently widely used in entertainment, education, training and other fields. In the field of skill training, the application of virtual reality technology is relatively mature. Specifically, virtual reality technology is used to build a simulated virtual environment, allowing users to perform purposeful actions or mental interactions in it to achieve the purpose of skill training. The application of virtual reality in skill training is mainly distributed in the fields of driving, medical treatment, sports, etc.

[0024] Currently, when virtual reality is applied to skill training, the setting of situational interaction cannot fully consider the requirements of skill training, resulting in excessively high skill training costs or poor skill training mastery, and there are technical problems with poor skill training effects.

[0025] In response to the above technical problems, the overall idea of ​​the technical solution provided by this application is as follows:

[0026] The technical solution provided by the embodiment of the present application obtains information related to the skill training purpose of the current virtual reality for situational interaction, divides into multiple interactive scenes, collects multi-dimensional interactive information in multiple interactive scenes, reduces the dimensionality of the multi-dimensional interactive information, obtains multiple interactive information sets, and sets requirement information for interactive actions in multiple interactive scenes according to the multiple interactive information sets, and sets assessment requirement information for interactions in multiple interactive scenes according to the multiple interactive information sets, then constructs each interactive scene according to the action requirement information and the assessment requirement information in the multiple interactive scenes, and conducts situational interaction.

[0027] After introducing the basic principles of the present application, the technical solutions in the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described here. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0028] Embodiment 1

[0029] like Figure 1 As shown, the present application provides a scenario-based interactive method based on virtual reality, the method comprising:

[0030] S100: Obtaining current situational interaction purpose information;

[0031] In the embodiment of the present application, when it is necessary to construct and interact with a virtual reality scenario-based interactive environment, it is necessary to construct the interactive environment according to the current scenario-based interactive purpose.

[0032] In the embodiment of the present application, the purpose of the virtual reality scenario interaction is skill training. Specifically, the user can experience the information in the virtual scenario through multiple senses and make interactive actions to achieve skill training. For example, the specific skill training purpose may include the training of driving skills, sports movements, medical skills, etc.

[0033] The situational interaction purpose information of the current virtual reality interactive skill training is obtained. Optionally, the situational interaction purpose information can be the above-mentioned driving skills, sports movements, medical skills and other skill training purposes, and can also be other virtual reality interaction purposes in the prior art.

[0034] S200: Dividing the interactive scenes according to the scenario-based interactive purpose information to obtain a plurality of interactive scenes;

[0035] In the embodiment of the present application, the above-mentioned scenario-based interactive purpose information generally includes multiple interactive scenes. For example, if the scenario-based interactive purpose information is driving skill training, it includes multiple vehicles, multiple road conditions, and multiple driving situations such as reversing, turning, and other interactive scenes.

[0036] Based on multiple interactive scenes with different interactive purposes, the interactive scenes are divided according to the situational interactive purpose information to obtain multiple interactive scenes. Specific and different interactions can be carried out in the multiple interactive scenes to achieve the interactive purposes in different specific scenes.

[0037] S300: Collect and obtain multi-dimensional interaction information in the multiple interaction scenes to obtain multiple multi-dimensional interaction information sets;

[0038] Specifically, different interactive scenes include a variety of different situational interactive actions. For example, in different interactive scenes for driving skill training, there are interactive actions corresponding to steering, reversing, deceleration, acceleration, etc. Each interactive action requires the user to perform different actions in the virtual reality environment, such as steering angle, braking amplitude, etc. These actions can be detected and converted into information through contact devices in the virtual reality, and reflected in the virtual environment to form interactive information. The virtual environment gives feedback based on the interactive information, thereby realizing interaction. For complex interactive actions in more complex interactive scenes, multi-dimensional interactive information is included.

[0039] In this way, multi-dimensional interaction information in multiple interaction scenes is collected respectively. Exemplarily, the multi-dimensional interaction information set in multiple interaction scenes includes information such as action type, action amplitude and action time of multiple different interaction actions.

[0040] S400: performing dimensionality reduction processing on the multiple multi-dimensional interaction information sets to obtain multiple interaction information sets;

[0041] Specifically, since the amount of interactive information in the above-mentioned multi-dimensional interactive information set is relatively large, for example, the same action can be completed through multiple similar actions of different amplitudes and speeds, and the interactive information corresponding to these similar actions is different, if an interactive scene is constructed based on all the interactive information, the data processing and construction work will be too complicated.

[0042] Therefore, in the embodiment of the present application, the data in the multiple multi-dimensional interactive information sets are respectively subjected to dimensionality reduction processing to reduce the data dimension while retaining the characteristics of the interactive information to obtain the above multiple interactive information sets. In this way, the workload of data processing and construction can be reduced, the cost can be reduced, and the efficiency can be improved.

[0043] Optionally, the method for performing dimensionality reduction processing on data in multiple multi-dimensional interactive information sets in the embodiments of the present application may include principal component analysis, factor analysis, or other nonlinear dimensionality reduction algorithms.

[0044] S500: setting interaction action requirement information in the plurality of interaction scenes according to the plurality of interaction information sets;

[0045] In the embodiment of the present application, based on the above-mentioned multiple interaction information sets, different interaction actions that need to be performed in each interaction scene and interaction information when different interaction actions are performed can be obtained.

[0046] In the process of skill training based on virtual reality, the virtual environment needs to determine whether the user has completed the current interactive action and make corresponding feedback. The basis for judgment is whether the user performs the interactive action and whether the interactive action is completed. Generally, the interactive action requirement information can be set as the basis for judgment when constructing the interactive scene. Specifically, the interactive action requirement information can be set to determine whether the user performs the interactive action and whether the interactive information corresponding to the interactive action satisfies the corresponding interactive action requirement information to obtain the judgment result.

[0047] Specifically, according to the above-mentioned multiple interactive information sets, according to the interactive information of each interactive action in each interactive information set, interactive action requirement information that meets the skill training requirements is set, and interactive action requirement information in multiple interactive scenes is obtained. In this way, when the user makes an interactive action in a certain interactive scene, it can be judged whether the interactive action meets the requirements according to the corresponding interactive action requirement information, and then corresponding feedback is given.

[0048] S600: setting interaction assessment requirement information in the plurality of interaction scenarios according to the plurality of interaction information sets;

[0049] Specifically, when conducting interactive skill training based on virtual reality, after the training in a certain interactive scene is completed, the user's training can generally be assessed and scored based on the overall level of the user's interactive actions, so that the user can understand his or her training level in this skill.

[0050] In an embodiment of the present application, based on the types of interactive actions in the aforementioned multiple interactive information sets and the interactive information corresponding to each interactive action, an analysis is performed to obtain the level of interactive information corresponding to the interactive actions actually completed by the user on the basis of satisfying the required information for the interactive action.

[0051] For example, if a certain interactive action is a turn, after the interactive turn action made by the user satisfies the interactive action requirement information, a certain interactive information threshold is set as the interactive assessment requirement information to analyze whether the interactive information of the user's turn action is excellent. The interactive assessment requirement information can be specifically set through interactive information data of dimensions such as turning speed and accuracy, and corresponds one-to-one with the data type in the interactive information set.

[0052] By setting different interaction assessment requirement information in multiple interaction scenes according to the interactive information set after dimensionality reduction, it is possible to evaluate the completion status after the user conducts skill training and completes the corresponding actions, which can then be used as reference data for the quality of skill training and improve the effect of virtual reality skill training.

[0053] S700: constructing and interacting the plurality of interactive scenes according to the interactive action requirement information and the interactive assessment requirement information.

[0054] Based on the interactive action requirement information, interactive assessment requirement information and the interactive information set in each interactive scene in the aforementioned content, multiple interactive scenes are constructed, and then a virtual reality training scene for skill training is obtained based on the synthesis of multiple interactive scenes for users to experience, interact and train.

[0055] The embodiment of the present application obtains multiple interactive scenes according to the purpose information of the current virtual reality scenario-based interaction, and can perform interactive training of different actions in each scene, and collect multi-dimensional interactive information in each scene for dimensionality reduction, which can reduce the data dimension while retaining the data characteristics of the interactive information, thereby improving the efficiency of constructing each interactive scene, and dynamically and specifically setting the interactive action requirements and assessment requirements in each interactive scene according to the interactive information set, which can reduce the intensity of unnecessary interactive training actions and improve the requirements for necessary interactive training actions, thereby achieving the technical effect of reducing the cost of skill training and improving the effect of scenario-based interactive skill training.

[0056] Step S200 in the method provided in the embodiment of the present application includes:

[0057] S210: Obtaining interactive skill training information according to the situational interactive purpose information;

[0058] S220: Dividing the interactive skill training information according to the interactive skill scenarios to obtain a plurality of interactive skill scenarios;

[0059] S230: Obtain the multiple interactive scenes according to the multiple interactive skill scenes.

[0060] Specifically, according to the skills that need to be trained corresponding to the current situational interaction purpose information, interactive skill training information is obtained, which includes information such as the skills that need to be trained and the training content of the skills in different scenarios.

[0061] Furthermore, according to the training content in different scenarios in the interactive skill training information, the interactive skill training information is divided according to the interactive skill scenarios to obtain multiple interactive skill scenarios. For example, if the interactive skill training information includes vehicle driving skill training, the multiple interactive skill scenarios include different vehicle models, different road conditions, different driving operations, etc.

[0062] According to the multiple interactive skill scenes, different interactive skill training needs to be carried out in different virtual scenes, so a plurality of interactive scenes are divided accordingly, which correspond one to one with the interactive skill scenes, and skill training is carried out in a variety of different specific scenes.

[0063] The embodiment of the present application divides a plurality of interactive scenes according to the skill training scenes within the virtual reality scenario-based interactive purpose information, and can perform training of different specific types of skills in each interactive scene. It can also dynamically design skill training requirements based on the skills within the scene, thereby achieving dynamic adjustment of learning intensity and assessment requirements for skill training.

[0064] Step S400 in the method provided in the embodiment of the present application includes:

[0065] S410: Decentralizing the plurality of multi-dimensional interactive information sets respectively;

[0066] S420: Calculate and obtain a plurality of first covariance matrices for the plurality of multi-dimensional interaction information sets obtained through processing;

[0067] S430: Calculate the first covariance matrices to obtain a plurality of first eigenvalues ​​and a plurality of first eigenvectors respectively;

[0068] S440: Projecting the multiple multi-dimensional interaction information sets onto the multiple first feature vectors respectively to obtain the multiple interaction information sets.

[0069] In the embodiment of the present application, principal component analysis is preferably used for data dimension reduction. The multi-dimensional interactive information in the above-mentioned multiple multi-dimensional interactive information sets is digitized, and for example, specific operations in the driving skill operation are numerically marked.

[0070] Then, the data in the processed multiple multi-dimensional interactive information sets are decentralized respectively. First, the mean of the interactive information data of each dimension in each multi-dimensional interactive information set is solved, and then the mean is subtracted from all the data to complete the decentralization of the interactive information data of each dimension.

[0071] The covariance formula is used to calculate the interactive information data of each dimension in each of the multiple multi-dimensional interactive information sets that have been processed, and multiple first covariance matrices are calculated respectively. Then, the first eigenvalue and the first eigenvector of each first covariance matrix are calculated through matrix operation.

[0072] Based on the multiple first eigenvectors and first eigenvalues ​​obtained by calculation, the largest first K first eigenvalues ​​and the corresponding first eigenvectors are selected, where K is a positive integer, and the data in the multiple decentralized multi-dimensional interactive information sets are projected onto the K first eigenvectors to obtain multiple interactive information sets after data dimensionality reduction.

[0073] The embodiment of the present application adopts the principal component analysis data dimensionality reduction method to perform dimensionality reduction processing on multiple multi-dimensional interactive information sets to obtain multiple interactive information sets. It can remove redundant data in the multi-dimensional interactive information sets, specifically remove multiple groups of complicated interactive information in the training process of the same skill step, and retain the characteristics of the interactive information corresponding to each skill step, which will reduce the amount of subsequent data loss, improve the efficiency of interactive scene construction and reduce data processing workload.

[0074] like Figure 2 As shown, step S500 in the method provided in the embodiment of the present application includes:

[0075] S510: performing weight allocation according to the importance of the plurality of interaction scenarios to the realization of the scenario-based interaction purpose information, and obtaining a first weight allocation result;

[0076] S520: setting a plurality of interactive actions according to the plurality of interactive information sets;

[0077] S530: extracting action requirement information according to the plurality of interactive actions to obtain a plurality of preliminary action requirement information;

[0078] S540: According to the first weight distribution result, the plurality of preliminary action requirement information are adjusted respectively to obtain the plurality of interactive action requirement information.

[0079] Specifically, when conducting interactive skill training through virtual reality, the importance of interactive skills in different interactive scenes to the skill training corresponding to the situational interactive purpose information is different. For example, in vehicle driving skill training, the importance of reverse skill training is greater than the importance of acceleration training during stable driving. Correspondingly, the interactive scene of reverse skill training is more important to the realization of situational interactive purpose information than the interactive scene of acceleration training during stable driving.

[0080] In this way, according to the importance of multiple interactive scenes to the realization of situational interactive purpose information, weight distribution is performed to obtain the weight value of each interactive scene, and then obtain the first weight distribution result.

[0081] According to the aforementioned multiple interactive information sets and corresponding interactive scenes, the interactive actions corresponding to the interactive information sets in each interactive scene are obtained. For example, if an interactive information set includes turning speed, turning time, turning accuracy, etc., then an interactive action in the corresponding interactive scene is turning.

[0082] Furthermore, based on the various interactive information of the interactive action, action requirement information is extracted. For example, interactive information that meets the training requirements can be extracted from the steering speed, steering time, and steering accuracy information, and used as preliminary action requirement information. When the training action meets the preliminary action requirement information, the virtual reality scene will give feedback that the action training is completed, otherwise it will give feedback that it is incomplete.

[0083] Furthermore, based on the aforementioned multiple preliminary action requirement information of the multiple interactive actions and the first weight allocation result, it can be considered that the interactive actions in the interactive scenes with greater importance for the realization of the situational interactive purpose information need to be performed more standardly during training before they can be considered completed, and the interactive action requirements of the interactive actions in the interactive scenes with larger weight values ​​are more stringent. Therefore, the weight values ​​in the first weight allocation result are used to adjust the preliminary action requirement information of the interactive actions in each interactive scene.

[0084] During the specific adjustment process, if the weight value of an interactive scene is large, the preliminary action requirement information of the interactive action in the interactive scene will be adjusted in a more stringent direction, otherwise it will be adjusted in a simpler direction. The adjustment range is set according to the specific size of the weight value and the actual business. After the adjustment is completed, multiple of the above-mentioned interactive action requirement information are obtained.

[0085] The embodiments of the present application distribute weights according to the importance of different interactive scenarios to the achievement of the purpose of skill-based situational interactive training, and use the weight distribution results to adjust the strictness of the action requirement information of the interactive actions in each interactive scenario. This can dynamically and adaptively set the interactive action requirements, thereby enabling users to reduce the training intensity of unimportant actions during interactive training and strengthen the training requirements for more important actions, thereby achieving better training results.

[0086] like Figure 3 As shown, step S600 in the method provided in the embodiment of the present application includes:

[0087] S610: extracting action requirement information intervals according to the plurality of preliminary action requirement information to obtain a plurality of action requirement information intervals;

[0088] S620: setting a plurality of preliminary assessment requirement thresholds according to the plurality of action requirement information intervals;

[0089] S630: According to the first weight distribution result, the plurality of preliminary assessment requirement thresholds are adjusted within the action requirement information interval to obtain a plurality of interactive assessment requirement thresholds;

[0090] S640: Combine the plurality of interaction assessment requirement thresholds to obtain a plurality of interaction assessment requirement information respectively.

[0091] Specifically, according to the aforementioned multiple preliminary action requirement information, which includes action requirements for determining whether different interactive actions in multiple interactive scenes are completed during training, the multiple preliminary action requirement information are subjected to action requirement information interval extraction. Optionally, each preliminary action requirement information includes several action requirement thresholds, such as steering operation, including thresholds such as steering speed and steering accuracy, and the action requirement information near the several action requirement thresholds forms an action requirement information interval.

[0092] Within the action requirement information interval, if it is less than the action requirement threshold, it can be considered that the corresponding interactive action is not completed; if it is greater than the action requirement threshold, it can be considered that the corresponding interactive action is completed; if it is greater than the action requirement threshold to a large extent, it can be considered that the interactive action is completed well.

[0093] In this way, according to multiple action requirement information intervals, multiple data that are larger than the action requirement threshold are selected as multiple preliminary assessment requirement thresholds. If the assessment requirement threshold is reached when the interactive action is completed, the action requirement information is exceeded to a large extent, and the interactive action is completed well. The assessment requirement threshold can be set according to the actual difficulty of the interactive action.

[0094] Furthermore, based on the multiple preliminary assessment requirement thresholds and the aforementioned first weight allocation result, the multiple preliminary assessment requirement thresholds are adjusted, specifically, the adjustment is performed within the action requirement information interval where each preliminary assessment requirement threshold is located.

[0095] In the specific adjustment process, if the weight value corresponding to a certain interactive scene is large, the preliminary assessment requirement threshold of the interactive action in the interactive scene is adjusted to a more stringent degree, that is, adjusted to a larger value in the corresponding action requirement information interval, and, conversely, the preliminary assessment requirement threshold of the interactive action in the interactive scene is adjusted to a more recommended degree, that is, adjusted to a smaller value in the corresponding action requirement information interval, but still greater than the preliminary action requirement information in the action requirement information interval. The specific degree of adjustment can be set according to the weight value and actual business needs.

[0096] In this way, the adjustment of multiple preliminary assessment requirement thresholds is completed, and the interaction assessment requirement thresholds of multiple interactive actions are obtained. Then, the interaction assessment requirement information in the above-mentioned multiple interaction scenarios is obtained by combining the multiple interaction assessment requirement thresholds in each interaction scenario.

[0097] The embodiment of the present application sets a preliminary assessment requirement threshold within the action requirement information interval of the interactive action, and adjusts the preliminary assessment requirement threshold of the interactive action in each interactive scene according to the first weight distribution result. This can conduct a more stringent assessment of the quality of interactive actions in more important interactive scenes, thereby enhancing users' interest in interactive skill training and further improving the effect of virtual reality scenario-based interactive training.

[0098] Step S510 in the method provided in the embodiment of the present application includes:

[0099] S511: constructing and obtaining a weight distribution model, wherein the weight distribution model includes a plurality of weight distribution channels;

[0100] S512: inputting the plurality of interaction scenarios into the plurality of weight allocation channels, each of the weight allocation channels performing weight allocation according to the importance of the plurality of interaction scenarios to the realization of the scenario-based interaction purpose information, to obtain a plurality of weight allocation results;

[0101] S513: Calculating weight values ​​of the plurality of interaction scenes according to the plurality of weight allocation results;

[0102] S514: Obtain the first weight distribution result according to the result of the weight value calculation.

[0103] Wherein, step S511 includes:

[0104] S511-1: According to the scenario-based interaction purpose information, multiple weight distribution subjects are set and obtained;

[0105] S511-2: Constructing a plurality of information-isolated weight allocation channels according to the plurality of weight allocation subjects;

[0106] S511-3: Merge multiple weight distribution channels, and construct an input layer and an output layer to obtain the weight distribution model.

[0107] Specifically, a weight allocation model is constructed according to the current weight allocation requirements. Based on the training skill types corresponding to the above-mentioned situational interaction purpose information, multiple weight allocation subjects are set.

[0108] Specifically, each weight allocation subject is an expert in the field of the training skill type corresponding to the scenario-based interaction purpose information, and each weight allocation subject can allocate weights for multiple interactive scenes in the scenario-based interaction purpose information based on the expert survey weight method. For example, if the scenario-based interaction purpose information corresponds to the training skill type of vehicle driving, multiple weight allocation subjects can be driving skill training or driving field experts.

[0109] According to multiple weight allocation subjects, multiple information-isolated weight allocation channels are constructed respectively. Each weight allocation subject can perform independent weight allocation in multiple weight allocation channels without being affected by other weight allocation subjects.

[0110] The plurality of weight distribution channels are combined, and the input layer and output layer of the model are constructed to obtain a weight distribution model. When performing weight distribution, it is only necessary to input a plurality of interactive scenes that need to be weighted into the weight distribution model, and then the weight distribution and related calculations can be performed and the corresponding weight distribution results can be output.

[0111] The above-mentioned multiple interaction scenarios are input into multiple weight allocation channels, wherein the weight allocation subjects in each weight allocation channel can allocate weights according to the importance of the multiple interaction scenarios to the realization of the situational interaction purpose information, and obtain multiple weight allocation results.

[0112] According to the multiple weight distribution results, the weight average of each interactive scene in the multiple weight distribution results is calculated to obtain the weight value of each interactive scene, and then the weight values ​​of all interactive scenes are combined to obtain the first weight distribution result.

[0113] The embodiment of the present application constructs a weight distribution model, sets up multiple information-isolated weight distribution channels, and performs independent weight distribution on multiple interactive scenarios, thereby avoiding the bias that may occur in separate weight distribution and making the weight distribution more accurate and fair.

[0114] To summarize, the embodiments of the present application obtain multiple interactive scenes according to the purpose information of the current virtual reality scenario-based interaction, and can perform interactive training of different actions in each scene, and collect multi-dimensional interactive information in each scene for dimensionality reduction, which can reduce the data dimension while retaining the data characteristics of the interactive information, thereby improving the efficiency of constructing each interactive scene, and assigning weights to each interactive scene. According to the weight assignment results and the interactive information set, the interactive action requirements and assessment requirements in each interactive scene are dynamically and specifically set, which can reduce the intensity of unnecessary interactive training actions, and increase the requirements for necessary interactive training actions, thereby enhancing the user's interest in virtual reality experience training, thereby achieving the technical effect of reducing the cost of skill training and improving the effect of scenario-based interactive skill training.

[0115] Embodiment 2

[0116] Based on the same inventive concept as the scenario-based interactive method based on virtual reality in the aforementioned embodiment, Figure 4 As shown, the present application provides a scenario-based interactive system based on virtual reality, wherein the system comprises:

[0117] A first obtaining unit 11 is used to obtain current situational interaction purpose information;

[0118] A first processing unit 12 is used to divide the interactive scenes according to the scenario-based interactive purpose information to obtain multiple interactive scenes;

[0119] A second obtaining unit 13 is used to collect and obtain the multi-dimensional interaction information in the multiple interaction scenes to obtain multiple multi-dimensional interaction information sets;

[0120] A second processing unit 14 is used to perform dimensionality reduction processing on the multiple multi-dimensional interaction information sets to obtain multiple interaction information sets;

[0121] The third processing unit 15 is used to set the interactive action requirement information in the multiple interactive scenes according to the multiple interactive information sets;

[0122] The fourth processing unit 16 is used to set the interaction assessment requirement information in the plurality of interaction scenarios according to the plurality of interaction information sets;

[0123] The first execution unit 17 is used to construct and interact with the multiple interactive scenes according to the interactive action requirement information and the interactive assessment requirement information.

[0124] Furthermore, the system also includes:

[0125] A third obtaining unit is used to obtain interactive skill training information according to the situational interactive purpose information;

[0126] A fifth processing unit, configured to divide the interactive skill training information according to the interactive skill scenarios to obtain a plurality of interactive skill scenarios;

[0127] The fourth obtaining unit is used to obtain the multiple interactive scenes according to the multiple interactive skill scenes.

[0128] Furthermore, the system also includes:

[0129] a sixth processing unit, configured to perform decentralized processing on the plurality of multi-dimensional interactive information sets respectively;

[0130] A seventh processing unit, configured to calculate and obtain a plurality of first covariance matrices for the plurality of multi-dimensional interaction information sets obtained by processing;

[0131] an eighth processing unit, configured to operate on the plurality of first covariance matrices to obtain a plurality of first eigenvalues ​​and a plurality of first eigenvectors respectively;

[0132] The fifth obtaining unit is used to project the multiple multi-dimensional interaction information sets onto the multiple first feature vectors respectively to obtain the multiple interaction information sets.

[0133] Furthermore, the system also includes:

[0134] A ninth processing unit, configured to perform weight allocation according to the importance of the plurality of interaction scenarios to the realization of the scenario-based interaction purpose information, and obtain a first weight allocation result;

[0135] a tenth processing unit, configured to set a plurality of interactive actions according to the plurality of interactive information sets;

[0136] a sixth obtaining unit, configured to extract action requirement information according to the plurality of interactive actions, and obtain a plurality of preliminary action requirement information;

[0137] The eleventh processing unit is used to adjust the plurality of preliminary action requirement information respectively according to the first weight distribution result to obtain the plurality of interactive action requirement information.

[0138] Furthermore, the system also includes:

[0139] a seventh obtaining unit, configured to extract action requirement information intervals according to the plurality of preliminary action requirement information, and obtain a plurality of action requirement information intervals;

[0140] A twelfth processing unit, configured to set a plurality of preliminary assessment requirement thresholds according to the plurality of action requirement information intervals;

[0141] A thirteenth processing unit, configured to adjust the plurality of preliminary assessment requirement thresholds within the action requirement information interval according to the first weight allocation result, to obtain a plurality of interactive assessment requirement thresholds;

[0142] The eighth obtaining unit is used to combine the multiple interaction assessment requirement thresholds to obtain multiple pieces of interaction assessment requirement information respectively.

[0143] Furthermore, the system also includes:

[0144] A first construction unit is used to construct a weight distribution model, wherein the weight distribution model includes a plurality of weight distribution channels;

[0145] A fourteenth processing unit is used to input the plurality of interaction scenes into the plurality of weight allocation channels, each of the weight allocation channels performs weight allocation according to the importance of the plurality of interaction scenes to the realization of the scenario-based interaction purpose information, and obtain a plurality of weight allocation results;

[0146] A fifteenth processing unit, configured to calculate weight values ​​of the plurality of interaction scenes according to the plurality of weight allocation results;

[0147] The ninth obtaining unit is used to obtain the first weight distribution result according to the result of the weight value calculation.

[0148] Furthermore, the system also includes:

[0149] A tenth obtaining unit, configured to obtain a plurality of weight distribution subjects according to the scenario-based interaction purpose information;

[0150] A second construction unit is used to construct a plurality of information-isolated weight distribution channels according to the plurality of weight distribution subjects;

[0151] The third construction unit is used to merge the multiple weight distribution channels and construct an input layer and an output layer to obtain the weight distribution model.

[0152] Embodiment 3

[0153] Based on the same inventive concept as the scenario-based interactive method based on virtual reality in the aforementioned embodiment, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method in embodiment one is implemented.

[0154] Exemplary Electronic Devices

[0155] Reference below Figure 5 To describe the electronic device of this application,

[0156] Based on the same inventive concept as the scenario-based interactive method based on virtual reality in the aforementioned embodiment, the present application also provides a scenario-based interactive system based on virtual reality, including: a processor, the processor is coupled to a memory, the memory is used to store a program, when the program is executed by the processor, the system executes the steps of the method described in embodiment one.

[0157] The electronic device 300 includes: a processor 302, a communication interface 303, and a memory 301. Optionally, the electronic device 300 may also include a bus architecture 304. The communication interface 303, the processor 302, and the memory 301 may be interconnected via the bus architecture 304; the bus architecture 304 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus architecture 304 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0158] The processor 302 may be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the present application.

[0159] The communication interface 303 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), wired access networks, etc.

[0160] The memory 301 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus architecture 304. The memory may also be integrated with the processor.

[0161] The memory 301 is used to store computer-executable instructions for executing the solution of the present application, and the execution is controlled by the processor 302. The processor 302 is used to execute the computer-executable instructions stored in the memory 301, thereby realizing a scenario-based interactive method based on virtual reality provided in the above embodiment of the present application.

[0162] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0163] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present application and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application is intended to include these modifications and variations.

Claims

1. A scenario-based interactive method based on virtual reality, characterized in that: The method comprises: Get the current situational interaction purpose information; According to the situational interaction purpose information, the interaction scenes are divided to obtain multiple interaction scenes; Collecting and acquiring multi-dimensional interaction information in the multiple interaction scenes to obtain multiple multi-dimensional interaction information sets; Performing dimensionality reduction processing on the multiple multi-dimensional interaction information sets to obtain multiple interaction information sets; According to the plurality of interaction information sets, setting interaction action requirement information within the plurality of interaction scenes; According to the plurality of interaction information sets, setting interaction assessment requirement information in the plurality of interaction scenarios; Constructing and interacting the plurality of interactive scenes according to the interactive action requirement information and the interactive assessment requirement information; The step of setting the interactive action requirement information in the multiple interactive scenes according to the multiple interactive information sets includes: According to the importance of the plurality of interaction scenarios to the realization of the scenario-based interaction purpose information, weight allocation is performed to obtain a first weight allocation result; According to the plurality of interaction information sets, setting a plurality of interaction actions; Extracting action requirement information according to the plurality of interactive actions to obtain a plurality of preliminary action requirement information; According to the first weight distribution result, the plurality of preliminary action requirement information are adjusted respectively to obtain the plurality of interactive action requirement information; The step of setting the interaction assessment requirement information in the plurality of interaction scenarios according to the plurality of interaction information sets includes: Extracting action requirement information intervals according to the plurality of preliminary action requirement information to obtain a plurality of action requirement information intervals; According to the plurality of action requirement information intervals, setting a plurality of preliminary assessment requirement thresholds; According to the first weight distribution result, the plurality of preliminary assessment requirement thresholds are adjusted within the action requirement information interval to obtain a plurality of interactive assessment requirement thresholds; Combining a plurality of the interaction assessment requirement thresholds to obtain a plurality of the interaction assessment requirement information respectively; The weight allocation is performed according to the importance of the plurality of interaction scenarios to the realization of the scenario-based interaction purpose information, including: Constructing a weight distribution model, wherein the weight distribution model includes a plurality of weight distribution channels; Inputting the plurality of interaction scenarios into the plurality of weight allocation channels, each of the weight allocation channels performs weight allocation according to the importance of the plurality of interaction scenarios to the realization of the scenario-based interaction purpose information, to obtain a plurality of weight allocation results; Calculating weight values ​​of the plurality of interactive scenes according to the plurality of weight allocation results; Obtaining the first weight distribution result according to a result of calculating the weight value; The step of constructing a weight distribution model includes: According to the situational interaction purpose information, multiple weight distribution subjects are set and obtained; Constructing a plurality of information-isolated weight distribution channels according to the plurality of weight distribution subjects; The weight distribution model is obtained by merging a plurality of the weight distribution channels and constructing an input layer and an output layer.

2. The method according to claim 1, characterized in that The dividing of the interactive scenes according to the scenario-based interactive purpose information includes: Obtaining interactive skill training information according to the situational interactive purpose information; According to the interactive skill training information, the interactive skill scenes are divided according to the interactive skill scenes to obtain a plurality of interactive skill scenes; According to the multiple interactive skill scenarios, the multiple interactive scenarios are obtained.

3. The method according to claim 1, characterized in that The performing dimensionality reduction processing on the plurality of multi-dimensional interactive information sets includes: Decentralizing the plurality of multi-dimensional interactive information sets respectively; For the plurality of multi-dimensional interaction information sets obtained by processing, respectively calculating a plurality of first covariance matrices; Performing operations on the plurality of first covariance matrices to obtain a plurality of first eigenvalues ​​and a plurality of first eigenvectors respectively; The multiple multi-dimensional interaction information sets are projected onto the multiple first feature vectors respectively to obtain the multiple interaction information sets.

4. A situational interactive system based on virtual reality, characterized in that: For executing the method according to any one of claims 1 to 3, the system comprises: A first obtaining unit, used to obtain current situational interaction purpose information; A first processing unit is used to divide the interactive scenes according to the scenario-based interactive purpose information to obtain a plurality of interactive scenes; A second obtaining unit is used to collect and obtain the multi-dimensional interaction information in the multiple interaction scenes to obtain multiple multi-dimensional interaction information sets; A second processing unit, configured to perform dimensionality reduction processing on the plurality of multi-dimensional interaction information sets to obtain a plurality of interaction information sets; A third processing unit, configured to set interaction action requirement information in the plurality of interaction scenes according to the plurality of interaction information sets; A fourth processing unit, configured to set interaction assessment requirement information in the plurality of interaction scenarios according to the plurality of interaction information sets; The first execution unit is used to construct and interact with the multiple interactive scenes according to the interactive action requirement information and the interactive assessment requirement information.

5. A scenario-based interactive system based on virtual reality, characterized in that: include: A processor, the processor is coupled to a memory, the memory is used to store a program, when the program is executed by the processor, the system is enabled to perform the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which implements the steps of the method according to any one of claims 1 to 3 when executed by a processor.

Citation Information

Patent Citations

  • Motion estimation system based on VR (Virtual Reality) and motion capture

    CN109675289A

  • Evaluation system and evaluation method based on virtual reality skill training

    CN113867532A

  • Method and system for skill learning

    US20210142692A1