A virtual training scenario simulation and scoring system based on VR and AR

Abstract

The application relates to the technical field of artificial intelligence, and particularly discloses a virtual training scene simulation and scoring system based on VR and AR, which comprises a virtual scene module, a real-time feedback and scoring system, a training and examination module, an analysis and strategy module, a role and interaction module, a learning target and task module and a learning path management module; the virtual scene module establishes a real financial sales scene by using VR technology and adds interactive elements and information display by using AR technology; the training and examination module is used for actual operation, simulation of the scene by learners in the virtual scene, and feedback and guidance are provided; the application provides a realistic virtual customer dialogue experience by combining generative AI technology of VR, simulates a financial product sales scene, constructs an immersive virtual financial sales scene by fusing AR technology, so that learners can practice in a realistic environment, actual sales skills are improved, and the system can be popularized in a larger range and training cost is reduced.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a virtual training scenario simulation and scoring system based on VR and AR. Background Technology

[0002] With the rapid development of financial markets and the increasing variety of financial products, the importance of financial sales training has become increasingly prominent. Traditional financial sales training methods often rely on classroom lectures and case studies, making it difficult to provide a realistic and interactive sales environment. This limits trainees' in-depth understanding of the financial product sales process and their mastery of skills. The widespread application of virtual reality (VR) and augmented reality (AR) technologies now offers new training methods. Virtual Reality (VR) is a three-dimensional virtual world created using computer simulation, which users can enter and interact with through specialized devices (such as head-mounted displays). Augmented Reality (AR) is a technology that combines virtual information with real-world information. It creates a hybrid augmented reality environment by overlaying computer-generated images, sounds, and videos onto the real world.

[0003] Currently, financial sales training tools on the market mainly include online courses, simulated trading, and customer role-playing. While these methods can provide some training effectiveness, they often lack interactivity and personalization, making it difficult to adapt to the needs of different financial sales personnel. Online courses typically consist of recorded videos and fixed quizzes, lacking real-time interaction and feedback. Simulated trading, while providing some practical opportunities, struggles to accurately assess sales personnel's emotional changes and actual abilities due to a lack of sentiment analysis and personalized feedback. Furthermore, traditional methods are insufficiently applied in big data analysis and customer profiling, failing to provide detailed customer information and personalized sales strategy recommendations.

[0004] Therefore, it is necessary to propose a virtual training scenario simulation and scoring system based on VR and AR to at least partially solve the problems existing in the current technology.

[0005] The information disclosed above in this background section is only for enhancing the understanding of the background section of this invention, and therefore may include prior art that is not known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a virtual training scenario simulation and scoring system based on VR and AR to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A VR and AR-based virtual training scenario simulation and scoring system includes: a virtual scenario module, a real-time feedback and scoring system, a training and assessment module, an analysis and strategy module, a role and interaction module, a learning objective and task module, and a learning path management module. The virtual scenario module utilizes VR technology to create a realistic financial sales scenario and adds interactive elements and information display through AR technology. The training and assessment module allows learners to perform practical operations and simulate scenarios within the virtual scenario, providing feedback and guidance. The analysis and strategy module analyzes customer data, extracts customer preferences and needs, and generates personalized strategy suggestions. The role and interaction module establishes virtual roles and interaction mechanisms, promoting interaction between learners and these roles. The learning objective and task module defines learning objectives and creates various types of tasks to help learners focus their efforts and improve learning efficiency. The learning path management module sets personalized learning paths, tracks and manages progress, and enhances learning efficiency and outcomes. The real-time feedback and scoring system provides immediate behavioral assessment, error correction, participation monitoring, real-time performance recording, and evaluation of learner performance, helping learners understand their performance and areas for improvement.

[0009] Preferably, the virtual scene module uses DirectX and generative AI technology combined with VR to create virtual scenes and simulate real financial sales dialogue scenarios. It uses a random number generation algorithm to set various financial sales scenarios and random events, increasing the diversity and challenge of training.

[0010] In virtual scenarios, supporting multimodal interaction makes the interaction more natural and intuitive, such as gesture recognition and speech recognition. At the same time, the application of perception technology enables the system to perceive the learner's position, actions and environment, such as spatial tracking (inertial policy unit (IMU), visual tracking technology, etc.), and select network connection according to the actual situation to realize data transmission and synchronization between multiple users.

[0011] Preferably, the real-time feedback and scoring system uses sentiment analysis technology to detect the emotional changes of learners and virtual customers in real time, provides immediate feedback, uses fuzzy evaluation methods to score learners' performance based on sales skills, communication effectiveness, and customer satisfaction indicators, and provides personalized training plans.

[0012] The system utilizes a cloud environment to store data generated during real-time feedback and scoring processes, and leverages edge computing technology to push the analysis and evaluation process to the network edge, thereby reducing data latency and bandwidth consumption during transmission and improving system response speed and efficiency.

[0013] Preferably, the training and assessment module dynamically adjusts the training content and difficulty based on the learner's skill level and learning progress using LSTM, identifies the learner's strengths and weaknesses, provides targeted training, and conducts regular assessments and performance evaluations to classify the learner's ability level and provide corresponding improvement suggestions.

[0014] Based on the assessment and evaluation results, machine learning technology is used to analyze learners' learning history data to determine the level of learners' abilities. Tasks are then redefined and reassigned according to the learning objectives, and the complexity and difficulty of the tasks are adjusted.

[0015] Preferably, the analysis and strategy module uses big data and machine learning technologies to analyze customer historical data, extract customer preferences and needs characteristics, provide learners with detailed customer profiles, and generate personalized financial product sales strategy suggestions based on customer analysis results, helping learners to more effectively promote financial products.

[0016] Preferably, the learning path management module adjusts the difficulty and content of the next virtual scene based on the learner's learning performance and feedback, providing a personalized learning experience. At the same time, it records the user's learning process and progress, analyzes weaknesses and potential areas for improvement, and provides guidance for further training.

[0017] Training administrators can monitor learners' progress in real time and provide support or adjust learning plans as needed to ensure learners complete training on time. The system can provide targeted virtual simulations, situational exercises, or additional support resources based on learners' reaction times, error patterns, or task changes to enhance learning effectiveness and maintain motivation.

[0018] Preferably, the learning objectives and tasks module sets clear learning objectives for each virtual scenario, and formulates tasks and challenges based on the learning objectives, requiring learners to complete specific sales targets in the virtual scenario. After learners complete the tasks, their performance is automatically or semi-automatically evaluated and relevant data is recorded.

[0019] Preferably, the role and interaction module creates virtual roles through behavior pattern generation, uses affective computing to simulate different emotional states and reaction patterns of the roles, and combines natural language processing and dialogue management technologies to allow learners to interact with the roles through voice, action, or text to simulate real sales scenario interactions.

[0020] Preferably, the system utilizes AR and VR technologies to collaboratively create an immersive training environment, making learners feel as if they are in a real sales scenario. Furthermore, by combining AR technology with mobile phones and wearable hardware, the immersive environment becomes simpler and easier to use. At the same time, the system creates a virtual digital tutor to provide real-time guidance and feedback on learners' performance, offering personalized training suggestions and psychological support.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] This invention provides a realistic virtual customer dialogue experience by combining VR generative AI technology to simulate financial product sales scenarios. It integrates AR technology to construct an immersive virtual financial sales scene, allowing learners to practice in a realistic environment and improve their actual sales skills. AR technology can be combined with mobile phones, utilizing the computing power of mobile phones, plus wearable hardware, to make the immersive environment simpler and easier to use. Compared with traditional training methods, this invention's system can be promoted on a larger scale and reduce training costs.

[0023] This invention employs a data-driven customer analysis and strategy recommendation engine to provide learners with detailed customer profiles and personalized sales strategy suggestions and strategies, enhancing customer satisfaction and increasing sales success rates. It utilizes sentiment analysis technology and digital human mentors to provide learners with instant feedback and psychological support, helping them improve rapidly. The system dynamically adjusts training content based on the capabilities and needs of financial sales personnel, providing targeted training to ensure personalized training effectiveness, especially for sales skills and strategies for financial products.

[0024] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0025] Figure 1 This is a framework diagram of the virtual training scenario simulation and scoring system of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example 1:

[0028] Please see Figure 1 As shown, a virtual training scenario simulation and scoring system based on VR and AR includes: a virtual scenario module and a role and interaction module. The virtual scenario module uses VR technology to create a realistic financial sales scenario and adds interactive elements and information display through AR technology. The role and interaction module is responsible for creating virtual roles and interaction mechanisms and promoting the interaction between learners and these roles.

[0029] The virtual scene module uses DirectX and generative AI technology combined with VR to create virtual scenes and simulate real financial sales dialogue scenarios. It uses random number generation algorithms to set up various financial sales scenarios and random events, increasing the diversity and challenge of training.

[0030] DirectX is an application programming interface (API) that enables Windows-based animation and multimedia programs to achieve higher performance, enhance 3D graphics and sound effects, and provide designers with a common hardware driver standard. This eliminates the need for developers to write different drivers for each brand of hardware and reduces the complexity of hardware installation and setup for users. Literally, "Direct" means direct, and the "X" represents many things; DirectX was created to provide direct services to a wide range of software.

[0031] The role and interaction module creates virtual characters through behavioral pattern generation, uses affective computing to simulate different emotional states and reaction patterns of the characters, and combines natural language processing and dialogue management technologies to allow learners to interact with the characters through voice, action, or text to simulate real sales scenarios.

[0032] Behavior pattern generation: Finite state machine (FSM): For the basic behavioral patterns of virtual characters, FSM can be used to define states and transitions, such as customer attitudes (friendly, suspicious, rejecting, etc.) and salesperson responses (responding to questions, providing information, etc.); Behavior trees: Used for complex character behavior design, capable of handling multiple concurrency and priorities, such as salespeople adjusting sales strategies based on customer reactions.

[0033] As can be seen from the above, this invention provides a realistic virtual customer dialogue experience by combining VR generative AI technology to simulate financial product sales scenarios. It integrates AR technology to construct an immersive virtual financial sales scenario, enabling learners to practice in a realistic environment and improve their actual sales skills. AR technology can be combined with mobile phones, utilizing the computing power of mobile phones, plus wearable hardware, to make the immersive environment simpler and easier to use. Compared with traditional training methods, the system of this invention can be promoted on a larger scale and reduce training costs.

[0034] Example 2:

[0035] Please see Figure 1 As shown, a virtual training scenario simulation and scoring system based on VR and AR also includes: a training and assessment module, a learning path management module, a real-time feedback and scoring system, an analysis and strategy module, and a learning objective and task module. The training and assessment module is used for learners to perform practical operations and simulate scenarios in a virtual environment, and provides feedback and guidance. The learning path management module is used for personalized learning path setting, progress tracking and management to improve learning efficiency and results. The real-time feedback and scoring system is used for immediate behavior assessment, error correction, participation monitoring, real-time performance recording and evaluation of learner performance, helping learners understand their performance and areas for improvement. The analysis and strategy module is used to analyze customer data, extract customer preferences and needs, and generate personalized strategy suggestions. The learning objective and task module is used to determine learning objectives and create various types of tasks to help learners focus their energy and improve learning efficiency.

[0036] The real-time feedback and scoring system uses sentiment analysis technology to detect changes in the emotions of learners and virtual customers in real time, providing instant feedback. It uses fuzzy evaluation methods to score learners' performance based on sales skills, communication effectiveness, and customer satisfaction indicators, and provides personalized training plans.

[0037] The training and assessment module dynamically adjusts the training content and difficulty based on the learner's skill level and learning progress using LSTM, identifies the learner's strengths and weaknesses, provides targeted training, and conducts regular assessments and performance evaluations to classify the learner's ability level and provide corresponding improvement suggestions.

[0038] The Analysis and Strategy module uses big data and machine learning technologies to analyze historical customer data, extract customer preferences and needs, provide learners with detailed customer profiles, and generate personalized financial product sales strategy suggestions based on the customer analysis results, helping learners to promote financial products more effectively.

[0039] The learning path management module adjusts the difficulty and content of the next virtual scenario based on learners' performance and feedback, providing a personalized learning experience. It also records users' learning journey and progress, analyzes weaknesses and potential areas for improvement, and provides guidance for further training.

[0040] The learning objectives and tasks module sets clear learning objectives for each virtual scenario and formulates tasks and challenges based on these objectives, requiring learners to achieve specific sales targets in the virtual scenario. After learners complete the tasks, their performance is automatically or semi-automatically evaluated and relevant data is recorded.

[0041] As can be seen from the above, this invention uses a data-driven customer analysis and strategy recommendation engine to provide learners with detailed customer profiles and personalized sales strategy suggestions and strategies, thereby enhancing customer satisfaction and increasing sales success rates. It utilizes sentiment analysis technology and digital human mentors to provide learners with instant feedback and psychological support, helping them to improve quickly. The system will dynamically adjust the training content according to the capabilities and needs of financial sales personnel, providing targeted training to ensure personalized training effectiveness, especially for sales skills and strategies for financial products.

[0042] Example 3:

[0043] This invention utilizes AR and VR technologies to collaboratively create an immersive training environment, making learners feel as if they are in a real sales scenario. Furthermore, by combining AR technology with mobile phones and wearable hardware, the immersive environment becomes simpler and easier to use. At the same time, the system creates a virtual digital mentor to provide real-time guidance and feedback on learners' performance, offering personalized training suggestions and psychological support.

[0044] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described data processing method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0045] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0046] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a program, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 The method of specifying the function in one or more boxes.

[0047] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0048] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0049] The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to general designs. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other.

[0050] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A virtual training scenario simulation and scoring system based on VR and AR, characterized in that, include: The system comprises a virtual scenario module, a real-time feedback and scoring system, a training and assessment module, a role and interaction module, and a learning path management module. The virtual scenario module utilizes VR technology to create a realistic financial sales scenario and adds interactive elements and information display through AR technology. The training and assessment module allows learners to perform practical operations and simulate scenarios within the virtual environment, providing feedback and guidance. The role and interaction module is responsible for establishing virtual roles and interaction mechanisms, and promoting interaction between learners and these roles. The learning path management module is used for personalized learning path setting, progress tracking and management, improving learning efficiency and outcomes. The real-time feedback and scoring system is used for immediate behavioral assessment, error correction, participation monitoring, real-time performance recording, and evaluation of learner performance, helping learners understand their performance and areas for improvement.

2. The virtual training scenario simulation and scoring system based on VR and AR according to claim 1, characterized in that, Also includes: The system comprises an analysis and strategy module and a learning objectives and task module. The analysis and strategy module is used to analyze customer data, extract customer preferences and needs, and generate personalized strategy suggestions. The learning objectives and task module is used to determine learning objectives and create various types of tasks to help learners focus their energy and improve learning efficiency.

3. The virtual training scenario simulation and scoring system based on VR and AR according to claim 2, characterized in that: The virtual scene module uses DirectX and generative AI technology combined with VR to create virtual scenes and simulate real financial sales dialogue scenarios. It uses a random number generation algorithm to set up various financial sales situations and random events, increasing the diversity and challenge of training.

4. The virtual training scenario simulation and scoring system based on VR and AR according to claim 3, characterized in that: The real-time feedback and scoring system uses sentiment analysis technology to detect changes in the emotions of learners and virtual customers in real time, providing immediate feedback. It uses fuzzy evaluation methods to score learners' performance based on sales skills, communication effectiveness, and customer satisfaction indicators, and provides personalized training plans.

5. A virtual training scenario simulation and scoring system based on VR and AR according to claim 4, characterized in that: The training and assessment module dynamically adjusts the training content and difficulty based on the learner's skill level and learning progress using LSTM, identifies the learner's strengths and weaknesses, provides targeted training, and conducts regular assessments and performance evaluations to classify the learner's ability level and provide corresponding improvement suggestions.

6. The virtual training scenario simulation and scoring system based on VR and AR according to claim 5, characterized in that: The analysis and strategy module uses big data and machine learning technologies to analyze historical customer data, extract customer preferences and needs, provide learners with detailed customer profiles, and generate personalized financial product sales strategy suggestions based on the customer analysis results, helping learners to promote financial products more effectively.

7. A virtual training scenario simulation and scoring system based on VR and AR according to claim 6, characterized in that: The learning path management module adjusts the difficulty and content of the next virtual scenario based on the learner's learning performance and feedback, providing a personalized learning experience. It also records the user's learning process and progress, analyzes weaknesses and potential areas for improvement, and provides guidance for further training.

8. A virtual training scenario simulation and scoring system based on VR and AR according to claim 7, characterized in that: The role and interaction module creates virtual characters through behavior pattern generation, uses affective computing to simulate different emotional states and reaction patterns of the characters, and combines natural language processing and dialogue management technologies to allow learners to interact with the characters through voice, action, or text to simulate real sales scenarios.

9. A virtual training scenario simulation and scoring system based on VR and AR according to claim 8, characterized in that: The learning objectives and tasks module sets clear learning objectives for each virtual scenario and formulates tasks and challenges based on these objectives, requiring learners to achieve specific sales targets in the virtual scenario. After learners complete the tasks, their performance is automatically or semi-automatically evaluated and relevant data is recorded.

10. A virtual training scenario simulation and scoring system based on VR and AR according to claim 9, characterized in that: The system utilizes AR and VR technologies to create an immersive training environment, making learners feel as if they are in a real sales scenario. It combines AR technology with mobile phones and wearable hardware to make the immersive environment simpler and easier to use. At the same time, the system creates a virtual digital tutor to provide real-time guidance and feedback on learners' performance, and offers personalized training suggestions and psychological support.

Images