Motion intervention method, device and system based on virtual reality and deep reinforcement learning technology

Abstract

The invention relates to a virtual reality and deep reinforcement learning technology-based motion intervention method, device and system, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a motion knowledge template matched with a virtual motion scene associated with virtual reality equipment; performing reinforcement learning training on a to-be-trained exercise guidance model according to the exercise knowledge template to obtain a pre-trained universal exercise guidance model; the pre-trained universal motion guidance model is used for providing a universal standardized motion guidance prompt for a target object wearing the virtual reality equipment; acquiring virtual motion interaction data of the target object in the virtual motion scene; performing fine tuning training based on reinforcement learning on a pre-trained universal motion guidance model according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object; the personalized motion guidance model is used for providing personalized motion guidance prompts for the target object. By adopting the method, the intelligence of motion guidance can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and virtual reality technology, and in particular to a motion intervention method, device, system, computer-readable storage medium, and computer program product based on virtual reality and deep reinforcement learning technology. Background Art

[0002] Sports science and technology refers to a discipline that uses scientific principles and technical means to study, design, develop and apply sports. It covers multiple disciplines such as sports physiology, sports psychology, sports biomechanics, and sports training, and aims to improve people's physical fitness and promote the development and innovation of sports technology.

[0003] However, in modern society, people are facing problems such as increasing time spent on electronic devices and reducing outdoor activities, which leads to insufficient exercise, thus affecting their physical health and mental state. In related technologies, real-world exercise intervention usually relies on traditional teacher teaching and is limited by venues. It cannot effectively stimulate people's interest in exercise and long-term participation, and cannot provide scientific exercise feedback and intelligent personalized exercise guidance.

[0004] Therefore, there is a problem in the related art that the intelligence of motion guidance is low. Summary of the invention

[0005] Based on this, it is necessary to provide a motion intervention method, device, computer equipment, computer-readable storage medium and computer program product based on virtual reality and deep reinforcement learning technology, which can improve the intelligence of motion guidance in response to the above-mentioned technical problems.

[0006] In a first aspect, the present application provides a motion intervention method based on virtual reality and deep reinforcement learning technology, comprising:

[0007] Acquire a motion knowledge template matching a virtual motion scene associated with a virtual reality device; the virtual motion scene is obtained by digital twin modeling based on a real motion scene; the motion knowledge template includes standardized motion knowledge content;

[0008] Performing reinforcement learning training on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained general motion guidance model; the pre-trained general motion guidance model is used to provide a general standardized motion guidance prompt to the target object wearing the virtual reality device;

[0009] Acquire virtual motion interaction data of the target object in the virtual motion scene; the virtual motion interaction data is motion interaction data obtained according to the action performed by the target object in the virtual motion scene;

[0010] The pre-trained general motion guidance model is fine-tuned based on reinforcement learning according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object; the personalized motion guidance model is used to provide personalized motion guidance prompts to the target object according to the motion state of the target object.

[0011] In one embodiment, the step of obtaining a motion knowledge template matching a virtual motion scene associated with a virtual reality device includes:

[0012] Acquire a parameterized static modeling result of a virtual object associated with the virtual motion scene; the parameterized static modeling result is obtained by modeling according to static characteristics of a real object corresponding to the virtual object when the real object is in a stationary state;

[0013] Acquire a parameterized dynamic modeling result of a virtual object associated with the virtual motion scene; the parameterized dynamic modeling result is obtained by modeling according to the dynamic characteristics of the real object when it is in motion;

[0014] The motion knowledge template is constructed according to the parameterized static modeling result and the parameterized dynamic modeling result.

[0015] In one embodiment, the step of obtaining a parameterized static modeling result of a virtual object associated with the virtual motion scene includes:

[0016] Acquire physical property data corresponding to the real object, and construct a virtual model corresponding to the real object according to the physical property data;

[0017] Assigning target physical properties and target model maps to the virtual model corresponding to the real object to obtain a target virtual model corresponding to the real object; the target physical properties are physical properties that match the real object; the target model map is a model map that matches the real object;

[0018] The parametric static modeling result is obtained according to the target virtual model corresponding to the real object.

[0019] In one embodiment, the obtaining of the parameterized dynamic modeling result of the virtual object associated with the virtual sports scene includes:

[0020] Acquire a dynamic equation when the real object is in motion; the dynamic equation is used to characterize the dynamic characteristics of the real object when it is in motion;

[0021] The kinetic equation is subjected to parameterized modeling to obtain the parameterized dynamic modeling result.

[0022] In one embodiment, the acquiring virtual motion interaction data of the target object in the virtual motion scene includes:

[0023] The current virtual environment information of the virtual sports scene is obtained, and the current virtual environment information is input into the pre-trained general motion guidance model; the strategy function of the pre-trained general motion guidance model is used to generate motion guidance action prompts in response to the current virtual environment information; the motion guidance action prompts are used to instruct the target object to perform corresponding actions in the virtual sports scene to obtain the virtual sports interaction data.

[0024] In one embodiment, the fine-tuning training based on reinforcement learning is performed on the pre-trained general motion guidance model according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object, including:

[0025] Acquire virtual environment information of the virtual sports scene after responding to the virtual sports interaction data, obtain new virtual environment information, and acquire reward data obtained after the pre-trained general sports guidance model outputs the sports guidance action prompt;

[0026] According to the new virtual environment information and the reward data, the strategy function of the pre-trained general motion guidance model is updated until the strategy function converges to obtain the personalized motion guidance model.

[0027] In a second aspect, the present application also provides a motion intervention device based on virtual reality and deep reinforcement learning technology, comprising:

[0028] A first acquisition module is used to acquire a motion knowledge template matching a virtual motion scene associated with a virtual reality device; the virtual motion scene is obtained by performing digital twin modeling based on a real motion scene; the motion knowledge template includes standardized motion knowledge content;

[0029] A training module, used for performing reinforcement learning training on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained universal motion guidance model; the pre-trained universal motion guidance model is used to provide universal standardized motion guidance prompts to the target object wearing the virtual reality device;

[0030] A second acquisition module is used to acquire virtual motion interaction data of the target object in the virtual motion scene; the virtual motion interaction data is motion interaction data obtained according to the action performed by the target object in the virtual motion scene;

[0031] A fine-tuning module is used to perform reinforcement learning-based fine-tuning training on the pre-trained general motion guidance model according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object; the personalized motion guidance model is used to provide personalized motion guidance prompts to the target object according to the motion state of the target object.

[0032] In a third aspect, the present application also provides a sports intervention system based on virtual reality and deep reinforcement learning technology. The system includes a memory and a processor, the memory stores a computer program, and the computer program implements the steps of the above method when executed by the processor.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0034] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0035] The above-mentioned motion intervention method, device, system, computer-readable storage medium and computer program product based on virtual reality and deep reinforcement learning technology obtain a motion knowledge template matching a virtual motion scene associated with a virtual reality device; the virtual motion scene is obtained by digital twin modeling based on a real motion scene; the motion knowledge template includes standardized motion knowledge content; reinforcement learning training is performed on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained general motion guidance model; the pre-trained general motion guidance model is used to provide general standardized motion guidance prompts to the target object wearing the virtual reality device; virtual motion interaction data of the target object in the virtual motion scene is obtained; the virtual motion interaction data is motion interaction data obtained according to the actions performed by the target object in the virtual motion scene; reinforcement learning-based fine-tuning training is performed on the pre-trained general motion guidance model according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object; the personalized motion guidance model is used to provide personalized motion guidance prompts to the target object according to the motion state of the target object.

[0036] In this way, a motion guidance model that can provide standardized and personalized motion guidance is trained through a two-stage deep reinforcement learning method, wherein, in the digital twin stage, the motion guidance model to be trained is trained according to the motion knowledge template created based on the standardized motion knowledge content drive to learn the standardized motion guidance and obtain a pre-trained general motion guidance model; in the feedback stage of human-computer interaction, the pre-trained general motion guidance model is fine-tuned through the virtual motion interaction data of the target object in the virtual motion scene, so that a personalized motion guidance model for the target object can be obtained, so that the personalized motion guidance model for the target object can adjust the general standardized motion guidance prompts according to the motion state of the target object, so that the adjusted standardized motion guidance prompts can meet the current motion state of the target object, and the personalized motion guidance prompts for the target object are obtained, which realizes that on the basis of providing standardized motion guidance for the target object, personalized motion guidance is further provided to the target object, effectively improving the intelligence of the motion guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A flowchart of a motion intervention method based on virtual reality and deep reinforcement learning technology in one embodiment;

[0039] Figure 2 A schematic flow chart of the steps of obtaining a motion knowledge template matching a virtual motion scene associated with a virtual reality device in one embodiment;

[0040] Figure 3 A schematic diagram of a flow chart of a physical simulation algorithm for parameterized modeling in one embodiment;

[0041] Figure 4 A schematic diagram of a framework of a deep reinforcement learning algorithm based on motion knowledge templates and human-computer interaction in one embodiment;

[0042] Figure 5 is a flow chart of a motion intervention method based on virtual reality and deep reinforcement learning technology in another embodiment;

[0043] Figure 6 is a structural block diagram of a motion intervention device based on virtual reality and deep reinforcement learning technology in one embodiment;

[0044] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0047] In one embodiment, Figure 1 As shown, a motion intervention method based on virtual reality and deep reinforcement learning technology is provided. This embodiment takes the method applied to a computer device as an example. It can be understood that the computer device can be a terminal device, and the terminal device can include but is not limited to: smart phones, tablet computers, portable personal computers and other devices. The embodiment of the present application does not limit the type of terminal device, which is explained here. Optionally, the computer device can be a server, and the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The computer device can also be a system including a terminal and a server, and the motion intervention method based on virtual reality and deep reinforcement learning technology is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0048] Step S110, obtaining a motion knowledge template that matches a virtual motion scene associated with the virtual reality device.

[0049] Among them, virtual reality devices may include virtual reality (VR) head-mounted devices and motion controllers.

[0050] Among them, the virtual sports scene is obtained by digital twin modeling based on the real sports scene.

[0051] The virtual sports scene may be a virtual sports scene associated with a motion controller.

[0052] Different motion controllers can be provided according to different types of sports. For example, in table tennis, a simulated racket controller is provided to provide a more realistic serving and receiving operation experience. In football, a leg controller is provided to provide a more realistic kicking experience.

[0053] Among them, the sports knowledge template includes standardized sports knowledge content.

[0054] Among them, the standardized sports knowledge content can be standardized sports knowledge content related to the sports type corresponding to the virtual sports scene.

[0055] In a specific implementation, the computer device may obtain a motion knowledge template that matches a virtual sports scene associated with a motion controller in a virtual reality device, and the motion knowledge template includes standardized motion knowledge content related to the type of motion corresponding to the virtual sports scene. For example, if the motion controller is a simulated racket controller and the associated virtual sports scene is a virtual table tennis scene, the matched motion knowledge template may include standardized motion knowledge content related to table tennis; if the motion controller is a leg controller and the associated virtual sports scene is a virtual football scene, the matched motion knowledge template may include standardized motion knowledge content related to football.

[0056] Step S120 , performing reinforcement learning training on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained general motion guidance model.

[0057] Among them, the pre-trained general motion guidance model is used to provide general standardized motion guidance prompts to the target object wearing the virtual reality device.

[0058] In a specific implementation, a computer device can perform reinforcement learning training on a motion guidance model to be trained according to a motion knowledge template to obtain a pre-trained general motion guidance model. The pre-trained general motion guidance model can be used to provide general standardized motion guidance prompts to a target object wearing a virtual reality device.

[0059] Among them, the universal standardized exercise guidance prompts can provide the target objects with universal posture guidance and skill tips that are applicable to a wide range of people during exercise to ensure the safety and effectiveness of exercise.

[0060] Among them, after the target object wears the virtual reality head-mounted device and the motion controller, it can enter the intervention software to perform sports. The target object can choose sports of different difficulty and different types according to its own situation. There are different pre-trained general sports guidance models corresponding to sports of different difficulty and different types. The embodiment of the present application explains the sports guidance model corresponding to a certain sports project. It can be understood that the method provided by the present application can be applied to the adjustment of the sports guidance model of sports of different difficulty and different types.

[0061] Step S130, obtaining virtual motion interaction data of the target object in the virtual motion scene.

[0062] The virtual motion interaction data is motion interaction data obtained according to the actions performed by the target object in the virtual motion scene.

[0063] In a specific implementation, the computer device can obtain virtual motion interaction data of the target object in the virtual motion scene according to the action performed by the target object in the virtual motion scene. Specifically, the target object has a corresponding virtual character in the virtual motion scene, and the computer device can obtain the virtual motion interaction data according to the action performed by the virtual character corresponding to the target object in the virtual motion scene.

[0064] In some embodiments, the action performed by the target object in the real world can be synchronously mapped to the virtual character corresponding to the target object, thereby obtaining the action performed by the virtual character corresponding to the target object in the virtual sports scene.

[0065] For example, a real hand gesture may correspond to a specific action or effect in a virtual environment. For example:

[0066] Example 1: Jump

[0067] Real world: The target object makes a jump in reality, and the sensor detects its vertical displacement and time information.

[0068] Mapping process: By analyzing the height and duration of the jump, the computer device maps the jump event to the corresponding effect in the virtual sports scene.

[0069] Virtual motion scene: The virtual character corresponding to the target object jumps in the virtual motion scene, which may trigger a reaction of a virtual prop, such as opening a door or starting a task.

[0070] Example 2: Walking movement

[0071] Real world: The target object walks in reality, and the sensor captures its position changes, speed and direction.

[0072] Mapping process: The computer device maps the walking motion data of the target object to the position coordinates in the virtual motion scene.

[0073] Virtual motion scene: The virtual character corresponding to the target object moves accordingly in the virtual motion scene, perhaps walking in a virtual city, moving along the same path in reality.

[0074] Step S140 , performing fine-tuning training based on reinforcement learning on the pre-trained general motion guidance model according to the virtual motion interaction data, to obtain a personalized motion guidance model for the target object.

[0075] Among them, the personalized motion guidance model is used to provide personalized motion guidance prompts to the target object according to the motion state of the target object.

[0076] The motion state of the target object may include at least one of the target object's body data (such as heart rate, posture, speed, motion range, etc.) and current virtual motion interaction data.

[0077] In a specific implementation, a computer device can perform reinforcement learning-based fine-tuning training on a pre-trained general motion guidance model according to virtual motion interaction data to obtain a personalized motion guidance model for the target object. The personalized motion guidance model can adjust the general standardized motion guidance prompts according to the motion state of the target object to provide personalized motion guidance prompts for the target object.

[0078] Among them, personalized motion guidance prompts can provide the target object with personal posture guidance and skill tips during exercise to meet the target object's unique situation and goals, achieve precise motion guidance for the target object, and enhance the intelligence of motion guidance.

[0079] In the above-mentioned motion intervention method based on virtual reality and deep reinforcement learning technology, a motion knowledge template matching a virtual motion scene associated with a virtual reality device is obtained; the virtual motion scene is obtained by digital twin modeling based on a real motion scene; the motion knowledge template includes standardized motion knowledge content; reinforcement learning training is performed on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained general motion guidance model; the pre-trained general motion guidance model is used to provide general standardized motion guidance prompts to a target object wearing a virtual reality device; virtual motion interaction data of the target object in the virtual motion scene is obtained; the virtual motion interaction data is motion interaction data obtained according to the actions performed by the target object in the virtual motion scene; reinforcement learning-based fine-tuning training is performed on the pre-trained general motion guidance model according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object; the personalized motion guidance model is used to provide personalized motion guidance prompts to the target object according to the motion state of the target object.

[0080] In this way, a motion guidance model that can provide standardized and personalized motion guidance is trained through a two-stage deep reinforcement learning method, wherein, in the digital twin stage, the motion guidance model to be trained is trained according to the motion knowledge template created based on the standardized motion knowledge content drive to learn the standardized motion guidance and obtain a pre-trained general motion guidance model; in the feedback stage of human-computer interaction, the pre-trained general motion guidance model is fine-tuned through the virtual motion interaction data of the target object in the virtual motion scene, so that a personalized motion guidance model for the target object can be obtained, so that the personalized motion guidance model for the target object can adjust the general standardized motion guidance prompts according to the motion state of the target object, so that the adjusted standardized motion guidance prompts can meet the current motion state of the target object, and the personalized motion guidance prompts for the target object are obtained, which realizes that on the basis of providing standardized motion guidance for the target object, personalized motion guidance is further provided to the target object, effectively improving the intelligence of the motion guidance.

[0081] In one embodiment, Figure 2 As shown, step S110, obtaining a motion knowledge template matching a virtual motion scene associated with a virtual reality device, includes the following steps:

[0082] Step S210, obtaining a parameterized static modeling result of a virtual object associated with a virtual motion scene.

[0083] The virtual objects associated with the virtual motion scene can be used to represent real objects in the real motion scene, that is, to represent real objects.

[0084] For example, in a table tennis scene, the virtual objects associated with the virtual sports scene may include virtual objects corresponding to real objects such as table tennis balls and table tennis rackets. In a football scene, the virtual objects associated with the virtual sports scene may include virtual objects corresponding to real objects such as football fields and footballs.

[0085] The parametric static modeling result is obtained by modeling the static characteristics of the real object corresponding to the virtual object when it is in a stationary state.

[0086] The static characteristics may include static position information and static direction information when in a stationary state.

[0087] In a specific implementation, a computer device may obtain a parameterized static modeling result of a virtual object associated with a virtual motion scene, wherein the parameterized static modeling result is modeled according to static characteristics of a real object corresponding to the virtual object when the real object is in a stationary state.

[0088] Step S220, obtaining the parameterized dynamic modeling result of the virtual object associated with the virtual sports scene.

[0089] The parametric dynamic modeling result is obtained by modeling the dynamic characteristics of the real object when it is in motion.

[0090] Among them, dynamic characteristics may include changes in the position of real objects, collisions, captures, scaling, surface deformations, etc.

[0091] In a specific implementation, the computer device can obtain a parameterized dynamic modeling result of a virtual object associated with a virtual motion scene, where the parameterized dynamic modeling result is modeled based on the dynamic characteristics of a real object when it is in motion.

[0092] Step S230, constructing a motion knowledge template according to the parameterized static modeling results and the parameterized dynamic modeling results.

[0093] In a specific implementation, the computer device can construct a motion knowledge template based on the parametric static modeling results and the parametric dynamic modeling results.

[0094] The technical solution of this embodiment is to obtain the parameterized static modeling results of the virtual object associated with the virtual motion scene; the parameterized static modeling results are used to characterize the static characteristics of the virtual object when it is in a stationary state; obtain the parameterized dynamic modeling results of the virtual object associated with the virtual motion scene; the parameterized dynamic modeling results are used to characterize the dynamic characteristics of the virtual object when it is in motion; and construct a motion knowledge template based on the parameterized static modeling results and the parameterized dynamic modeling results. In this way, by obtaining the parameterized static modeling results and the parameterized dynamic modeling results obtained by modeling the static characteristics and dynamic characteristics of the real object corresponding to the virtual object when it is in a stationary state, in order to construct a motion knowledge template containing standardized motion knowledge content, more accurate motion simulation and analysis can be achieved.

[0095] In one embodiment, a parametric static modeling result of a virtual object associated with a virtual motion scene is obtained, including: obtaining physical property data corresponding to the real object, and constructing a virtual model corresponding to the real object according to the physical property data; assigning target physical properties and target model maps to the virtual model corresponding to the real object to obtain a target virtual model corresponding to the real object; and obtaining a parametric static modeling result based on the target virtual model corresponding to the real object.

[0096] The target physical properties are physical properties that match the real objects.

[0097] The target model map is a model map that matches the real object.

[0098] Among them, physical property data may include data such as size, shape, weight, and material.

[0099] Among them, physical properties may include density, friction coefficient, elastic modulus, etc.

[0100] In a specific implementation, when a computer device obtains the parametric static modeling result of a virtual object associated with a virtual motion scene, the computer device can obtain the physical property data of the real object and construct a virtual model corresponding to the real object according to the physical property data. For example, the physical property data of the real object is collected, a three-dimensional model that conforms to the actual shape is constructed using design software, and the size and shape are set as adjustable parameters to obtain a virtual model. Then, the virtual model corresponding to the real object can be assigned target physical properties and target model maps to obtain a target virtual model corresponding to the real object; wherein the target physical properties are physical properties that match the real object; and the target model map is a model map that matches the real object. In this way, by assigning the target physical properties, the movement and interactive behavior of the real object during the motion process can be accurately simulated; by assigning the target model map, the visual effect can be enhanced, and further, the material setting (such as reflection, glossiness, etc.) can be performed for the virtual model corresponding to the real object to improve the sense of reality. Thus, the computer device can obtain the parametric static modeling result of the virtual object according to the target virtual model corresponding to the real object.

[0101] The technical solution of this embodiment is to obtain the physical property data corresponding to the real object, and construct the virtual model corresponding to the real object according to the physical property data; assign the target physical properties and the target model map to the virtual model corresponding to the real object, and obtain the target virtual model corresponding to the real object; the target physical properties are the physical properties that match the real object; the target model map is the model map that matches the real object; and the parametric static modeling result is obtained according to the target virtual model corresponding to the real object. In this way, a virtual model that conforms to the actual shape can be constructed through the physical property data, and the movement and interactive behavior of the real object during the movement process can be accurately simulated by assigning the target physical properties. By assigning the target model map, the visual effect can be enhanced, so that the obtained parametric static modeling result can more accurately simulate the static characteristics of the real object when it is in a stationary state.

[0102] In one embodiment, obtaining a parametric dynamic modeling result of a virtual object associated with a virtual motion scene includes: obtaining a dynamic equation when the real object is in motion; the dynamic equation is used to characterize the dynamic characteristics of the real object when it is in motion; and parametrically modeling the dynamic equation to obtain a parametric dynamic modeling result.

[0103] In a specific implementation, when a computer device obtains a parametric dynamic modeling result of a virtual object associated with a virtual motion scene, the computer device can obtain a dynamic equation when the real object is in motion; the dynamic equation is used to characterize the dynamic characteristics of the real object when it is in motion; the dynamic equation is parametrically modeled to obtain a parametric dynamic modeling result.

[0104] The technical solution of this embodiment is to obtain the dynamic equation of the real object when it is in motion; the dynamic equation is used to characterize the dynamic characteristics of the real object when it is in motion; and the dynamic equation is parameterized to model the dynamic equation to obtain the parameterized dynamic modeling result. In this way, by parameterizing the dynamic equation of the real object when it is in motion, the dynamic characteristics of the real object when it is in motion can be simulated more accurately.

[0105] In order to facilitate the understanding of those skilled in the art, Figure 3 A flow chart of a physical simulation algorithm for parametric modeling is provided. Specifically, Figure 3 The flowchart of the physical simulation algorithm based on human in the loop is shown. In order to enable the sports guidance model to master the standardized sports knowledge content in the digital twin stage, a physical simulation algorithm based on human in the loop is designed to construct the VR environment and the preset sports knowledge template. For real objects, such as sports venues, table tennis rackets, footballs, etc., computer-assisted methods are used for parametric modeling, and high-resolution maps are given to obtain parametric static modeling results. For dynamic modeling, such as the interaction force between the racket and the ball, the rotation of the ball, etc., computer-assisted methods are used to parametrically model the dynamic equations to obtain parametric dynamic modeling results. For the virtual sports scene (virtual sports world) created, professional coaches and young physical education teachers (professional people) test and provide feedback. Based on the feedback, the parametric modeling method is adjusted to achieve high realism, high immersion and safety level for the user population.

[0106] In one embodiment, obtaining virtual motion interaction data of a target object in a virtual motion scene includes: obtaining current virtual environment information of the virtual motion scene, and inputting the current virtual environment information into a pre-trained general motion guidance model; the strategy function of the pre-trained general motion guidance model is used to generate motion guidance action prompts in response to the current virtual environment information; the motion guidance action prompts are used to instruct the target object to perform corresponding actions in the virtual motion scene to obtain virtual motion interaction data.

[0107] The current virtual environment information may include the current scene, the position of the ball, the rotation of the ball, the speed of the ball, etc.

[0108] The strategy function of the pre-trained general motion guidance model may be a neural network.

[0109] In a specific implementation, when a computer device is obtaining virtual motion interaction data of a target object in a virtual motion scene, the computer device can obtain current virtual environment information of the virtual motion scene, and input the current virtual environment information into a pre-trained general motion guidance model. The strategy function of the pre-trained general motion guidance model is used to generate motion guidance action prompts in response to the current virtual environment information. In this way, the target object can perform corresponding actions according to the motion guidance action prompts, and by synchronously mapping the actions to the virtual character corresponding to the target object in the virtual motion scene, the virtual character corresponding to the target object can perform corresponding actions in the virtual motion scene to obtain virtual motion interaction data.

[0110] Specifically, the motion guidance action prompt may refer to a motion guidance action prompt that maximizes the predicted probability of the target object continuing to participate in the motion interaction, determined by the pre-trained general motion guidance model according to the current virtual environment information among various candidate motion guidance action prompts. In this way, the target object can be accurately guided for motion according to the current virtual environment information.

[0111] In one embodiment, a pre-trained general motion guidance model is fine-tuned based on reinforcement learning according to virtual motion interaction data to obtain a personalized motion guidance model for a target object, including: obtaining virtual environment information after the virtual motion scene responds to the virtual motion interaction data to obtain new virtual environment information, and obtaining reward data obtained after the pre-trained general motion guidance model outputs motion guidance action prompts; according to the new virtual environment information and reward data, the strategy function of the pre-trained general motion guidance model is updated until the strategy function converges to obtain a personalized motion guidance model.

[0112] The reward data is determined based on the predicted probability of the target object continuing to participate in the sports interaction.

[0113] In a specific implementation, when a computer device performs reinforcement learning-based fine-tuning training on a pre-trained general motion guidance model according to virtual motion interaction data to obtain a personalized motion guidance model for a target object, the computer device can obtain virtual environment information after the virtual motion scene responds to the virtual motion interaction data, obtain new virtual environment information, and obtain reward data obtained after the pre-trained general motion guidance model outputs motion guidance action prompts; based on the new virtual environment information and reward data, the strategy function of the pre-trained general motion guidance model is updated until the strategy function converges to obtain a personalized motion guidance model.

[0114] The technical solution of this embodiment obtains new virtual environment information by obtaining virtual environment information after the virtual sports scene responds to virtual sports interaction data, and obtains reward data obtained after the pre-trained general sports guidance model outputs sports guidance action prompts; according to the new virtual environment information and reward data, the strategy function of the pre-trained general sports guidance model is updated until the strategy function converges to obtain a personalized sports guidance model. In this way, by continuously obtaining new virtual environment information and reward data after the virtual sports scene responds to virtual sports interaction data, the pre-trained general sports guidance model can adaptively adjust its strategy function to better provide personalized training plans and sports guidance for the target object, so that it is more in line with individual needs and goals, and improves the accuracy and safety of sports guidance for the target object.

[0115] In order to facilitate the understanding of those skilled in the art, Figure 4 A framework for deep reinforcement learning algorithms based on sports knowledge templates and human-computer interaction is provided. The sports guidance model can be understood as a virtual intelligent coach, such as Figure 4 As shown in the figure, a virtual intelligent coach (REVERIE Coach) is built using a transformer network to learn sports knowledge content. REVERIECoach uses a multimodal encoder to obtain the current virtual environment information from the virtual sports scene, including the current scene, the position of the ball, the rotation of the ball, the speed of the ball, etc. In the digital twin stage, REVERIE Coach is trained with sports knowledge templates in the virtual sports scene to master standardized sports knowledge content for standardized sports guidance. In the human-computer interaction stage, REVERIE Coach will fine-tune the virtual sports interaction data fed back by the target object to develop personalized sports technology suitable for each target object. After two stages of deep reinforcement learning training, REVERIE Coach can provide standardized and personalized sports guidance for the target object.

[0116] Among them, the server can use the virtual environment information as the state, the motion guidance action prompt as the action, and establish the RL (Reinforcement Learning) framework to maximize long-term benefits (such as user retention rate). Among them, the RL framework contains two important components, namely the state of the environment and the interaction between the environment and the decision maker. In a changing environment, at a certain moment, the environment is in a certain state (state), and the decision maker needs to make a decision based on the state and take a certain action (action). Each action will cause the environment to change and reach a new state, and this new state will affect the action of the decision maker at the new moment. In the RL framework, different states and different actions will generate rewards (reward data, which can be negative). In the continuous interaction and development of the entire environment and decision makers, RL hopes to find a policy, that is, how to decide on actions for different states, so that the long-term cumulative reward can be maximized. In motion guidance, the virtual environment information is the state that affects the action guidance decision, and the output motion guidance action prompt is the action that will affect the next state. The policy is how to perform motion guidance action prompts for the virtual environment information.

[0117] In another embodiment, Figure 5 As shown, a motion intervention method based on virtual reality and deep reinforcement learning technology is provided, and the method is applied to a computer device as an example for explanation, including the following steps:

[0118] Step S502: Acquire physical property data corresponding to the real object, and construct a virtual model corresponding to the real object according to the physical property data.

[0119] Step S504, assigning target physical properties and a target model map to the virtual model corresponding to the real object, to obtain the target virtual model corresponding to the real object.

[0120] Step S506, obtaining a parametric static modeling result according to the target virtual model corresponding to the real object.

[0121] Step S508, obtaining the dynamic equation of the real object when it is in motion.

[0122] Step S510, performing parameterized modeling on the dynamics equation to obtain a parameterized dynamic modeling result.

[0123] Step S512: construct a motion knowledge template according to the parameterized static modeling results and the parameterized dynamic modeling results.

[0124] Step S514: Perform reinforcement learning training on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained general motion guidance model.

[0125] Step S516, obtaining current virtual environment information of the virtual sports scene, and inputting the current virtual environment information into the pre-trained general sports guidance model.

[0126] Step S518, obtaining virtual environment information after the virtual sports scene responds to the virtual sports interaction data, obtaining new virtual environment information, and obtaining reward data obtained after the pre-trained general sports guidance model outputs sports guidance action prompts.

[0127] Step S520 , updating the policy function of the pre-trained general motion guidance model according to the new virtual environment information and reward data until the policy function converges to obtain a personalized motion guidance model.

[0128] It should be noted that the specific limitations of the above steps can be found in the above specific limitations of a motion intervention method based on virtual reality and deep reinforcement learning technology.

[0129] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0130] Based on the same inventive concept, the embodiment of the present application also provides a motion intervention device based on virtual reality and deep reinforcement learning technology for implementing the motion intervention method based on virtual reality and deep reinforcement learning technology involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the motion intervention device based on virtual reality and deep reinforcement learning technology provided below can refer to the above limitations on the motion intervention method based on virtual reality and deep reinforcement learning technology, which will not be repeated here.

[0131] In an exemplary embodiment, Figure 6As shown, a motion intervention device based on virtual reality and deep reinforcement learning technology is provided, including: a first acquisition module 610, a training module 620, a second acquisition module 630 and a fine-tuning module 640, wherein:

[0132] The first acquisition module 610 is used to acquire a motion knowledge template matching a virtual motion scene associated with a virtual reality device; the virtual motion scene is obtained by digital twin modeling based on a real motion scene; the motion knowledge template includes standardized motion knowledge content.

[0133] The training module 620 is used to perform reinforcement learning training on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained general motion guidance model; the pre-trained general motion guidance model is used to provide general standardized motion guidance prompts to the target object wearing the virtual reality device.

[0134] The second acquisition module 630 is used to acquire virtual motion interaction data of the target object in the virtual motion scene; the virtual motion interaction data is motion interaction data obtained according to the action performed by the target object in the virtual motion scene.

[0135] The fine-tuning module 640 is used to perform reinforcement learning-based fine-tuning training on the pre-trained general motion guidance model according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object; the personalized motion guidance model is used to provide personalized motion guidance prompts to the target object according to the motion state of the target object.

[0136] In one of the embodiments, the first acquisition module 610 is specifically used to obtain a parameterized static modeling result of a virtual object associated with the virtual motion scene; the parameterized static modeling result is modeled according to the static characteristics of a real object corresponding to the virtual object when it is in a stationary state; obtain a parameterized dynamic modeling result of a virtual object associated with the virtual motion scene; the parameterized dynamic modeling result is modeled according to the dynamic characteristics of the real object when it is in motion; and construct the motion knowledge template based on the parameterized static modeling result and the parameterized dynamic modeling result.

[0137] In one of the embodiments, the first acquisition module 610 is specifically used to acquire physical property data corresponding to the real object, and construct a virtual model corresponding to the real object according to the physical property data; assign target physical properties and target model maps to the virtual model corresponding to the real object to obtain a target virtual model corresponding to the real object; the target physical properties are physical properties that match the real object; the target model map is a model map that matches the real object; and the parametric static modeling result is obtained based on the target virtual model corresponding to the real object.

[0138] In one embodiment, the first acquisition module 610 is specifically used to obtain the dynamic equation of the real object when it is in motion; the dynamic equation is used to characterize the dynamic characteristics of the real object when it is in motion; the dynamic equation is parameterized modeled to obtain the parameterized dynamic modeling result.

[0139] In one embodiment, the second acquisition module 630 is specifically used to obtain the current virtual environment information of the virtual sports scene, and input the current virtual environment information into the pre-trained general motion guidance model; the strategy function of the pre-trained general motion guidance model is used to generate motion guidance action prompts in response to the current virtual environment information; the motion guidance action prompts are used to instruct the target object to perform corresponding actions in the virtual sports scene to obtain the virtual sports interaction data.

[0140] In one embodiment, the fine-tuning module 640 is specifically used to obtain virtual environment information after the virtual sports scene responds to the virtual sports interaction data, obtain new virtual environment information, and obtain reward data obtained after the pre-trained general sports guidance model outputs the sports guidance action prompt; according to the new virtual environment information and the reward data, the strategy function of the pre-trained general sports guidance model is updated until the strategy function converges to obtain the personalized sports guidance model.

[0141] Each module in the above-mentioned sports intervention device based on virtual reality and deep reinforcement learning technology can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0142] In an exemplary embodiment, a computer device is provided, which can be a motion intervention system based on virtual reality and deep reinforcement learning technology, and its internal structure diagram can be shown in Figure 7. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store model parameter data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a motion intervention method based on virtual reality and deep reinforcement learning technology is implemented.

[0143] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0144] In one embodiment, a motion intervention system based on virtual reality and deep reinforcement learning technology is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0145] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0146] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0148] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0149] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0150] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A sports intervention method based on virtual reality and deep reinforcement learning technology, It is characterized in that The method comprises: Acquire a motion knowledge template matching a virtual motion scene associated with a virtual reality device; the virtual motion scene is obtained by digital twin modeling based on a real motion scene; the motion knowledge template includes standardized motion knowledge content; Performing reinforcement learning training on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained general motion guidance model; the pre-trained general motion guidance model is used to provide a general standardized motion guidance prompt to the target object wearing the virtual reality device; Acquire virtual motion interaction data of the target object in the virtual motion scene; the virtual motion interaction data is motion interaction data obtained according to the action performed by the target object in the virtual motion scene; The pre-trained general motion guidance model is fine-tuned based on reinforcement learning according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object; the personalized motion guidance model is used to provide personalized motion guidance prompts to the target object according to the motion state of the target object.

2. The method according to claim 1, It is characterized in that The step of acquiring a motion knowledge template matching a virtual motion scene associated with the virtual reality device includes: Acquire a parameterized static modeling result of a virtual object associated with the virtual motion scene; the parameterized static modeling result is obtained by modeling according to static characteristics of a real object corresponding to the virtual object when the real object is in a stationary state; Acquire a parameterized dynamic modeling result of a virtual object associated with the virtual motion scene; the parameterized dynamic modeling result is obtained by modeling according to the dynamic characteristics of the real object when it is in motion; The motion knowledge template is constructed according to the parameterized static modeling result and the parameterized dynamic modeling result.

3. The method according to claim 2, It is characterized in that The obtaining of the parameterized static modeling result of the virtual object associated with the virtual motion scene comprises: Acquire physical property data corresponding to the real object, and construct a virtual model corresponding to the real object according to the physical property data; Assigning target physical properties and target model maps to the virtual model corresponding to the real object to obtain a target virtual model corresponding to the real object; the target physical properties are physical properties that match the real object; the target model map is a model map that matches the real object; The parametric static modeling result is obtained according to the target virtual model corresponding to the real object.

4. The method according to claim 2, It is characterized in that The obtaining of the parameterized dynamic modeling result of the virtual object associated with the virtual motion scene comprises: Acquire a dynamic equation when the real object is in motion; the dynamic equation is used to characterize the dynamic characteristics of the real object when it is in motion; The kinetic equation is subjected to parameterized modeling to obtain the parameterized dynamic modeling result.

5. The method according to claim 1, It is characterized in that The acquiring virtual motion interaction data of the target object in the virtual motion scene includes: The current virtual environment information of the virtual sports scene is obtained, and the current virtual environment information is input into the pre-trained general motion guidance model; the strategy function of the pre-trained general motion guidance model is used to generate motion guidance action prompts in response to the current virtual environment information; the motion guidance action prompts are used to instruct the target object to perform corresponding actions in the virtual sports scene to obtain the virtual sports interaction data.

6. The method according to claim 5, It is characterized in that The fine-tuning training based on reinforcement learning is performed on the pre-trained general motion guidance model according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object, including: Acquire virtual environment information of the virtual sports scene after responding to the virtual sports interaction data, obtain new virtual environment information, and acquire reward data obtained after the pre-trained general sports guidance model outputs the sports guidance action prompt; According to the new virtual environment information and the reward data, the strategy function of the pre-trained general motion guidance model is updated until the strategy function converges to obtain the personalized motion guidance model.

7. A sports intervention device based on virtual reality and deep reinforcement learning technology, It is characterized in that The device comprises: A first acquisition module is used to acquire a motion knowledge template matching a virtual motion scene associated with a virtual reality device; the virtual motion scene is obtained by performing digital twin modeling based on a real motion scene; the motion knowledge template includes standardized motion knowledge content; A training module, used for performing reinforcement learning training on the motion guidance model to be trained according to the motion knowledge template to obtain a pre-trained universal motion guidance model; the pre-trained universal motion guidance model is used to provide universal standardized motion guidance prompts to the target object wearing the virtual reality device; A second acquisition module is used to acquire virtual motion interaction data of the target object in the virtual motion scene; the virtual motion interaction data is motion interaction data obtained according to the action performed by the target object in the virtual motion scene; A fine-tuning module is used to perform reinforcement learning-based fine-tuning training on the pre-trained general motion guidance model according to the virtual motion interaction data to obtain a personalized motion guidance model for the target object; the personalized motion guidance model is used to provide personalized motion guidance prompts to the target object according to the motion state of the target object.

8. A sports intervention system based on virtual reality and deep reinforcement learning technology, comprising a memory and a processor, wherein the memory stores a computer program, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images