Simulation training method and system based on real scene and virtual immersion sports

Abstract

The invention discloses a physical simulation training method and system based on a real scene and virtual immersion, and the method comprises the steps: constructing a virtual scene corresponding to a to-be-trained sports item, and the virtual scene comprises a virtual image corresponding to a target user and a virtual training prop corresponding to the to-be-trained sports item; configuring a first lens and a second lens in the virtual scene; in the process that the target user is trained through a real training prop corresponding to the to-be-trained exercise item, acquiring action information of the target user and motion information of the real training prop; if it is determined that the real training prop does not leave the target user, updating the virtual image based on the action information, and displaying a picture collected by the first lens; and if it is determined that the real training prop leaves the target user, determining a motion track of the virtual training prop based on the motion information, controlling the virtual training prop to move in the virtual scene according to the motion track, and displaying a picture collected by the second lens. The exercise training effect can be improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, in particular to a simulation training method and system based on real scenes and virtual immersive sports. Background Art

[0002] Currently, more and more users like to try various sports to enrich their lives.

[0003] However, there are many sports that have high requirements for venues. Without professional venues, training cannot be carried out, which greatly reduces the user's training frequency for these sports and fails to achieve good training results. Summary of the invention

[0004] In view of the technical problems in the prior art, the present invention provides a simulation training method and system based on real scenes and virtual immersive sports.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] In a first aspect of an embodiment of the present invention, a simulation training method based on real scenes and virtual immersive sports is provided, the method comprising:

[0007] Constructing a virtual scene corresponding to the sport to be trained, wherein the virtual scene includes a virtual image corresponding to the target user and virtual training props corresponding to the sport to be trained;

[0008] A first lens and a second lens are arranged in the virtual scene, wherein the first lens is arranged around the virtual image and used to capture action images of the virtual image, and the second lens is arranged around the virtual training prop and moves with the virtual training prop and used to capture images of the virtual training prop in the direction of its movement;

[0009] Acquiring movement information of the target user and movement information of the real training props during the process of the target user training with the real training props corresponding to the sport to be trained;

[0010] Determining whether the real training prop leaves the target user according to the action information and the motion information;

[0011] If it is determined that the real training prop has not left the target user, updating the virtual image based on the action information and displaying the image captured by the first lens;

[0012] If it is determined that the real training prop leaves the target user, the motion trajectory of the virtual training prop is determined based on the motion information, and the virtual training prop is controlled to move in the virtual scene according to the motion trajectory, and the picture captured by the second lens is displayed.

[0013] Optionally, the action information includes image information, the motion information includes acceleration information, and determining whether the real training prop leaves the target user according to the action information and the motion information includes:

[0014] Determining whether the target user is in contact with the real training prop according to the image information;

[0015] If it is determined that the target user is in contact with the real training prop, then it is determined that the real training prop has not left the target user;

[0016] If it is determined that the target user is not in contact with the real training prop, determining whether the acceleration information is 0;

[0017] If it is determined that the acceleration information is not 0, it is determined that the real training prop leaves the target user.

[0018] Optionally, the training props include handheld props and flying props, and determining whether the target user is in contact with the real training props according to the image information includes:

[0019] If it is determined according to the image information that the handheld prop is in contact with the flying prop, then it is determined that the target user is in contact with the real training prop;

[0020] If it is determined according to the image information that the handheld prop is not in contact with the flying prop, it is determined that the target user is not in contact with the real training prop.

[0021] Optionally, controlling the virtual training prop to move in a virtual scene according to the motion trajectory and displaying the image captured by the second lens includes:

[0022] In the process of controlling the virtual training prop to move in the virtual scene according to the motion trajectory, obtaining a first distance between the virtual training prop and the ground of the virtual scene;

[0023] The second distance between the second lens and the virtual training prop is adjusted according to the first distance.

[0024] Optionally, the first distance is positively correlated with the second distance.

[0025] Optionally, determining the motion trajectory of the virtual training prop based on the motion information includes:

[0026] Acquire a target motion trajectory prediction model corresponding to the virtual training prop from a plurality of preset motion trajectory prediction models;

[0027] The motion information is input into the target motion trajectory prediction model and the motion trajectory output by the target motion trajectory prediction model is obtained.

[0028] Optionally, the method further comprises:

[0029] Generate multiple frames of action images of the virtual image based on the images captured by the first lens;

[0030] For each frame of motion image in the plurality of frames of motion images, identifying a target action corresponding to the motion image;

[0031] Acquire a standard action image corresponding to the target action, and compare the motion image with the standard action image to obtain a first comparison result;

[0032] First guidance information is output based on the first comparison result, where the first guidance information is used to guide actions of the target user.

[0033] Optionally, the method further comprises:

[0034] Obtaining a standard motion trajectory corresponding to the virtual training prop;

[0035] Comparing the motion trajectory with the standard motion trajectory to obtain a second comparison result;

[0036] Second guidance information is output based on the second comparison result, where the second guidance information is used to guide actions of the target user.

[0037] Optionally, before constructing the virtual scene corresponding to the sports item to be trained, the method further includes:

[0038] In response to an item selection instruction, a sports item to be trained is determined from a plurality of preset training sports items, wherein each of the plurality of training sports items corresponds to a training prop, and the plurality of training sports items include football, basketball, shot put, baseball, golf, shooting, and frisbee.

[0039] In a second aspect of an embodiment of the present invention, a simulation training system based on real scenes and virtual immersive sports is provided, the system comprising:

[0040] A construction module, used to construct a virtual scene corresponding to the sport to be trained, wherein the virtual scene includes a virtual image corresponding to the target user and virtual training props corresponding to the sport to be trained;

[0041] A configuration module, configured to configure a first lens and a second lens in the virtual scene, wherein the first lens is arranged around the virtual image and used to capture action images of the virtual image, and the second lens is arranged around the virtual training prop and moves with the virtual training prop and used to capture images of the virtual training prop in its movement direction;

[0042] An acquisition module, used for acquiring the action information of the target user and the motion information of the real training props during the process of the target user training with the real training props corresponding to the sport to be trained;

[0043] A determination module, used to determine whether the real training prop leaves the target user according to the action information and the motion information;

[0044] A first display module, configured to update the virtual image based on the action information and display the image captured by the first lens if it is determined that the real training prop has not left the target user;

[0045] The second display module is used to determine the motion trajectory of the virtual training prop based on the motion information if it is determined that the real training prop leaves the target user, control the movement of the virtual training prop in the virtual scene according to the motion trajectory, and display the picture captured by the second lens.

[0046] The present invention provides a simulation training method and system based on real-life scenarios and virtual immersive sports. Compared with the prior art, it has the following beneficial effects:

[0047] The solution provided in this embodiment is to construct a virtual scene corresponding to the sports item to be trained, wherein the virtual scene includes a virtual image corresponding to the target user and a virtual training prop corresponding to the sports item to be trained; then, a first lens and a second lens are configured in the virtual scene, wherein the first lens is arranged around the virtual image to capture the action picture of the virtual image, and the second lens is arranged around the virtual training prop and moves with the virtual training prop to capture the picture of the virtual training prop in its movement direction; in the process of the target user training with the real training prop corresponding to the sports item to be trained, the action information of the target user and the motion information of the real training prop are obtained; then, based on the action information and the motion information, it is determined whether the real training prop leaves the target user; if it is determined that the real training prop does not leave the target user, the virtual image is updated based on the action information, and the picture captured by the first lens is displayed; if it is determined that the real training prop leaves the target user, the motion trajectory of the virtual training prop is determined based on the motion information, and the virtual training prop is controlled to move in the virtual scene according to the motion trajectory, and the picture captured by the second lens is displayed. Thus, a virtual scene of the sports item to be trained can be provided for the user to perform sports training, and the virtual scene is not limited by the venue, so that the training frequency can be greatly increased. In addition, by setting a first lens for capturing the action picture of the virtual image and a second lens for capturing the picture of the virtual training prop in its movement direction in the virtual scene, the first lens and the second lens are automatically switched when the target user makes a corresponding movement action, so as to more realistically simulate the perspective of the target user when performing the sports item to be trained in reality in the displayed virtual scene, thereby achieving an immersive experience effect, thereby improving the training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a schematic diagram of an application scenario of a simulation training method based on a real scene and virtual immersive sports according to an exemplary embodiment;

[0049] Figure 2 is a flow chart showing a simulation training method based on real-life scenarios and virtual immersive sports according to an exemplary embodiment;

[0050] Figure 3 is a schematic diagram of a switching process between a first lens acquisition picture and a second lens acquisition picture according to an exemplary embodiment;

[0051] Figure 4 is a schematic diagram showing the distances between the virtual training prop and the ground and the second camera during the flight process according to an exemplary embodiment;

[0052] Figure 5It is a principle block diagram of a simulation training system based on real scenes and virtual immersive sports according to an exemplary embodiment. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] It can be understood that in the specific implementation of the present application, data related to action information, image information, etc. is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0055] Figure 1 is a schematic diagram of an application environment of a simulation training method based on a real scene and virtual immersive sports according to an exemplary embodiment. Figure 1 As shown, the application environment may include a processor, an action collector, a motion collector, a display, etc., wherein the processor is communicatively connected with the action collector, the motion collector, and the display respectively.

[0056] The motion collector can be used to collect the motion information of the target user, wherein the motion information represents the motion state of the target user, and the motion information can include posture information (such as key points of bones, etc.). The motion collector can include posture sensors (such as accelerometers, gyroscopes, magnetometers, pressure sensors, ultrasonic sensors, etc.) worn on key parts of the target user, image collectors (such as high-speed cameras, depth cameras, three-dimensional point cloud cameras, etc.) set around the target user for collecting the target user, infrared sensors set around the target user, etc.

[0057] The motion collector can collect motion information of the real training props corresponding to the sports item to be trained, and the motion information represents the motion state of the real training props. The motion collector can be a motion sensor (such as an acceleration sensor, a gyroscope, etc.) set on the real training props. Optionally, the real training props include but are not limited to: basketball, football, shot put, golf, badminton, tennis, frisbee, baseball, etc.

[0058] The processor is used to construct a virtual scene corresponding to the sports item to be trained, and display the virtual scene by controlling the display, and update the virtual scene according to the sports information collected by the sports collector and the action information collected by the action collector. Optionally, the display includes but is not limited to: a display screen, a projector, AR glasses (augmented reality glasses), etc.

[0059] Figure 2 is a flowchart of a simulation training method based on real scenes and virtual immersive sports according to an exemplary embodiment. The sports virtual training method can be applied to Figure 1 Processors such as Figure 2 As shown, the above method comprises the following steps:

[0060] 101. Construct a virtual scene corresponding to the sport to be trained, wherein the virtual scene includes a virtual image corresponding to the target user and virtual training props corresponding to the sport to be trained;

[0061] Among them, the virtual scene corresponding to the sports to be trained can be constructed based on the three-dimensional image information and geographic data information of the real training scene corresponding to the sports to be trained. The specific construction process can refer to the construction process of the three-dimensional map, so it is not repeated here. For example, if the sports to be trained is basketball, a virtual scene of a basketball court can be constructed, and the virtual scene can include a virtual basketball stand and a virtual basket, and the corresponding virtual training prop is a virtual basketball. For another example, if the sports to be trained is golf, a virtual scene of a golf course can be constructed, and the virtual scene can include a virtual sky, a virtual grass, a virtual golf hole, etc.

[0062] In some implementations, before step 101, the method may further include:

[0063] In response to an item selection instruction, a sports item to be trained is determined from a plurality of preset training sports items, wherein each of the plurality of training sports items corresponds to a training prop, and the plurality of training sports items include football, basketball, shot put, baseball, golf, shooting, and frisbee.

[0064] Exemplarily, the application environment may also include an interactive device connected to the processor, which may be a touch screen, a voice acquisition device, etc. The target user may input an item selection instruction to the processor through the interactive device to enable the processor, and the item selection instruction is used to select the sports item to be trained from multiple preset training sports items and start the program of the sports item to be trained.

[0065] 102. A first lens and a second lens are configured in the virtual scene. The first lens is arranged around the virtual image to capture action images of the virtual image. The second lens is arranged around the virtual training prop and moves with the virtual training prop to capture images of the virtual training prop in the direction of its movement.

[0066] Optionally, the first lens may be arranged in at least one direction of the front, rear, left, right, and top of the virtual image to better detect the movement of the virtual image.

[0067] It is understandable that the images captured by the second lens include, in addition to the virtual training props, images of the virtual scene under the shooting angle of the second lens during the movement of the virtual training props, so as to enhance the user's real experience.

[0068] 103. When the target user is training with the real training props corresponding to the sport to be trained, obtain the motion information of the target user and the motion information of the real training props.

[0069] Exemplarily, the processor may obtain the motion information of the target user through the motion sensor, and obtain the motion information of the real training prop through the motion sensor.

[0070] 104. Determine whether the real training prop leaves the target user according to the action information and the movement information.

[0071] In some implementations, the action information includes image information, and the motion information includes acceleration information. In step 104, the specific implementation of determining whether the real training prop has left the target user based on the action information and the motion information may include:

[0072] Determine whether the target user is in contact with the real training prop based on the image information.

[0073] For example, a first image area corresponding to the target user and a second image area corresponding to the real training prop can be identified from the image information. Then, it is detected whether the first image area overlaps with the second image area. If the first image area overlaps with the second image area, it can be determined that the target user is in contact with the real training prop. Otherwise, it is determined that the target user is not in contact with the real training prop.

[0074] Exemplarily, the image information may include images taken at multiple shooting angles, all of which are facing the target user. If it is detected that the first image area and the second image area in any one of the images taken at the multiple shooting angles do not overlap, it can be determined that the target user is not in contact with the real training props, thereby accurately detecting whether the target user is in contact with the real training props.

[0075] If it is determined that the target user is in contact with the real training prop, it is determined that the real training prop has not left the target user.

[0076] If it is determined that the target user is not in contact with the real training prop, it is determined whether the acceleration information is zero.

[0077] If it is determined that the acceleration information is not 0, it is determined that the real training prop leaves the target user.

[0078] Considering that when it is determined that the target user has no contact with the real training props, but the acceleration information of the real training props is 0, there is a situation where the real training props may not be used by the target user. In this embodiment, the image information is combined with the acceleration information to detect whether the real training props leave the target user, thereby effectively improving the detection accuracy.

[0079] In some implementations, the training props include handheld props and flying props, and the specific implementation of the step of "determining whether the target user is in contact with the real training props according to the image information" may include:

[0080] If it is determined according to the image information that the handheld prop is in contact with the flying prop, then it is determined that the target user is in contact with the real training prop.

[0081] If it is determined according to the image information that the handheld prop is not in contact with the flying prop, it is determined that the target user is not in contact with the real training prop.

[0082] For example, the handheld prop is a badminton racket, and the flying prop is a badminton. For another example, the handheld prop is a bat, and the flying prop is a baseball.

[0083] 105. If it is determined that the real training prop has not left the target user, the virtual image is updated based on the action information, and the image captured by the first lens is displayed.

[0084] For example, Figure 3As shown, for example, in a virtual scene corresponding to a basketball event, when the virtual image follows the target user to make a shooting action, the display may first display the picture sense captured by the first lens, that is, the picture of the shooting action made by the virtual image.

[0085] 106. If it is determined that the real training prop leaves the target user, a motion trajectory of the virtual training prop is determined based on the motion information, and the virtual training prop is controlled to move in the virtual scene according to the motion trajectory, and the image captured by the second lens is displayed.

[0086] Using the above example, please refer to Figure 3 When it is determined that the real training prop leaves the target user, the display may first display the picture sense captured by the second lens, that is, the real-time picture of the virtual basketball flying toward the virtual basket.

[0087] according to Figure 3 It can be seen that when the target user is making a shooting action, the display shows the picture captured by the first lens to vividly show the target user's shooting action. After the target user makes the shooting action, the display can automatically and smoothly switch to the picture captured by the second lens to show the flight of the virtual basketball, so that the user can experience the real shooting process.

[0088] In some implementations, in step 106, based on the motion information, a specific implementation of determining the motion trajectory of the virtual training prop may include:

[0089] A target motion trajectory prediction model corresponding to the above-mentioned virtual training prop is obtained from a plurality of preset motion trajectory prediction models. The motion trajectory prediction model is a pre-trained machine learning model, which can output a corresponding motion trajectory according to the input motion information. It is understandable that, since the motion trajectories corresponding to different training props in the display are different (for example, the motion trajectory of a frisbee is different from that of a golf ball), different motion trajectory prediction models need to be used for different virtual training props to improve the prediction accuracy of the motion trajectory.

[0090] The motion information is input into the target motion trajectory prediction model and the motion trajectory output by the target motion trajectory prediction model is obtained.

[0091] In some implementations, in step 106, controlling the virtual training prop to move in the virtual scene according to the motion trajectory and displaying the image captured by the second lens may include:

[0092] In the process of controlling the virtual training prop to move in the virtual scene according to the motion trajectory, a first distance between the virtual training prop and the ground of the virtual scene is obtained.

[0093] The second distance between the second lens and the virtual training prop is adjusted according to the first distance.

[0094] Optionally, the first distance may be positively correlated with the second distance.

[0095] For example, Figure 4 As shown, taking the virtual training prop as a virtual golf ball as an example, when the first distance is L1, the second distance is L2, and when the first distance is L3, the second distance is L4, wherein the first distance L1 is smaller than the first distance L3, and the second distance L2 is smaller than the second distance L4.

[0096] Since the larger the first distance is, the farther the virtual golf ball is from the ground of the virtual scene, indicating that the current flight altitude of the virtual golf ball is relatively high, the second distance between the second lens and the virtual training prop is controlled to be larger, thereby providing the target user with a wider viewing angle, so that the target user can observe the flight trajectory of the virtual training prop. If the smaller the first distance is, the closer the virtual golf ball is to the ground of the virtual scene, indicating that the current flight altitude of the virtual golf ball is relatively low, the second distance between the second lens and the virtual training prop is controlled to be smaller, thereby providing the target user with a more detailed viewing angle, so as to observe the flight angle of the virtual training prop, and further provide the target user with more valuable training guidance.

[0097] In some implementations, after step 106, the method may further include:

[0098] Based on the images captured by the first lens, multiple frames of action images of the virtual image are generated.

[0099] For each frame of motion image in the plurality of frames of motion images, a target action corresponding to the motion image is identified.

[0100] Optionally, the motion image can be recognized by a pre-trained motion prediction model to obtain the target motion.

[0101] A standard action image corresponding to the target action is obtained, and the motion image is compared with the standard action image to obtain a first comparison result.

[0102] First guidance information is output based on the first comparison result, where the first guidance information is used to guide actions of the target user.

[0103] Exemplarily, for example, based on a frame of motion image, it is identified that the target action of the target user is a shooting action, and a standard action image corresponding to the shooting action can be obtained, and then the standard action image is compared with the motion image to obtain a first comparison result. Then, based on the above first comparison result, the first guidance information is output, for example, the elbow is too high, the knee is bent too much, etc.

[0104] In some implementations, after step 106, the method may further include:

[0105] Obtain the standard motion trajectory corresponding to the above virtual training props.

[0106] Comparing the motion trajectory with the standard motion trajectory to obtain a second comparison result;

[0107] Second guidance information is output based on the second comparison result, where the second guidance information is used to guide actions of the target user.

[0108] For example, if the virtual training prop is a golf project, the second guidance information may be how much the hitting angle deviates, whether the hitting force is too strong, etc.

[0109] It can be seen that in this embodiment, by constructing a virtual scene corresponding to the sports item to be trained, the virtual scene includes a virtual image corresponding to the target user and a virtual training prop corresponding to the sports item to be trained; then, a first lens and a second lens are configured in the virtual scene, the first lens is set around the virtual image to collect action pictures of the virtual image, and the second lens is set around the virtual training prop and moves with the virtual training prop to collect pictures of the virtual training prop in its movement direction; in the process of the target user training with the real training prop corresponding to the sports item to be trained, the action information of the target user and the motion information of the real training prop are obtained; then, according to the action information and the motion information, it is determined whether the real training prop leaves the target user; if it is determined that the real training prop does not leave the target user, the virtual image is updated based on the action information, and the picture captured by the first lens is displayed; if it is determined that the real training prop leaves the target user, the motion trajectory of the virtual training prop is determined based on the motion information, and the virtual training prop is controlled to move in the virtual scene according to the motion trajectory, and the picture captured by the second lens is displayed. Thus, a virtual scene of the sports item to be trained can be provided for the user to perform sports training, and the virtual scene is not limited by the venue, so that the training frequency can be greatly increased. In addition, by setting a first lens for capturing the action picture of the virtual image and a second lens for capturing the picture of the virtual training prop in its movement direction in the virtual scene, the first lens and the second lens are automatically switched when the target user makes a corresponding movement action, so as to more realistically simulate the perspective of the target user when performing the sports item to be trained in reality in the displayed virtual scene, thereby achieving an immersive experience effect, thereby improving the training effect.

[0110] In this embodiment, if Figure 5 As shown, a simulation training system based on real scenes and virtual immersive sports is also provided, and the sports virtual training system 300 includes:

[0111] A construction module 310 is used to construct a virtual scene corresponding to the sport to be trained, wherein the virtual scene includes a virtual image corresponding to the target user and virtual training props corresponding to the sport to be trained;

[0112] The configuration module 320 is used to configure a first lens and a second lens in the virtual scene, wherein the first lens is arranged around the virtual image to capture action images of the virtual image, and the second lens is arranged around the virtual training prop and moves with the virtual training prop to capture images of the virtual training prop in the direction of its movement;

[0113] An acquisition module 330 is used to acquire the motion information of the target user and the motion information of the real training props during the process of the target user training with the real training props corresponding to the sport to be trained;

[0114] A determination module 340 is used to determine whether the real training prop leaves the target user according to the action information and the movement information;

[0115] A first display module 350 is used to update the virtual image based on the action information and display the image captured by the first lens if it is determined that the real training prop has not left the target user;

[0116] The second display module 360 ​​is used to determine the motion trajectory of the virtual training prop based on the motion information if it is determined that the real training prop leaves the target user, and control the movement of the virtual training prop in the virtual scene according to the motion trajectory, and display the picture captured by the second lens.

[0117] In some implementations, the action information includes image information, the motion information includes acceleration information, and the determination module 340 is specifically configured to:

[0118] Determining whether the target user is in contact with the real training prop according to the image information;

[0119] If it is determined that the target user is in contact with the real training prop, it is determined that the real training prop has not left the target user;

[0120] If it is determined that the target user is not in contact with the real training prop, determining whether the acceleration information is 0;

[0121] If it is determined that the acceleration information is not 0, it is determined that the real training prop leaves the target user.

[0122] In some embodiments, the training props include handheld props and flying props, and the determination module 340 is further used to:

[0123] If it is determined according to the image information that the handheld prop is in contact with the flying prop, then it is determined that the target user is in contact with the real training prop;

[0124] If it is determined according to the image information that the handheld prop is not in contact with the flying prop, it is determined that the target user is not in contact with the real training prop.

[0125] In some implementations, the second display screen is specifically used for:

[0126] In the process of controlling the virtual training prop to move in the virtual scene according to the motion trajectory, obtaining a first distance between the virtual training prop and the ground of the virtual scene;

[0127] The second distance between the second lens and the virtual training prop is adjusted according to the first distance.

[0128] In some embodiments, the first distance is positively correlated with the second distance.

[0129] In some implementations, the second display screen is further used for:

[0130] Obtaining a target motion trajectory prediction model corresponding to the virtual training prop from a plurality of preset motion trajectory prediction models;

[0131] The motion information is input into the target motion trajectory prediction model and the motion trajectory output by the target motion trajectory prediction model is obtained.

[0132] In some embodiments, the system further comprises a first guidance module, the first guidance module being configured to:

[0133] Generate multiple frames of action images of the virtual image based on the images captured by the first lens;

[0134] For each frame of motion image in the plurality of frames of motion images, identifying a target action corresponding to the motion image;

[0135] Acquire a standard action image corresponding to the target action, and compare the motion image with the standard action image to obtain a first comparison result;

[0136] First guidance information is output based on the first comparison result, where the first guidance information is used to guide actions of the target user.

[0137] In some embodiments, the system further comprises a second guidance module, the second guidance module being configured to:

[0138] Obtaining the standard motion trajectory corresponding to the above virtual training props;

[0139] Comparing the motion trajectory with the standard motion trajectory to obtain a second comparison result;

[0140] Second guidance information is output based on the second comparison result, where the second guidance information is used to guide actions of the target user.

[0141] In some embodiments, the system further comprises a selection module, wherein the selection module is used to:

[0142] In response to an item selection instruction, a sports item to be trained is determined from a plurality of preset training sports items, wherein each of the plurality of training sports items corresponds to a training prop, and the plurality of training sports items include football, basketball, shot put, baseball, golf, shooting, and frisbee.

[0143] Based on the same aspect, the present disclosure also provides an electronic device, including:

[0144] a memory having a computer program stored thereon;

[0145] A processor is used to execute the above-mentioned computer program in the above-mentioned memory to implement the steps of any one of the above-mentioned simulation training methods based on real scenes and virtual immersive sports in the above-mentioned embodiments.

[0146] Based on the above ideal embodiments of this application, the relevant staff can make various changes and modifications without departing from the technical concept of this application through the above description. The technical scope of this application is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

[0147] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A simulation training method based on real-life scenarios and virtual immersive sports, characterized in that: include: Constructing a virtual scene corresponding to the sport to be trained, wherein the virtual scene includes a virtual image corresponding to the target user and virtual training props corresponding to the sport to be trained; A first lens and a second lens are arranged in the virtual scene, wherein the first lens is arranged around the virtual image and used to capture action images of the virtual image, and the second lens is arranged around the virtual training prop and moves with the virtual training prop and used to capture images of the virtual training prop in the direction of its movement; Acquiring movement information of the target user and movement information of the real training props during the process of the target user training with the real training props corresponding to the sport to be trained; Determining whether the real training prop leaves the target user according to the action information and the motion information; If it is determined that the real training prop has not left the target user, updating the virtual image based on the action information and displaying the image captured by the first lens; If it is determined that the real training prop leaves the target user, the motion trajectory of the virtual training prop is determined based on the motion information, and the virtual training prop is controlled to move in the virtual scene according to the motion trajectory, and the picture captured by the second lens is displayed.

2. The simulation training method based on real-life scenarios and virtual immersive sports according to claim 1, characterized in that: The action information includes image information, the motion information includes acceleration information, and determining whether the real training prop leaves the target user according to the action information and the motion information includes: Determining whether the target user is in contact with the real training prop according to the image information; If it is determined that the target user is in contact with the real training prop, then it is determined that the real training prop has not left the target user; If it is determined that the target user is not in contact with the real training prop, determining whether the acceleration information is 0; If it is determined that the acceleration information is not 0, it is determined that the real training prop leaves the target user.

3. The simulation training method based on real-life scenarios and virtual immersive sports according to claim 2, characterized in that: The training props include handheld props and flying props, and determining whether the target user is in contact with the real training props according to the image information includes: If it is determined according to the image information that the handheld prop is in contact with the flying prop, then it is determined that the target user is in contact with the real training prop; If it is determined according to the image information that the handheld prop is not in contact with the flying prop, it is determined that the target user is not in contact with the real training prop.

4. The simulation training method based on real-life scenarios and virtual immersive sports according to claim 1, characterized in that: The step of controlling the virtual training prop to move in the virtual scene according to the motion trajectory and displaying the image captured by the second lens includes: In the process of controlling the virtual training prop to move in the virtual scene according to the motion trajectory, obtaining a first distance between the virtual training prop and the ground of the virtual scene; The second distance between the second lens and the virtual training prop is adjusted according to the first distance.

5. The simulation training method based on real scene and virtual immersive sports according to claim 4, characterized in that: The first distance is positively correlated with the second distance.

6. The simulation training method based on real scene and virtual immersive sports according to claim 1, characterized in that: The step of determining the motion trajectory of the virtual training prop based on the motion information includes: Acquire a target motion trajectory prediction model corresponding to the virtual training prop from a plurality of preset motion trajectory prediction models; The motion information is input into the target motion trajectory prediction model and the motion trajectory output by the target motion trajectory prediction model is obtained.

7. The simulation training method based on real scene and virtual immersive sports according to claim 1, characterized in that: The method further comprises: Generate multiple frames of action images of the virtual image based on the images captured by the first lens; For each frame of motion image in the plurality of frames of motion images, identifying a target action corresponding to the motion image; Acquire a standard action image corresponding to the target action, and compare the motion image with the standard action image to obtain a first comparison result; First guidance information is output based on the first comparison result, where the first guidance information is used to guide actions of the target user.

8. The simulation training method based on real scene and virtual immersive sports according to claim 1, characterized in that: The method further comprises: Obtaining a standard motion trajectory corresponding to the virtual training prop; Comparing the motion trajectory with the standard motion trajectory to obtain a second comparison result; Second guidance information is output based on the second comparison result, where the second guidance information is used to guide actions of the target user.

9. The simulation training method based on real scene and virtual immersive sports according to any one of claims 1 to 8, characterized in that: Before constructing the virtual scene corresponding to the sports item to be trained, the method further includes: In response to an item selection instruction, a sports item to be trained is determined from a plurality of preset training sports items, wherein each of the plurality of training sports items corresponds to a training prop, and the plurality of training sports items include football, basketball, shot put, baseball, golf, shooting, and frisbee.

10. A simulation training system based on real-life scenes and virtual immersive sports, characterized in that: include: A construction module, used to construct a virtual scene corresponding to the sport to be trained, wherein the virtual scene includes a virtual image corresponding to the target user and virtual training props corresponding to the sport to be trained; A configuration module, configured to configure a first lens and a second lens in the virtual scene, wherein the first lens is arranged around the virtual image and used to capture action images of the virtual image, and the second lens is arranged around the virtual training prop and moves with the virtual training prop and used to capture images of the virtual training prop in its movement direction; An acquisition module, used for acquiring the action information of the target user and the motion information of the real training props during the process of the target user training with the real training props corresponding to the sport to be trained; A determination module, used to determine whether the real training prop leaves the target user according to the action information and the motion information; A first display module, configured to update the virtual image based on the action information and display the image captured by the first lens if it is determined that the real training prop has not left the target user; The second display module is used to determine the motion trajectory of the virtual training prop based on the motion information if it is determined that the real training prop leaves the target user, control the movement of the virtual training prop in the virtual scene according to the motion trajectory, and display the picture captured by the second lens.

Images